Quick HOWTO: Run SSH sessions in Parallel


 You often used "for loop" in shell script to gather an information from large number of your Linux / Unix infrastructure servers. However if your server count is more, it may take lot of time to finish the task. Do you think it will be good if the ssh sessions to remote servers run in parallel? Since the modern day servers having multi-core CPU's we can utilize the real parallel and multi processing of them. 

xargs is your friend here to accomplish your this.

Here is the example to run the SSH sessions in parallel:

xrags -a -n1 -P50 -I{} sh -c "ssh -q {} 'df -Ph /var' "


In this example,

-a --  Takes the input from the file instead of STDIN

-P -- number to processes to execute in parallel. In our example we are starting 50 processes in parallel.

-I -- Used to specify a name to the variable passing to the command we are executing. In our example, we have used {}. For example, You can specify like -I"server" instead of I{}.

Another example:

xrags -a -n1 -P50 -I"server" sh -c "scp server:/tmp/"



One disadvantage in xargs is the output will not be in a order. you need to make your script writes to a log with clear host identification to gather a report from multiple hosts.

Not only SSH and SCP, you can run any process / task which you want to start and run in parallel. Alternatively you can download and use "parallel" tool which is slightly better than xargs.