Threading in Perl

Threading in Perl may seem daunting at first, but it's actually easier than it looks. In some ways, threading is more straight forward than forking. Communication between the threads and a parent process is much more straight forward in my opinion than communication between child and parent processes.

Here I've written a little sample function that will take a number of threads to launch at a time, a reference to a function and an array of hostnames. Then it will launch one thread for each hostname passing the hostname to the function, until the limit has been reached. It then waits for each thread to become joinable (finishes) and then launches more threads till the limit is reached again. When all threads are finished, it returns the output form all threads as a hash keyed by the hostnames.


######################################
# http://greg-techblog.blogspot.com
# Spawns threads for a list of hosts running a function
# you pass to it. Passes host to function as argument.
# args:
# needs numbers of threads to launch at a time
# ref to sub to run
# ref array of hosts or instances (will be passed to sub as arg)
# returns:
# a hash of returns from sub keyed by host or instance name
######################################
use threads;
my $debug;
sub run_threaded {
my ($threads,$func_to_thread, $hosts) = @_;
my (%stdout, %threads);
my $n = 0;

# This makes sure we don't start more than $threads threads at a time
my $m = $threads-1 > $#{$hosts} ? $#{$hosts} : $threads-1;
while (1) {
last if $n >= $#{$hosts}+1;
foreach my $host (@{$hosts}[$n..$m]) {
print "Launching thread for $host $n-$m\n" if $debug;
unless ( $threads{$host} = new threads $func_to_thread, $host) {
print "Error on $host\n";
}
}
map { $stdout{$_} = $threads{$_}->join if $threads{$_}; } @{$hosts}[$n..$m];
# work out the next range of instances to work on
$n = $m == 0 ? 1 : $m+1;
$m = $n+$threads-1 < $#{$hosts} ? $n+$threads-1 : $#{$hosts};
print "$n-$m\n\n" if $debug;
}
return %stdout;
}


To use this, simply do something like this;

use Data::Dumper;
my @hosts = qw{ host1 host2 host3 host4 };
sub myFunc { my $hostname = shift; return `ping $hostname` ; }

# Launch two threads at a time that ping hosts
my %stdout = run_threaded( 2, \&myFunc, \@hosts);
print Dumper \%stdout;