Controlling concurrency in Go using semaphores

Go provides a very easy way to start up threads(goroutines 1) so you can fire off multiple asynchronous tasks. Starting up goroutines is very simple and they are also very light weight.

If you use goroutines to start multiple asynchronous tasks, you might run into a problem where you have many goroutines running at the same time. This can be a problem if, say, you’re making web calls in each goroutine and all these calls put a lot of load on the service you’re calling.

To overcome this problem you have to control the maximum number of goroutines that can run at a given time. The following code allows us to do exactly that.

c.semaphore = make(chan int, THREAD_LIMIT)
//Fill up the semaphore
for i := 0; i < THREAD_LIMIT; i++ {
  c.semaphore <- 1

<-c.semaphore   //Pop from the semaphore if available
go task()

func task() {
  c.semaphore <-1  //Push back into the semaphore once done


We’re basically creating a semaphore here with THREAD_LIMIT ‘tokens’ in it. Whenever you need to start a task, you check to see if there are any available tokens over the channel.



This is a blocking call and will pop a value off the channel whenever it’s available. Basically this process gets a token to begin execution.

Once the task is done, it pushes the token back on to the channel for other listeners.

c.semaphore <-1


We don’t care what the value on the channel is, we’re just pushing 1s to donate a single token.

This simple code ensures that there are at most THREAD_LIMIT number of goroutines running. And since they all listen to the same semaphore channel, they can all basically communicate with each other and coordinate..

PHP asynchronous tasks with beanstalkd

I was running a very crude task queue on QilaPages using Redis to queue up tasks and running cron jobs to fire those tasks periodically. This arrangement was very simple to maintain but was not robust enough to handle complex situations, and I didn’t want to spend a lot of time on it.

I did a little research and found beanstalkd, which is a very simple to use task queue. A really good (and fast) beanstalkd client library I found is pheanstalk.

After you download and install beanstalkd, simply start the service:

beanstalkd -l -p 11300

Next, you can queue up tasks:

$beanstalk_settings = array( 'host' => '', 'port' => '11300');

$task = array(
    'function' => 'simple_task',
    'params' => array('param1', 'param2')

$phean = new Pheanstalk($beanstalk_settings['host']);

beanstalkd stores tasks as strings. When a task in invoked, beanstalkd would simply return the string (or payload) that was put in and your workers would handle task based on what you pass into the payload. In our case, the payload is the JSON object in $task.

On the worker side, I simple decode the string to get the name of function to call and the parameters to pass:

try {
    while($job = $pheanstalk
        ->reserve(0)) {

        $task = json_decode($job->getData());
        call_user_func_array($task['function'], $task['params']);
}catch(Exception $e)
    echo $e->getMessage();

The above code will call

simple_task('param1', 'param2');