Comments on: Phusion Passenger 2.0.4 released; 37signals’s Ta-da List now using Passenger

By: - Passenger 2.0.4 Released: Now With Global Queuing and Background Process Support -325i.org

Sun, 14 Dec 2008 14:42:55 +0000

[...] Passenger 2.0.4 has just been released – a point release in name, but like most Passenger releases, another feature comes to the fore. The big feature this time around is global queueing. A feature sponsored by 37 Signals, global queueing effectively changes Passenger from deploying requests to back-end processes with their own process queues to a global queue scheme (where processes take requests off the queue). Income requests will now be balanced across the back-end processes so that no processes are running idle while others are struggling. The result is a smoother performance profile and higher reliability overall. [...]

By: Marius

Marius — Sat, 13 Dec 2008 17:02:27 +0000

Thank you hongli!

By: hongli

hongli — Fri, 12 Dec 2008 16:41:47 +0000

@Marius: We suspect that the 5% penalty is caused by context switching overhead and waiting. It’s like standing in front of a line in the supermarket. Compare global queuing with having multiple counters and a single door which gives access to all the counters. Everybody is queued behind this door. Whenever a cashier is done, he opens the door and calls “next person please!”. There’s a little delay involved in opening the door, communicating with the next person, and walking back to the counter.
There is no such delay if there is one door for each counter (= global queuing off). But there’s a wall between each counter, and the door can only be opened from the outside, so once you step into a door you cannot go to another counter even if the other counter doesn’t have a queue.

By: Marius

Marius — Fri, 12 Dec 2008 16:08:00 +0000

Ok, so there is something like a 5% performance penalty for using global queueing. Why is that? Is that explained somewhere?

By: Ruby On Rails With Mod Rails

Ruby On Rails With Mod Rails — Fri, 05 Dec 2008 07:13:25 +0000

We are currently on 2.0.4 it was an easy upgrade. BTW great work guys.

By: hongli

hongli — Thu, 04 Dec 2008 13:05:22 +0000

Yes. It’s documented in the manual.

By: Marius

Marius — Thu, 04 Dec 2008 11:55:56 +0000

Is there any downside to global queueing?

By: hongli

hongli — Thu, 04 Dec 2008 09:43:03 +0000

@Matthijs, Matthew: We’ve already identified and fixed the problem. It only occurs when using global queuing in combination with the worker MPM. The bug fix is already in git and we’ll release a new version soon.

By: Mathijs Kwik

Mathijs Kwik — Wed, 03 Dec 2008 23:42:34 +0000

I can confirm Matthew Higgins’ report.
When turning Global Queueing on, apache crashes.
It can be triggered in under 10 seconds when using passenger-stress-test.

What seems to happen is that requests land up in the global queue while some ruby processes don’t exit (or don’t notify passenger or something). Apache then at some point won’t accept more connections, hence the maxclients error in the log.

Killing apache the normal way doesn’t work very well. It needs a killall on my system, and even then it takes 15 sec. the logs show that apache is waiting for some child processes to shut down.

This happens on 2 of our production systems. They both run ubuntu server 8.04 on 64-bit.
I turned the global queue back off as soon as I noted the error, but I would like to use this feature.

If I can help by providing some apache logs I will gladly help.

For the rest… keep up the great work!

By: hongli

hongli — Wed, 03 Dec 2008 12:17:37 +0000

@Chris: We don’t get paid for implementing Solaris support.

That said, people have contributed Solaris support and it’s in the development version.