Documentation update: spawning methods explained

By Hongli Lai February 27th, 2009

We are delighted to see that the Phusion Passenger community sponsorship campaign was a success. Our thanks to all who have contributed!

We are very close to releasing 2.1; the documentation is coming along quite nicely. One of the areas of Phusion Passenger that is not widely understood, is the different spawning methods that it supports, i.e. smart spawning vs conservative spawning. We’ve taken our time to document this in detail, by explaining how it works, the pros and cons of smart vs conservative spawning, possible gotchas and how to deal with them.

If you’re interested, please review this and post any comments that you may have. Thanks!

Click here to lend your support to: Phusion Passenger first community sponsorship and make a donation at www.pledgie.com !

Hongli Lai Ninh Bui

Phusion Passenger | Comments (0)

Phusion Passenger Pledgie Paypal page, now in English again

By Hongli Lai February 18th, 2009

The Phusion Passenger community sponsorship campaign has progressed very quickly. It’s been a few days and we’re already at 27%! Thanks for your support!

However, many people have reported that the Pledgie Paypal page is in Dutch. We’ve contacted Pledgie, and it turns out to be a bug in Paypal. The nice guys at Pledgie have implemented a workaround for this problem, so everything should be in English again. Our apologies for the inconvenience!

Click here to lend your support to: Phusion Passenger first community sponsorship and make a donation at www.pledgie.com !

Phusion Passenger | Comments (0)

Phusion Passenger community sponsorship campaign

By Hongli Lai February 16th, 2009

We are happy to see that more and more people are using Phusion Passenger for deploying their favorite Ruby web applications. Although Phusion Passenger is very much usable at its current state, it is under constant development and is constantly being improved. Those who have been following the development in our git repository are probably aware of the exciting improvements that the next Phusion Passenger release – version 2.1 beta – will bring. Here’s a summary:

Improved compatibility with other Apache modules, such as mod_rewrite
Phusion Passenger is now fully compatible with mod_rewrite! The “RailsAllowModRewrite” option is now obsolete, and everything should work as expected.

While this might sound like a trivial improvement, please think about this for a moment, because supporting mod_rewrite has been anything but trivial. The way mod_rewrite interacts with Apache is for the most part undocumented. We have spent many, many, many man hours on reading mod_rewrite’s source code, Apache’s source code and many Apache modules’ source code, and reverse engineering its behavior through debuggers, in order to figure out how to make Phusion Passenger fully compatible. To get an idea of how much effort we spent into this, take a look at the Phusion Passenger Apache module source code. Even if you’re not familiar with C++, you might be interested in the comments associated with the following functions: prepareRequest, saveOriginalFilename, saveStateBeforeRewriteRules, undoRedirectionToDispatchCgi, startBlockingModDir, startBlockingModAutoIndex.
The information in all these comments are obtained through hard reverse engineering work, because the behavior that these comments describe are not documented anywhere. We had to obtain this information the hard way.

Ruby 1.9 support
As announced here.
Support for NFS setups
Using Phusion Passenger to deploy applications that live on NFS shares is currently a bit awkward, and performance isn’t that good either because of filesystem access overhead. We’ve made Phusion Passenger more NFS friendly! Performance on NFS shares has been increased by up to 8 times thanks to smart filesystem access caching. Restarting applications that live on NFS shares now works seamlessly.
Various I/O handling and scaling improvements and fixes
Phusion Passenger will now try much harder to enforce I/O timeouts, so that a bad, slow or frozen HTTP client cannot hog server resources forever.

We’ve improved mod_xsendfile support. If a backend process sends an X-Sendfile response, then Phusion Passenger used to keep that backend process locked until mod_xsendfile has sent the file. This has been solved: Phusion Passenger will now immediately release the backend process, making things much more scalable.

Phusion Passenger now fully supports applications that stream large amounts of data. Apache has the tendency to buffer the entire response in memory before sending it to the HTTP client. We’ve suppressed this Apache behavior, so now your Ruby web application can happily stream hundreds of megabytes of data, and things will keep working smoothly.

Ability to disable Phusion Passenger for arbitrary URLs (PassengerEnabled option)
This allows you to, for example, integrate PHP applications alongside Ruby/Rails applications on the same virtual host.

Improved application compatibility
If your application has a model named ‘Passenger’ then it will not work on Phusion Passenger 2.0.x because that’s how our namespace is called. We’ve renamed our namespace to ‘PhusionPassenger’, so this problem is now a thing of the past.



Some applications are incompatible with Phusion Passenger’s smart spawning mode, and have to be deployed in conservative spawning mode. As you might know, smart spawning can reduce a Rails application’s startup time by as much as 90% in the right circumstances, as well as decreasing its memory usage, so it’s no surprise that smart spawning is preferred over conservative spawning. We’ve added various hooks which allows developers of incompatible applications to make their applications compatible with smart spawning.

Phusion Passenger depends on the ‘rack’ gem in order to support Rack applications. The latest ‘rack’ gem is version 0.9.1, but some applications have (incorrectly) specified a hard dependency on rack 0.4.0. If both 0.9 and 0.4 are installed, then these applications will break when run in Phusion Passenger, because the application tries to load 0.4 after 0.9 has already been loaded. We’ve implemented a fix so that even these applications will now work correctly.

Better cross-platform support
MacOS X support as well as support for 64-bit platforms have been much improved. Sun Solaris is now officially supported, thanks to contributions by Alex Osborne, Jacob Harris, alex.kiernan, Alex Tomlins and J Aaron Farr.
Non-interactive installer
The installer can now be run in non-interactive mode, ideal for scripting. Use the –auto command line option.
Improved command-line admin tools
For example, the ‘passenger-status’ tool now displays additional useful information such as a worker process’s uptime and how many requests it has processed so far.
Ability to display backtraces for all threads
If you’re using the latest version of Ruby Enterprise Edition, or if you’re using a Ruby interpreter with the caller_for_all_threads patch, then Phusion Passenger gives you the ability do dump the backtraces of all running threads to a log file. This makes it much easier to debug multithreaded web applications.
Improved security
We’ve taken various precautions in order to improve overall security. For example, if user switching is disabled, then all Phusion Passenger helper daemons will be run as non-root (they must be root for user switching to work). Temp directory permissions have been tightened to prevent malicious tampering.
More customization options for exotic systems/setups
It is now possible to customize the ‘tmp’ directory that Phusion Passenger uses. This is especially useful on systems on which Apache can’t write to /tmp (e.g. on many systems with SELinux), or systems on which /tmp is not a good candidate for temporary files for whatever reason. It’s now also possible to customize the directory in which Phusion Passenger looks for ‘restart.txt’. Global queuing usage can now be customized per-virtual host. It’s now possible to explicitly specify the location of the web application’s root directory, in case DocumentRoot + “/..” is not the correct directory.
Various usability improvements
In particular, many error messages have been improved so that end users don’t have to stare at the screen for minutes wondering what the computer is trying to tell them. There are also many small usability improvements here and there.
Various other minor improvements and bug fixes
PassengerPoolIdleTime can now be set to 0, which means that the backend application must never idle timeout. This feature has been contributed by redmar.
The passenger-status tool will now display Phusion Passenger’s own backtraces for C++ code, in order to make it possible to detect potential freezes in C++ code.
Phusion Passenger error pages now return HTTP 500 errors, as they should.
The ApplicationSpawner server and FrameworkSpawner server idle times can now be customized.
In the 2.0.x series, sometimes more backend processes might be spawned than is allowed by the ‘PassengerMaxPoolSize’ option. This has been fixed.

Announcing the first Phusion Passenger community sponsorship campaign

Phusion Passenger is open source and 100% free (as in gratis), so before we release 2.1, we’d like to call upon the community to support Phusion Passenger’s development by sponsoring us. This campaign is public, so anybody can join.

There’s one crucial thing missing from the above list of improvements: documentation. Most of the improvements are currently undocumented. By sponsoring us, you’ll also ensure that the 2.1 release will come with high-quality and up-to-date documentation. We’ve already started working on the documentation, but there’s a lot that remains to be done.

The goal of this campaign is $14000. We already have one sponsor, Geni.com., who has donated $2000, so only $12000 remains.

Once the campaign goal has been reached, we’ll release 2.1 as soon as possible. Highlights:

  • All sponsors will be given credit on the related Phusion Passenger release announcement on our blog. If the sponsor is an organization then we’ll link to the website, if available.
  • The money that we receive through this campaign shall be treated like donations.
  • An invoice is available if both of the following conditions apply:
    • You are an organization based in the European Union.
    • You’ve sent 300 USD or more.

To sponsor this campaign, please click on the following button:
Click here to lend your support to: Phusion Passenger first community sponsorship and make a donation at www.pledgie.com !

NOTE: Some people might see the Paypal page in Dutch. We do not yet know why this happens and we’re currently investigating it. But if you’re wondering what to choose in the country drop-down box: “United States” is “Verenigde Staten” in Dutch.

This has now been fixed.

Phusion Passenger | Comments (19)

Getting ready for Ruby 1.9.1

By Hongli Lai February 2nd, 2009

We are excited about Ruby 1.9.1. Of course, with all the performance improvements, who wouldn’t be? Unfortunately a large number of Ruby libraries and extensions still don’t work on 1.9.1, so Ruby 1.9 cannot be considered production-ready yet. Ryan Bigg has done an excellent job on documenting most of the problems that one would encounter when trying to get a basic Rails app up-and-running on Ruby 1.9.1. Basically, the problems he countered were:

  • 2.2.2 isn’t compatible with 1.9.1. Use Rails 2.3.0 RC1 or Rails edge.
  • The mysql gem needs patching.
  • The hpricot gem needs patching.
  • The postgres gem needs patching.
  • Thin needs patching.
  • The fastthread gem needs patching.
  • Mongrel needs patching.

But what about Phusion Passenger? Good news:
Phusion Passenger is Ruby 1.9.1-compatible since this commit (today).

Here’s a screenshot of a Rails 2.3.0 app running in Phusion Passenger on Ruby 1.9.1:

passenger-ruby19

Do you see the changes? Me neither. That’s the point. :)

We’ve encountered the following issues upon trying to get a simple Rails 2.3 app up running with Phusion Passenger and Ruby 1.9.1:

Fastthread isn’t compatible with 1.9
Both Mongrel and Phusion Passenger depend on Fastthread, which is a threading library that fixes some threading implementation bugs in older versions of Ruby 1.8. Fastthread is only a required dependency when running on older versions of Ruby 1.8. Unfortunately there’s no way to tell RubyGems “we depend on fastthread, but only when running on older versions of Ruby 1.8, and not on JRuby”.

Fastthread doesn’t compile on Ruby 1.9 (or on JRuby or other Ruby implementations for that matter), so when you type “gem install passenger” or “gem install mongrel” on Ruby 1.9, the installation fails with a ton of compile errors.

We’ve patched fastthread so that it becomes a no-op on Ruby 1.9.1 and on JRuby (that is, fastthread will install correctly but it won’t do anything). These patches have been submitted to Mentalguy, the maintainer of fastthread.

The sqlite3-ruby gem doesn’t work on 1.9
Jeremy Kemper submitted a 1.9 compatibility patch in the past, which had been committed. Unfortunately even with this patch, sqlite3-ruby isn’t compatible with 1.9.1.

We’ve gone ahead and fixed 1.9.1 support. The patch can be found here: http://rubyforge.org/tracker/index.php?func=detail&aid=23792&group_id=254&atid=1045

Hongli Lai Ninh Bui

Phusion Passenger, Ruby | Comments (26)

Ruby Enterprise Edition 1.8.6-20090201 released

By Hongli Lai February 1st, 2009

What is Ruby Enterprise Edition?

Ruby Enterprise Edition (REE) is a server-oriented distribution of the official Ruby interpreter, and includes various additional enhancements. REE’s main benefits are the ability to reduce Ruby on Rails applications’ memory usage by 33% on average, the ability to increase applications’ performance and the inclusion of various analysis and debugging features. The lower memory usage and increased performance are possible because REE includes – among other enhancements – a copy-on-write friendly garbage collector, as well as an improved memory allocator. REE has been out for several months now and is already used by many high-profile websites and organizations, such as New York Times, Shopify and 37signals.

“We switched to enterprise ruby to get the full benefit of the [copy-on-write] memory characteristics and we can absolutely confirm the memory savings of 30% some others have reported. This is many thousand dollars of savings even at today’s hardware prices.”
Tobias Lütke (Shopify)

And just like Phusion Passenger, Ruby Enterprise Edition is 100% open source.

Changes

Version 1.8.6-20090201 is a minor bugfix release. The changes are as follows:

  • Fixed a memory corruption bug in the readline library. If you were having problems with readline or irb, try this release.
  • Preserve file attributes when installing tcmalloc. Symlinks are now properly copied as symlinks.
  • For those who do not want to or can’t use the REE installer, there are now manual installation instructions available.

Download & upgrade

To install Ruby Enterprise Edition, please visit the download page. To upgrade from a previous version, simply install into the same prefix that you installed to last time. Please also refer to the documentation for upgrade instructions.

Hongli Lai Ninh Bui

Ruby Enterprise Edition | Comments (2)