Phusion Corporate Blog

Bundler and public applications

I think Bundler is a great tool. Its strength lies not in its ability to install all the gems that you’ve specified, but in automatically figuring out a correct dependency graph so that nothing conflicts with each other, and in the fact that it gives you rock-solid guarantees that whatever gems you’re using in development is exactly what you get in production. No more weird gem version conflict errors.

This is awesome for most Ruby web apps that are meant to be used internally, e.g. things like Twitter, Basecamp, Union Station. Unfortunately, this strength also turns in a kind of weakness when it comes to public apps like Redmine and Juvia. These apps typically allow the user to choose their database driver through config/database.yml. However the driver must also be specified inside Gemfile, otherwise the app cannot load it. The result is that the user has to edit both database.yml and Gemfile, which introduces the following problems:

The user may not necessarily be a Ruby programmer. The Gemfile will confuse him.
The user is not able to use the Gemfile.lock that the developer has provided. This makes installing in deployment mode with the developer-provided Gemfile.lock impossible.

This can be worked around in a very messy form with groups. For example:

group :driver_sqlite do
  gem 'sqlite3'
end

group :driver_mysql do
  gem 'msyql'
end

group :driver_postgresql do
  gem 'pg'
end

And then, if the user chose to use MySQL:

bundle install --without='driver_postgresql driver_sqlite'

This is messy because you have to exclude all the things you don’t want. If the app supports 10 database drivers then the user has to put 9 drivers on the exclusion list.

How can we make this better? I propose supporting conditionals in the Gemfile language. For example:

condition :driver => 'sqlite' do
  gem 'sqlite3'
end

condition :driver => 'mysql' do
  gem 'mysql'
end

condition :driver => 'postgresql' do
  gem 'pg'
end

condition :driver => ['mysql', 'sqlite'] do
  gem 'foobar'
end

The following command would install the mysql and the foobar gems:

bundle install --condition driver=mysql

Bundler should enforce that the driver condition is set: if it’s not set then it should raise an error. To allow for the driver condition to not be set, the developer must explicitly define that the condition may be nil:

condition :driver => nil do
  gem 'null-database-driver'
end

Here, bundle install will install null-database-driver.

With this proposal, user installation instructions can be reduced to these steps:

Edit database.yml and specify a driver.
Run bundle install --condition driver=(driver name)

I’ve opened a ticket for this proposal. What do you think?

XCode 4 ships a buggy compiler

You may have heard of LLVM, a compiler infrastructure library. You may have heard of GCC, the GNU C and C++ compiler. Those two are completely separate software products, but there exists llvm-gcc which is a GCC frontend that utilizes LLVM. All OS X versions <= Snow Leopard originally shipped with regular gcc, but as of Xcode 4 (which is also the default on OS X Lion) Apple has switched to llvm-gcc as their default compiler. “Hurray”, some people may think, as they heard that LLVM generates better code and/or is faster than default GCC. Unfortunately, the reality is not that great.

llvm-gcc is unmaintained and is considered deprecated by its developers. It has many bugs that will never be fixed. Unfortunately these are not obscure bugs that few people will notice: today, while working on Ruby Enterprise Edition, we encountered several! Calling alloca(0) should return a pointer to the top of the stack. On llvm-gcc however, that code returns NULL but only when compiled with -O2! llvm-gcc also generates subtly different code that causes the MBARI patch set to fail completely.

To make matters worse, consider this program:

#include <alloca.h>

int
main() {
    if (alloca(0) != NULL) {
        return 0;
    } else {
        return 1;
    }
}

Question: when compiled with “llvm-gcc -O2″, does this program exit with 0 or with 1?

That was a trick question. It always returns with 0, even when alloca(0) is broken and actually returns NULL. Turns out the optimizer in -O2 thinks that alloca(0) doesn’t return NULL (even though it does) and replaces if (alloca(0) != NULL) with if (true).

It is unfortunate that we have to declare OS X’s default compiler as being broken. We do not recommend its use. Instead, we recommend people to use /usr/bin/gcc-4.2 (which is the non-LLVM GCC compiler) instead of /usr/bin/gcc (which is a symlink to /usr/bin/llvm-gcc). You can enable this on most build systems by setting these two environment variables:

export CC=gcc-4.2
export CXX=g++-4.2

On a side note, Ruby Enterprise Edition 2011.12 is being released. The official announcement hasn’t been written yet, but we’ve made haste because of the recent Ruby security vulnerability. You can already get the files on the Ruby Enterprise Edition website. An announcement will follow soon.

Update

some people have pointed out that they consider alloca(0) undefined. I guess it depends on the way you interpret the documentation. It would seem pretty clear to me what alloca(0) means, just like what memcpy(x, y, 0) means, but it’s understandable when some people would interpret it as undefined.
Still, alloca(0) failing is only one of the problems that prevented Ruby from compiling properly.

Why alloca(0)?

alloca(0) is used in Ruby Enterprise Edition’s conservative garbage collector, as a way to detect the end of the stack. There’s a bunch of fallback #ifdefs in the code: on platforms that don’t have alloca, it detects the end of the stack by calling a forced-non-inline function which returns the address of its sole local variable, but that assumes that the compiler supports the ‘noinline’ keyword. In any case, all of versions depend on highly platform-specific behavior.

Phusion Passenger 3.0.11 released

Phusion Passenger is an Apache and Nginx module for deploying Ruby web applications. It has a strong focus on ease of use, stability and performance. Phusion Passenger is built on top of tried-and-true, battle-hardened Unix technologies, yet at the same time introduces innovations not found in most traditional Unix servers. Since version 3.0 it can also run standalone without an external web server, making it not only easier for first-time users but also ideal on development environments.

Recent changes

Phusion Passenger is under constant maintenance and development. We are pleased to announce Phusion Passenger version 3.0.11. This is a bug fix release.

3.0.10 was skipped because we discovered a last-minute compilation problem on FreeBSD after having already pushed the gem to RubyGems.org.

[Nginx] Dropped support for Nginx versions older than 1.0.0
[Nginx] Fixed support for Nginx 1.1.4+
[Nginx, Standalone] Upgraded default Nginx version to 1.0.10: The previously default version was 1.0.5.
[Nginx] New option passenger_max_requests: This is equivalent to the PassengerMaxRequests option in the Apache version: Phusion Passenger will automatically shutdown a worker process once it has processed the specified number of requests. Contributed by Paul Kmiec.
[Apache] New option PassengerBufferResponse: The Apache version did not buffer responses. This could block the Ruby worker process in case of slow clients. We now enable response buffering by default. It can be turned off through this option. Feature contributed by Ryo Onodera.
Fixed remaining Ruby 1.9.3 compatibility problems: We already supported Ruby 1.9.3 since 3.0.8, but due to bugs in Ruby 1.9.3′s build system Phusion Passenger would fail to detect Ruby 1.9.3 features on some systems. Fixes issue #714.
Fixed a bug in PassengerPreStart: A regression was introduced in 3.0.8, causing the prespawn script to connect to the host name instead of to 127.0.0.1. Fix contributed by Andy Allan.
Fixed compatibility with GCC 4.6
Affected systems include Ubuntu 11.10.
Fixed various compilation problems.
Fixed some Ruby 1.9 encoding problems.
Fixed some Ruby 1.9.3 deprecation warnings.
Improved performance and solved some warnings on Xen systems by compiling with `-mno-tls-direct-seg-refs`. Patch contributed by Michał Pokrywka.

How do I upgrade to 3.0.11?

Via a gem

First install the gem with the following command:

gem install passenger

If you’re using Phusion Passenger for Apache or for Nginx, then re-run the Apache or Nginx module installer, whichever is appropriate:

passenger-install-apache2-module
passenger-install-nginx-module

At the end the installer will tell you to paste a configuration snippet into your web server config file. Replace the old snippet that you already had with this new one.

Phusion Passenger Standalone users don’t need to run anything else. Whenever you type

passenger start

it will automatically upgrade itself.

Via Ubuntu packages

John Leach from Brightbox has kindly provided Ubuntu packages for Phusion Passenger. The package is available from the Brightbox repository which you can find at:

http://apt.brightbox.net

Add the following line to the Third Party Software Sources:

deb http://apt.brightbox.net hardy main

(The simplest way to do that is to create a file in /etc/apt/sources.list.d/ containing the deb instruction, and then run ‘apt-get update’).

Once you’ve done this then you can install Phusion Passenger by running:

sudo apt-get install libapache2-mod-passenger

-or-

sudo apt-get install nginx-brightbox

(Note that John is currently packaging 3.0.11, so it might take a while before this release shows up in the apt repository.)

Via RedHat/CentOS packages

YUM repositories with RPMs are maintained by Erik Ogan and Stealthy Monkeys Consulting. Please note that Erik is currently packaging 3.0.11, so it might take a while before this release shows up in the yum repositories.

Step 1: install the release package

The easiest way to install Phusion Passenger and keep it up to date is to install the passenger-release package from the main repository:

Fedora Core 15:

yum install http://passenger.stealthymonkeys.com/fedora/15/passenger-release.noarch.rpm

Fedora Core 14:

yum install http://passenger.stealthymonkeys.com/fedora/14/passenger-release.noarch.rpm

RHEL 5 / CentOS 5 / ScientificLinux 5:
(Note: these packages depend on EPEL.)

rpm -Uvh http://passenger.stealthymonkeys.com/rhel/5/passenger-release.noarch.rpm

RHEL 6 / CentOS 6 / ScientificLinux 6:

yum install http://passenger.stealthymonkeys.com/rhel/6/passenger-release.noarch.rpm

Step 2: use Yum

From there you can use Yum to install packages. For example, try one of these:

yum install nginx-passenger

yum install mod_passenger

yum install passenger-standalone

Building your own packages

There are instructions for building your own packages and Yum repositories in the rpm directory ReadMe within the GitHub repository.

Final

Phusion Passenger is provided to the community for free. If you like Phusion Passenger, please consider sending us a donation. Thank you!

Installing Ruby Enterprise Edition on OS X Lion

We’ve received some reports about users being unable to install Ruby Enterprise Edition on OS X Lion. Apparently the compilation process segfaults. It turns out that OS X Lion has switched to llvm-gcc as the default compiler. We currently suspect that the segfaults are caused by incompatibility between llvm-gcc’s code generation and the MBARI patch set (similar problems have been seen in the past but with other compilers on other platforms).

To install Ruby Enterprise Edition on Lion one needs to force the compiler to plain GCC instead of llvm-gcc. This can be done by setting the environment variable CC to /usr/bin/gcc-4.2. For example:

$ sudo bash
# export CC=/usr/bin/gcc-4.2
# ./installer

Or, when using RVM (from Stack Overflow):

$ rvm remove ree
$ export CC=/usr/bin/gcc-4.2
$ rvm install --force ree

Phusion Passenger 3.0.9 released

Recent changes

Phusion Passenger is under constant maintenance and development. We are pleased to announce Phusion Passenger version 3.0.9. This is a bug fix release.

[Nginx] Fixed a NULL pointer crash that occurs on HTTP/1.0 requests when the Host header isn’t given.
Fixed deprecation warnings on RubyGems >= 1.6.
Improved Union Station support stability.

How do I upgrade to 3.0.9?

Via a gem

First install the gem with the following command:

gem install passenger

If you’re using Phusion Passenger for Apache or for Nginx, then re-run the Apache or Nginx module installer, whichever is appropriate:

passenger-install-apache2-module
passenger-install-nginx-module

At the end the installer will tell you to paste a configuration snippet into your web server config file. Replace the old snippet that you already had with this new one.

Phusion Passenger Standalone users don’t need to run anything else. Whenever you type

passenger start

it will automatically upgrade itself.

Via Ubuntu packages

John Leach from Brightbox has kindly provided Ubuntu packages for Phusion Passenger. The package is available from the Brightbox repository which you can find at:

http://apt.brightbox.net

Add the following line to the Third Party Software Sources:

deb http://apt.brightbox.net hardy main

(The simplest way to do that is to create a file in /etc/apt/sources.list.d/ containing the deb instruction, and then run ‘apt-get update’).

Once you’ve done this then you can install Phusion Passenger by running:

sudo apt-get install libapache2-mod-passenger

-or-

sudo apt-get install nginx-brightbox

(Note that John is currently packaging 3.0.9, so it might take a while before this release shows up in the apt repository.)

Via RedHat/CentOS packages

YUM repositories with RPMs are maintained by Erik Ogan and Stealthy Monkeys Consulting. Please note that Erik is currently packaging 3.0.9, so it might take a while before this release shows up in the yum repositories.

Step 1: install the release package

The easiest way to install Phusion Passenger and keep it up to date is to install the passenger-release package from the main repository:

Fedora Core 15:

yum install http://passenger.stealthymonkeys.com/fedora/15/passenger-release.noarch.rpm

Fedora Core 14:

yum install http://passenger.stealthymonkeys.com/fedora/14/passenger-release.noarch.rpm

RHEL 5 / CentOS 5 / ScientificLinux 5:
(Note: these packages depend on EPEL.)

rpm -Uvh http://passenger.stealthymonkeys.com/rhel/5/passenger-release.noarch.rpm

RHEL 6 / CentOS 6 / ScientificLinux 6:

yum install http://passenger.stealthymonkeys.com/rhel/6/passenger-release.noarch.rpm

Step 2: use Yum

From there you can use Yum to install packages. For example, try one of these:

yum install nginx-passenger

yum install mod_passenger

yum install passenger-standalone

Building your own packages

There are instructions for building your own packages and Yum repositories in the rpm directory ReadMe within the GitHub repository.

Final

Phusion Passenger is provided to the community for free. If you like Phusion Passenger, please consider sending us a donation. Thank you!

Rendering Rails 3.1 assets to string

The upcoming Rails 3.1 will come with a powerful asset pipeline, which is a framework for allowing developers to preprocess, concatenate, minify and compress Javascripts and CSS files. Javascripts can be written in CofeeScript, which is compiled on-the-fly to Javascript. Similarly, CSS can be written in Sass or SCSS, which are compiled on-the-fly to plain old CSS. Many people have already written about this topic.

Today we ran into a situation in which we wanted to render an SCSS file to a string so that we can put it in a JSON document. The way to do this is non-obvious. First try:

render :template => 'assets/stylesheets/api.css'

Rails complained that it cannot find this file in its search path, which only includes ‘app/views’.

Next try involved using the :file option because it also searches in Rails.root:

render :file => 'app/assets/stylesheets/api.css'

This time it successfully found the file, but couldn’t render it because there’s no :scss template engine. Apparently the asset pipeline does not integrate into the Rails template engine framework.

After some code digging, it turned out that asset serving is completely powered by Sprockets. The Rack middleware responsible for serving the /assets is Sprockets::Server, which looks up asset information using a Sprockets::Environment object. This object also handles rendering. The Rails integration allows such an object to be accessed through the YourApp::Application.assets method.

So the correct way to render an asset to a string is as follows:

YourApp::Application.assets.find_asset('api.css').body

Here, YourApp is your application’s module name as found in config/application.rb.

default_value_for 1.0.5 released: default attribute values for ActiveRecord models

We’ve just released default_value_for version 1.0.5. default_value_for is a Rails plugin for supporting default values in ActiveRecord models. Like this:

class User < ActiveRecord::Base
  default_value_for :age, 10
end

User.new.age   # => 10

You can read more about this plugin at Github.

Version 1.0.5 fixes support for Rails 3.0 and Rails 3.1. Unfortunately we had to remove one feature due to changes in ActiveRecord 3.1. It is no longer possible to access associations in the default_value_for_block. The following is no longer possible:

class User < ActiveRecord::Base
  belongs_to :organization

  default_value_for :name do |model|
    model.name = "Drone for #{model.organization.name}"
  end
end

evil_organization.name  # => "Evil Organization"
user = evil_organization.users.create
user.name               # => "Drone for Evil Organization"??

One would expect the last expression to evaluate to “Drone for Evil Organization”. On Rails 2 and Rails 3.0, it does, but on Rails 3.1 it does not. Because of the way the ‘organization’ attribute is assigned to the model by ActiveRecord, we are no longer able to support this behavior. Inside the default_value_for block, model.organization would evaluate to nil.

Launching unofficial, automatically updated Github mirror for the Nginx SVN repository

I probably don’t need to tell you what a great web server Nginx is. It’s lightweight, it’s fast, it’s scalable, and many people like it for its configuration file syntax yet flexible features. It’s no surprise that Nginx is doing a great job serving as a core for Phusion Passenger Standalone (our Ruby/Rails web application server which supports Apache and Nginx).

Nginx’s development has traditionally been behind closed doors. Nginx is written for the most part by one brilliant man, Igor Sysoev, who accepts patches on the Nginx mailing list. But for a long time there was no source repository with which contributors and interested people can track the development process, and no official bug tracker. All of that have changed in 2011. Igor has established Nginx as a company. The Nginx SVN repository is now open to the public and a few days ago they even opened a Trac. In short: a lot of great news lately.

Announcing the Github Nginx mirror

These days a lot of people have switched from Subversion to Git. When you’ve worked with Git for a while, Subversion probably feels archaic and unproductive. The “killer app” for Git is Github, which provides an unbeatable collaboration experience. Indeed, many projects that have switched to Github reported a dramatic increase in contributions!

In order to stimulate Nginx development, we’ve launched a Github mirror of the Nginx SVN repository:

Github page: https://github.com/phusion/nginx
Git URL: git://github.com/phusion/nginx.git

A few notes about this mirror:

It’s automatically updated twice a day.
It’s read-only. We don’t accept pull requests. Changes should be sent to the Nginx mailing list in the form of patches.
We intentionally don’t mirror all the branches and tags, only the most recent ones, because we don’t believe the old branches and tags are useful to anybody.
Please feel free to contact us if you have an issue with our mirror.

Happy developing!

Phusion Passenger 3.0.8 released

Recent changes

Phusion Passenger is under constant maintenance and development. We are pleased to announce Phusion Passenger version 3.0.8. This is a bug fix release.

[Nginx] Upgraded preferred Nginx version to 1.0.5.
[Nginx] Fixed various compilation problems on various platforms.
[Nginx] We now ensure that SERVER_NAME is equal to HTTP_HOST without the port part.
This is needed for Rack compliance. By default Nginx sets SERVER_NAME to whatever is specified in the server_name directive, but that’s not necessarily the correct value. This fixes, for example, the use of the ‘map’ statement in config.ru.
[Nginx] Added the options passenger_buffer_size, passenger_buffers and passenger_busy_buffers_size.
These options are similar to proxy_module’s similarly named options. You can use these to e.g. increase the maximum header size limit.
[Nginx] passenger_pre_start now supports virtual hosts that listen on Unix domain sockets.
[Apache] Fixed the pcre.h compilation problem.
[Standalone] Fixed ‘passenger stop’.
It didn’t work properly because it kept waiting for ‘tail’ to exit. We now properly terminate ‘tail’ as well.
Fixed compatibility with Rake 0.9.
Fixed various Ruby 1.9 compatibility issues.
Various documentation improvements.
New Union Station filter language features.
It now supports status codes and response times. Please refer to https://engage.unionstationapp.com/help#filtering for more information.

How do I upgrade to 3.0.8?

Via a gem

First install the gem with the following command:

gem install passenger

If you’re using Phusion Passenger for Apache or for Nginx, then re-run the Apache or Nginx module installer, whichever is appropriate:

passenger-install-apache2-module
passenger-install-nginx-module

At the end the installer will tell you to paste a configuration snippet into your web server config file. Replace the old snippet that you already had with this new one.

Phusion Passenger Standalone users don’t need to run anything else. Whenever you type

passenger start

it will automatically upgrade itself.

Via Ubuntu packages

John Leach from Brightbox has kindly provided Ubuntu packages for Phusion Passenger. The package is available from the Brightbox repository which you can find at:

http://apt.brightbox.net

Add the following line to the Third Party Software Sources:

deb http://apt.brightbox.net hardy main

(The simplest way to do that is to create a file in /etc/apt/sources.list.d/ containing the deb instruction, and then run ‘apt-get update’).

Once you’ve done this then you can install Phusion Passenger by running:

sudo apt-get install libapache2-mod-passenger

-or-

sudo apt-get install nginx-brightbox

(Note that John is currently packaging 3.0.8, so it might take a while before this release shows up in the apt repository.)

Via RedHat/CentOS packages

YUM repositories with RPMs are maintained by Erik Ogan and Stealthy Monkeys Consulting. Please note that Erik is currently packaging 3.0.8, so it might take a while before this release shows up in the yum repositories.

Step 1: install the release package

The easiest way to install Phusion Passenger and keep it up to date is to install the passenger-release package from the main repository:

Fedora Core 14:

rpm -Uvh http://passenger.stealthymonkeys.com/fedora/14/passenger-release.noarch.rpm

Fedora Core 13:

rpm -Uvh http://passenger.stealthymonkeys.com/fedora/13/passenger-release.noarch.rpm

RHEL 5 / CentOS 5 / ScientificLinux 5:
(Note: these packages depend on EPEL.)

rpm -Uvh http://passenger.stealthymonkeys.com/rhel/5/passenger-release.noarch.rpm

RHEL 6 / CentOS 6 / ScientificLinux 6:
These packages will be available as soon as there is a stable beta of either CentOS 6 or SL 6 to use as a build platform.

Step 2: use Yum

From there you can use Yum to install packages. For example, try one of these:

yum install nginx-passenger

yum install mod_passenger

yum install passenger-standalone

Building your own packages

There are instructions for building your own packages and Yum repositories in the rpm directory ReadMe within the GitHub repository.

Final

Phusion Passenger is provided to the community for free. If you like Phusion Passenger, please consider sending us a donation. Thank you!

Starting Union Station beta phase 2, reopening registrations

Union Station is our state-of-the-art web application performance monitoring and behavior analysis service. It is currently in public beta, which started on March 2. You can follow us on Twitter through @unionstationapp.

We’ve been in beta for little over a month now, and during that period in time, we’ve been fortunate enough to have been able to receive a lot of valuable feedback from our beta testers. As of today we are entering the next phase of the beta. Previous beta accounts have been removed, and all people are once again given a chance again to register. This is necessary due to some changes we’ve made to the terms of service (in particular the ability for beta testers to opt-in for announcements like these via email pertaining to Union Station) as well as a new offering in plans. Like last time there is a registration limit.

During the last beta a number of people have requested an account. These people were put on a waiting list. When the registration limit has been reached we’ll manually register as many people on this waiting list, as well as people who have given us feedback during the first beta, as we can.

Please note that the Terms of Service are still a work in progress and may be subject to changes in the near future, and that Union Station is still in beta.

New account plans

After having gained a better idea of our resource requirements we have been working on refining our offerings accordingly. In particular, we’ve decided to deprecate the Hobby plan in favor of buffing up the Startup plan in terms of retention time by doubling it from 7 to 14 days.

In place of the Hobby plan, we’ve introduced the Jumbo plan for the medium to large-sized sites, which allows up to 8 nodes.

Plans are still subject to change as we’re still in beta, but we believe we’ve found a much better balance with this new offering in terms of features and pricing.

Larger data retention time during the new beta

During the previous beta we capped data retention times for all plans to 7 days. During this phase of the beta however, we’re going to attempt to double that data retention time to 14 days to see how all the back-end changes hold up in the current situation.

UI polishes, bug fixes and features

We’ve fixed many of the bugs and UI quirks that users have reported. Some of the most important UI polishes and new features that we’ve introduced include:

HTTP response status codes are now logged as well. This requires Phusion Passenger 3.0.7 or later. The user interface highlights the requests for which the status code is not successful (i.e. in the 4xx-5xx range).
Requests with exceptions are now highlighted.
Requests now display all associated exceptions in a tab.
The Union Station filter language has been further improved.
Time range and active tab information are now saved. Previously, changing the tab would reset the viewed time range to the default value of “Past 30 minutes”. Clicking on another node would reset the active tab back to “Analysis”. The UI now remembers which time range you’ve chosen and which tab was active.
The ‘throughput’ chart has been revamped. It previously displayed requests per hour, but due to many reports about the unintuitiveness of the chart we’ve now changed it to display requests per minute.
We now work well with different time zones. We used to try to autodetect the user’s browser time zone through Javascript but this has proven to be far less reliable than we hoped it would be. We now store time zone information on the server and recalculate all dates according to the user’s preferred time zone. You can change your time zone on your user profile page.
Authorizing another user now works as expected. Previously when you wanted to authorize another person to view the data for your web app, that person must first register a Viewer account after which you can authorize his email address. This was a bit cumbersome and we’ve now streamlined the process by automatically sending that person an invitation email which allows him to sign up for a Viewer account immediately.

(Re-)register a beta account at the
Union Station registration page »