Phusion Corporate Blog

How to fix the Ruby 1.9 HTTPS/Bundler segmentation fault on OS X Lion

If you’ve installed a gem bundle on OS X Lion the past few weeks then you may have seen the dreaded “[BUG] Segmentation fault” error, where Ruby sees to crash in the connect C function in http.rb. Upgrading to the latest Ruby 1.9.3 version (p194) doesn’t seem to help. Luckily someone has found a solution for this problem.

It turns out the segmentation fault is caused by an incompatibility between MacPort’s OpenSSL and RVM. MacPorts installs everything to /opt/local but RVM does not look for OpenSSL in /opt/local. We solved the problem by reinstalling Ruby 1.9.3 with the MacPorts OpenSSL, as follows:

sudo port install libyaml
rvm reinstall ruby-1.9.3 --with-openssl-dir=/opt/local --with-opt-dir=/opt/local

A sneak preview of Phusion Passenger 3.2, part 2

In our last Phusion Passenger sneak preview article we described a number of large but exciting changes in the upcoming 3.2 series. Since then, much development progress has been made thanks to much-appreciated user feedback. A ton of bugs have been fixed and we’re almost ready to dogfood it (running it in production on our own servers). We’re already using the 3.2 pre-release code on our development workstations while developing web apps.

In this article I shall explain more changes that the 3.2 series brings.

More zero-copy I/O

In the Phusion Passenger 3.0 technology preview articles we first described the introduction of a zero-copy I/O architecture. In 3.2, this architecture has evolved a lot further. In most performance-critical places we now avoid copying data whenever we can, and we use scatter-gather I/O calls all over the place instead of traditional I/O calls.

What is scatter-gather I/O? Normally when you have strings from multiple memory addresses, and you want to write them over a file descriptor, you have two choices:

Concatenate all strings into one big string, and send the big string to the kernel. This requires more memory and involves copying data, but only involves one call to the kernel. A kernel call tends to be much more expensive than a concatenation operation unless you’re working with a lot of data.
Send each string individually to the kernel. You don’t need as much memory but you need a lot of expensive kernel calls.

Normal I/O

In a similar fashion, if you want to read some data from a file descriptor but you want different parts of the data to end up in different memory buffers, then you either have to read() the data into a big buffer and copy each parts to the individual buffers, or you have to read() each part individually into its own buffer.

With scatter-gather I/O you can pass an array of memory buffers to the kernel. This way you can tell the kernel to write multiple buffers to a file descriptor, as if they form a single contiguous buffer, but with only one kernel call. Similarly you can tell the kernel to put different parts of the read data into different buffers. On Unix systems this is done through the readv and writev() system calls. In Phusion Passenger 3.2 we use the latter system call extensively.

Unfortunately writev() has many quirks. Typical implementations cannot handle more than IOVEC_MAX buffers per call where IOVEC_MAX is a constant with an arbitrary number. On some implementations the call will fail if the limit is surpassed, but on Linux/glibc it will quietly concatenate everything into a big buffer for you! Neither are desirable properties in Phusion Passenger, but we at Phusion care about stability so we have written extensive code to take care of this issue.

Environment variable passing

Properly passing environment variables in Phusion Passenger 3 for Apache is quite a pain. People expect SetEnv to just work, but in practice it doesn’t because of various implementation details in Phusion Passenger. To pass things like LD_LIBRARY_PATH you had to write a wrapper script. Passing PATH works but only if you’re not expecting it to affect the search path for the Ruby interpreter itself.

In 3.2 we fully support passing environment variables with SetEnv, and it works as expected. This is actually a side effect of supporting multiple Ruby versions, but it works out rather nicely.

Less memory usage

Phusion Passenger has always been lightweight when it comes to memory usage, but in 3.2 we’ve reduced it even further. More parts have been moved from Ruby into C++. The Ruby component now only load code that’s absolutely necessary. The result is massive memory savings:

When using the smart spawn method, the Preloader process (formerly called the ApplicationSpawner process) now uses 300 KB less memory.
When using the direct spawn method (the new name for the conservative spawn method), the request handler now uses 500 KB less memory per application worker process.
The Ruby stack, as is reachable at the Rack application object’s starting point, has been reduced from about 10 levels to only 2 levels. This results in at least 8 KB of reduced stack size. If your application is multithreaded and you’re still on Ruby 1.8 then you should see faster thread context switching performance.

Memory measurements are done on OS X Lion. Your mileage may vary.

Release date

So people have been asking us when 3.2 will be released. We want to make sure that the 3.2 release is a rock-solid one, as we always do with our products. But we need your help. Please download the 3.2 pre-release code from Github and play with it and report any bugs you find. Or better yet: send us a patch. It’s getting more and more stable by the day but you can make the process even faster.

Tutorial: setting up Gitlab on Debian 6

We host many git repositories on our servers using SSH and file system ACLs for access control. However, having been spoiled by Github for so long, this method feels archaic and cumbersome. While Github provides private repositories, sometimes it’s just not an option because there are some things that may never leave the organization. If you still want to have a fancy web interface for your Git repositories, then there are several alternatives:

Gitlab is an excellent option. It’s not as fully featured as Github, but like Gitorious it is open source. We’ve found that it’s not only more user friendly than Gitorious but also easier to install.

This tutorial teaches you how to setup Gitlab on Debian 6, Ruby 1.9.3, Phusion Passenger and Nginx. It assumes that Gitlab and the git repositories are hosted on the same machine. Your Gitlab installation will be protected by SSL. Your users will be able to pull from and push to your repositories using the ssh:// protocol, but they won’t have actual shell access and you don’t need to have separate system accounts for each user to control access. Gitlab (or to be more correct, gitolite, which Gitlab uses) manages access without system accounts.

Step 1: Setup sudo

In this tutorial we’re going to use sudo to switch between user accounts. Sudo is not installed by default on Debian so install it if you don’t already have it:

$ su
# apt-get update
# apt-get install sudo

Now add your own account to /etc/sudoers:

# visudo

Add something like this to the end of the file:

your_username_here   ALL=(ALL) ALL

Step 2: Install the base software

Install basic dependencies:

$ sudo apt-get update
$ sudo apt-get install build-essential git-core wget curl gcc checkinstall libxml2-dev libxslt-dev sqlite3 libsqlite3-dev libcurl4-openssl-dev libreadline-dev libc6-dev libssl-dev libmysql++-dev make build-essential zlib1g-dev libicu-dev redis-server openssh-server python-dev python-pip libyaml-dev

Install Pygments, which Gitlab needs for syntax highlighting:

$ sudo pip install pygments

Gitlab needs the sendmail command in order to send emails (for things like lost password recovery). This command is provided by the exim4, postfix and sendmail packages but you can only have one of them installed. If you don’t already have one of them installed, then we recommend postfix.

First, check whether you already have the sendmail command:

$ ls /usr/sbin/sendmail

If you get a ‘file not found’ then install Postfix:

$ sudo apt-get install postfix

Step 3: Install gitolite

We need gitolite. Gitolite is a tool used by Gitlab for managing access control to git repositories. It works by providing a bunch of config files in which you can register users and their public keys. Gitolite modifies ~/.ssh/authorized_keys appropriately based on the config files’ contents. Gitolite is not supposed to run as root; instead, it runs as a single user.

Install gitolite:

$ sudo apt-get install gitolite

The Debian package will automatically create an account called gitolite. Gitlab controls gitolite by logging into gitolite’s user account through SSH, so we want to create a passwordless SSH keypair for this. This SSH keypair is what we call the gitolite admin key.

$ sudo -u gitolite ssh-keygen
...
Enter file in which to save the key (/var/lib/gitolite/.ssh/id_rsa):    <--- enter nothing here and press enter
...
Enter passphrase (empty for no passphrase):    <--- enter nothing here and press enter
Enter same passphrase again:                   <--  enter nothing here and press enter
...
Your public key has been saved in /var/lib/gitolite/.ssh/id_rsa.pub.
...

Now you need to tell gitolite that you want to use this key as the admin key. First, let’s print the content of the public key and copy it to the clipboard:

$ sudo -u gitolite cat /var/lib/gitolite/.ssh/id_rsa.pub
(now select the output in your terminal and copy it to the clipboard)

Now let’s configure gitolite:

$ sudo dpkg-reconfigure gitolite

When asked for a username, keep it at the default:

When asked for a repository path, keep it at the default:

When asked for the admin key, paste the contents of the key you copied to clipboard:

Finally, we need to set the REPO_MASK option to 0007.

$ sudo -u gitolite vi /var/lib/gitolite/.gitolite.rc

Look for:

$REPO_UMASK = 0077;         # gets you 'rwx------'

Change it to:

$REPO_UMASK = 0007;  # rwxrwx---

Tighten security

We think the default gitolite home directory security is a bit weak: everybody on the system can access the config files and the repository files.

$ ls -ld /var/lib/gitolite
drwxr-xr-x 5 gitolite gitolite 4096 Apr 21 08:40 /var/lib/gitolite

Let’s tighten it up a little bit so that only gitolite can access the files:

$ sudo -u gitolite chmod o-rx /var/lib/gitolite

Step 4: Install Ruby 1.9

Gitlab requires Ruby 1.9 and their developers recommend Ruby 1.9.2, but we don’t recommend 1.9.2 because it has a lot of critical bugs compared to 1.9.3. We’ve been running Gitlab in production on Ruby 1.9.3 for a while now and so far everything works great, so in this tutorial we’ll teach you how to install Ruby 1.9.3. But feel free to install a different version if you disagree with us.

Debian does not provide a recent enough version of Ruby through apt, so we need to install it manually.

$ wget http://ftp.ruby-lang.org/pub/ruby/1.9/ruby-1.9.3-p194.tar.gz
$ tar xzvf ruby-1.9.3-p194.tar.gz
$ cd ruby-1.9.3-p194
$ ./configure
$ make
$ sudo make install

Install Bundler:

$ sudo gem install bundler

Step 5: Install Gitlab

Download the source

$ cd /opt
$ sudo git clone git://github.com/gitlabhq/gitlabhq.git
$ sudo chown -R gitolite:gitolite gitlabhq
$ cd gitlabhq
$ sudo -u gitolite git checkout 9af14e4bda35

(You can also checkout the stable branch for the latest stable version, but this tutorial is written specifically for commit 9af14e4bda35.)

Configure

Setup the database configuration. Make sure you fill in the database details under the production section.

$ sudo -u gitolite cp config/database.yml.example config/database.yml
$ sudo -u gitolite vi config/database.yml

Setup other Gitlab configuration.

$ sudo -u gitolite cp config/gitlab.yml.example config/gitlab.yml
$ sudo -u gitolite vi config/gitlab.yml

Set email.from to what should be filled in the ‘From’ header in emails.
Set email.to to the domain name on which you want to host Gitlab, without the protocol scheme and without the path.
Set email.protocol to https.
Set git_host.admin_uri to gitolite@localhost:gitolite-admin.
Set git_host.base_path to /var/lib/gitolite/repositories/.
Set git_host.host to the system’s SSH domain name.
Set git_host.git_user to gitolite.

Now tighten up security:

$ sudo -u gitolite chmod o-rwx config/*.yml

Install gems and setup database

$ sudo -u gitolite -H bundle install --without development test --deployment
$ sudo -u gitolite bundle exec rake db:setup RAILS_ENV=production
$ sudo -u gitolite bundle exec rake db:seed_fu RAILS_ENV=production

This last command will output an initial administrator account’s username and password. Write it down somewhere.

Check status

$ sudo -u gitolite bundle exec rake gitlab:app:status RAILS_ENV=production

You should get all YES:

Starting diagnostic
config/database.yml............exists
config/gitlab.yml............exists
/home/git/repositories/............exists
/home/git/repositories/ is writable?............YES
remote: Counting objects: 603, done.
remote: Compressing objects: 100% (466/466), done.
remote: Total 603 (delta 174), reused 0 (delta 0)
Receiving objects: 100% (603/603), 53.29 KiB, done.
Resolving deltas: 100% (174/174), done.
Can clone gitolite-admin?............YES
UMASK for .gitolite.rc is 0007? ............YES

Run a Resque worker

This worker daemon is for processing background tasks.

$ sudo -u gitolite bundle exec rake environment resque:work QUEUE=* RAILS_ENV=production BACKGROUND=yes

Step 6: Generate an SSL certificate

Generate a self-signed certificate. You may enter arbitrary certificate information but the Common Name must be set to the domain name on which you want to host Gitlab.

$ cd /opt/nginx/conf
$ sudo openssl req -new -x509 -nodes -days 3560 -out gitlab.crt -keyout gitlab.key
$ sudo chmod o-r gitlab.key

Installing the certificate into your browser

This self-signed certificate by itself provides encryption, but not authentication, making it vulnerable to man-in-the-middle attacks. To solve this problem, install this certificate into your browser.

Here’s how you do it on Google Chrome on OS X. First, copy the certificate to your local machine:

$ scp your-server.com:/opt/nginx/conf/gitlab.crt .

Now open Keychain Access. Under the “Keychains” list, select “System”. Then click on the “+” button and add the certificate file.

Step 7: Deploy to Phusion Passenger and Nginx

Install Nginx and Phusion Passenger:

$ sudo gem install passenger --no-rdoc --no-ri
$ sudo passenger-install-nginx-module
(Choose option 1 and install Nginx to /opt/nginx)

Now edit the Nginx config file

$ sudo vi /opt/nginx/conf/nginx.conf

and add this to the http block:

# This is a normal HTTP host which redirects all traffic to the HTTPS host.
server {
    listen 80;
    server_name gitlab.yourdomain.com;
    root /nowhere;
    rewrite ^ https://gitlab.phusion.nl$request_uri permanent;
}

# The actual Gitlab HTTPS host.
server {
    listen 443;
    server_name gitlab.yourdomain.com;
    root /opt/gitlabhq/public;
    ssl on;
    ssl_certificate gitlab.crt;
    ssl_certificate_key gitlab.key;
    add_header Strict-Transport-Security "max-age=315360000";
    location / {
        passenger_enabled on;
    }
    location /assets {
        expires max;
        add_header Cache-Control public;
        passenger_enabled on;
    }
}

Now start Nginx:

$ sudo /opt/nginx/sbin/nginx

And viola, you’re up and running! You can access Gitlab through https://gitlab.yourdomain.com/.

Already have Phusion Passenger installed and already running Ruby 1.8?

Phusion Passenger for Nginx does not yet support multiple Ruby versions (but it will in 3.2). If you already have Ruby apps deployed on your server using Phusion Passenger, and those apps require Ruby 1.8, then you’ll have to run Gitlab on Ruby 1.9 using Phusion Passenger Standalone, and connect it to Nginx through a reverse proxy setup.

$ cd /opt/gitlabhq
$ sudo -u gitolite passenger start -d -e production --max-pool-size 2 -S gitlab.socket

The HTTPS server block should then look like this:

upstream gitlab {
    server unix:/opt/gitlabhq/gitlab.socket;
}

# The actual Gitlab HTTPS host.
server {
    listen 443;
    server_name gitlab.yourdomain.com;
    root /opt/gitlabhq/public;
    ssl on;
    ssl_certificate gitlab.crt;
    ssl_certificate_key gitlab.key;
    add_header Strict-Transport-Security "max-age=315360000";
    proxy_set_header Host $http_host;
    proxy_set_header X-Forwarded-Ssl on;
    location / {
        if (!-f $request_filename) {
            proxy_pass http://gitlab;
            break;
        }
    }
    location /assets {
        expires max;
        add_header Cache-Control public;
        if (!-f $request_filename) {
            proxy_pass http://gitlab;
            break;
        }
    }
}

Donate

Gitlab started a donation campaign a few weeks ago. It is excellent, high-quality open source software so if you like it, consider giving them a donation to support their development.

New Phusion Passenger documentation pages

The Phusion Passenger documentation is written with the excellent Asciidoc tool, which Git also uses for its manual pages. The output looks good but it’s nothing too fancy. There were a few features that we’ve wanted for a while but aren’t available in Asciidoc. We’ve written a wrapper tool around Asciidoc called Mizuho, which provides these features.

Today, we present you with the new Phusion Passenger documentation pages.

Top bar with useful shortcuts

enter image description here

The new documentation has a top bar which gives you quick access to the table of contents. It also tells you where in the documentation you are. Scroll down in the documentation, you’ll see the top bar’s title updating itself accordingly.

Per-section comments

enter image description here

You can now comment on a section by clicking on the bubble on the left of a section title. Comments are powered by Juvia.

The documentation

Don’t be perfectionist: big code dumps suck

We’ve been developing Phusion Passenger 3.2 on local repositories for a while now. We didn’t want to show the changes to the world until we know that it’s at least somewhat ready for public consumption. For a long time 3.2 didn’t even fully compile. The end results speak for themselves: Phusion Passenger 3.2 introduces quite a large number of changes. And yet there’s something that does not feel quite right here. Not only was there a large time gap between 3.0 and 3.2, we also feel that we haven’t properly communicated our development progress to the public, which could possibly have resulted in the perception that development was slow.

We have been too perfectionist. Dr Nic was right in booing our development method.

From an open source perspective as well, development behind closed doors is not the way to go. We plan on publishing changes in a more gradual manner in the future.

A sneak preview of Phusion Passenger 3.2

We’ve been working on Phusion Passenger 3.2 on local repositories for a while now. Today we just pushed this code to Github. You can find it in the experimental branch. Despite the relatively minor version bump, Phusion Passenger 3.2 has actually received a major internal overhaul. This is because our version number growth rate is based on the quantity of user-visible improvements, not internal (technical) improvements.

Rewritten ApplicationPool

One of the central subsystems in Phusion Passenger is the ApplicationPool, which spawns Ruby application processes when necessary and keeps track of them. It scales the number of processes according to the current traffic and it ensures that the number of processes do not go over your defined resource limits. It’s one of the most complex parts of Phusion Passenger and consists of a lot of carefully written code. The ApplicationPool also goes through a huge amount of effort to ensuring that the user is appropriately notified in the event of problems. One of the nerving things about FastCGI (what a lot of people used before Mongrel and Phusion Passenger were introduced) was that when something goes wrong, you often have absolutely no idea why. There is no error report. Or sometimes it prints an error report to a log file, but users often have no idea where that log file is or even realize that they have to look in the log file in the first place. In contrast, Phusion Passenger displays this beautiful error message page which tells you all the details of the problem, right there in the browser. And because Phusion Passenger is heavily multithreaded, ApplicationPool was designed to be thread-safe. However, our initial implementation (during the 1.x and 2.x days) was not concurrent enough: the ApplicationPool’s mutex was locked while an application process is being spawned. This meant that Phusion Passenger is essentially frozen whenever an application process is being spawned; it will be unable to serve any requests during this time. You can’t even run passenger-status while this is happening. Phusion Passenger 3.x partially solves this issue by spawning all processes in the background, except for the first one. Another problem is that if an application behaves badly and freezes during startup, then Phusion Passenger also stays frozen. The ApplicationPool subsystem in 3.0 is partially written in C++, partially in Ruby. However we’ve found that Ruby is not a good language for system software such as Phusion Passenger. Although we love Ruby and still think Ruby is great for lots of things (web apps, sysadmin software, etc) we realized that it’s better to move that part to C++ instead. And we did: in Phusion Passenger 3.2, we’ve rewritten the entire ApplicationPool subsystem in C++. The new subsystem is much more concurrent and more fault-tolerant. It no longer holds the lock for an extended period of time and never freezes, ever. It enforces a spawn timeout on all processes: if they fail to spawn within 60 seconds then they will be killed. The new code is also much easier to understand and to maintain. This opens the road towards many potential future enhancements. 3.2 also supports multiple Ruby versions at the same time, something which many people have been waiting for.

Request/response I/O handling is now evented

Phusion Passenger <= 3.0 uses a static number of threads to handle request/response I/O. This severely limits the amount of concurrent I/O that we can handle. in 3.2, our request/response I/O handling subsystem has been rewritten to be single-threaded and evented. We use the excellent libev and libeio libraries by Marc Lehmann. This evented I/O subsystem allows us to handle a virtually unlimited amount of I/O requests concurrently and also solves some pathological edge case I/O problems that Phusion Passenger currently suffers from. It also opens the road towards future support for WebSockets, long polling and other mechanisms which require a connection socket to stay open for a potentially long time. One of the best features in the new I/O handler is real-time disk-buffered response forwarding. Traditionally, the web server would buffer the entire response before sending it to the client. This is necessary because you don’t want slow HTTP clients to block the application. However it also means that it’s not possible to flush partial response data to the client immediately (e.g. Rails 3.2 streaming), at least not without risking slow clients blocking the application. In Phusion Passenger 3.2, if the client is slow then we buffer the response data in memory, or to disk if the data is larger than a certain threshold. In contrast to most web servers we do not wait until the entire response has finished before forwarding the data; we do it immediately! You don’t need to turn this on, it’s enabled by default. Now application developers need never worry about response buffering anymore, it Just Works(tm) and does the right thing.

Improved WSGI support

It’s not a well-known fact because we’ve never made a fuss about it, but Phusion Passenger actually supports Python WSGI as well, and has done this since the 1.x days. Here’s a demonstration of Django running on Phusion Passenger and a demonstration of Pylons running on Phusion Passenger. Our WSGI support worked, but wasn’t particularly good. For example if the application fails to spawn then we do not display an error message in the browser. But we do now, so WSGI is now unofficially supported as a first-class citizen. Python processes are managed just like Ruby processes; all the resource limits, response buffering, etc. work exactly in the same way.

Phusion Passenger 3.0.12 released

Phusion Passenger is an Apache and Nginx module for deploying Ruby web applications. It has a strong focus on ease of use, stability and performance. Phusion Passenger is built on top of tried-and-true, battle-hardened Unix technologies, yet at the same time introduces innovations not found in most traditional Unix servers. Since version 3.0 it can also run standalone without an external web server, making it not only easier for first-time users but also ideal on development environments.

Recent changes

Phusion Passenger is under constant maintenance and development. We are pleased to announce Phusion Passenger version 3.0.12. This is a bug fix release.

[Apache] Support Apache 2.4. The event MPM is now also supported.
[Nginx] Preferred Nginx version upgraded to 1.0.15.
[Nginx] Preferred PCRE version upgraded to 8.30.
[Nginx] Fixed compatibility with Nginx < 1.0.10.
[Nginx] Nginx is now installed with http_gzip_static_module by default.
[Nginx] Fixed a memory disclosure security problem.: The issue is documented at http://www.nginx.org/en/security_advisories.html and affects more modules than just Phusion Passenger. Users are advised to upgrade as soon as possible. Patch submitted by Gregory Potamianos.
[Nginx] passenger_show_version_in_header now hides the Phusion Passenger version number from the ‘Server:’ header too.: Patch submitted by Gregory Potamianos.
Fixed a /proc deprecation warning on Linux kernel >= 3.0.

How do I upgrade to 3.0.12?

Via a gem

First install the gem with the following command:

gem install passenger

If you’re using Phusion Passenger for Apache or for Nginx, then re-run the Apache or Nginx module installer, whichever is appropriate:

passenger-install-apache2-module
passenger-install-nginx-module

At the end the installer will tell you to paste a configuration snippet into your web server config file. Replace the old snippet that you already had with this new one.

Phusion Passenger Standalone users don’t need to run anything else. Whenever you type

passenger start

it will automatically upgrade itself.

Via Ubuntu packages

John Leach from Brightbox has kindly provided Ubuntu packages for Phusion Passenger. You can find installation instructions on the Brightbox website.
(Note that John is currently packaging 3.0.12, so it might take a while before this release shows up in the apt repository.)

Via RedHat/CentOS packages

YUM repositories with RPMs are maintained by Erik Ogan and Stealthy Monkeys Consulting. Please note that Erik is currently packaging 3.0.12, so it might take a while before this release shows up in the yum repositories.

Step 1: install the release package

The easiest way to install Phusion Passenger and keep it up to date is to install the passenger-release package from the main repository:

Fedora Core 15:

yum install http://passenger.stealthymonkeys.com/fedora/15/passenger-release.noarch.rpm

Fedora Core 14:

yum install http://passenger.stealthymonkeys.com/fedora/14/passenger-release.noarch.rpm

RHEL 5 / CentOS 5 / ScientificLinux 5:
(Note: these packages depend on EPEL.)

rpm -Uvh http://passenger.stealthymonkeys.com/rhel/5/passenger-release.noarch.rpm

RHEL 6 / CentOS 6 / ScientificLinux 6:

yum install http://passenger.stealthymonkeys.com/rhel/6/passenger-release.noarch.rpm

Step 2: use Yum

From there you can use Yum to install packages. For example, try one of these:

yum install nginx-passenger

yum install mod_passenger

yum install passenger-standalone

Building your own packages

There are instructions for building your own packages and Yum repositories in the rpm directory ReadMe within the GitHub repository.

Final

Phusion Passenger is provided to the community for free. If you like Phusion Passenger, please consider sending us a donation. Thank you!

Ruby Enterprise Edition 1.8.7-2012.02 released; End of Life imminent

What is Ruby Enterprise Edition?

Ruby Enterprise Edition (REE) is a server-oriented distribution of the official Ruby interpreter, and includes various additional enhancements, such as:

A “copy-on-write friendly” garbage collector, capable of reducing Ruby on Rails applications’ memory usage by 33% on average.
The tcmalloc memory allocator, which lowers overall memory usage and boosts memory allocation speed.
The ability to performance tune the garbage collector.
The MBARI patch set, for improved garbage collection efficiency.
Various analysis and debugging features.

REE can be easily installed in parallel to your existing Ruby interpreter, allowing you switch to REE with minimal hassle or risk. REE has been out for several years now and is already used by many high-profile websites and organizations, such as New York Times, Twitter, Shopify and 37signals.

Ruby Enterprise Edition is 100% open source.

Changes

Ruby Enterprise Edition 2011.12 and 2012.01 have actually also been quietly released, but they didn’t have proper announcements on our blog. Here’s a cumulative summary:

Upgraded to Ruby 1.8.7-p358

REE 2011.01 was based on 1.8.7-p330. This latest version includes not only bug fixes but also various security fixes.

Upgraded to RubyGems 1.8.15

The previous REE release included RubyGems 1.5.2.

For a long time we’ve been reluctant to upgrade RubyGems past 1.5 because many libraries were incompatible with RubyGems > 1.5. But today, the situation has been reversed: many gemspecs in the RubyGems.org repository are incompatible with RubyGems < 1.8. As such, we’ve upgraded RubyGems to the latest version.

Upgraded to the MBARI 8 patch set

Brent Roman has released a new version of his MBARI patch set, which solves many mysterious stability problems that plagued previous versions. This REE release should be significantly more stable.

Compatible with Xcode >= 4 and OS X Lion

The REE installer now properly advises Xcode >= 4 users and OS X Lion users on how to install the proper compiler that’s necessary for installing REE.

Experimental zero-copy context switch patch removed

This experimental patch set was never production-ready, so as of this release it has been removed.

End of Life

Support for Ruby 1.8.x is slowly being dropped by the upstream Ruby core developers in favor of Ruby 1.9 and beyond. The Rails team has recently announced that they will be dropping Ruby 1.8 support in Rails 4. As such, we are also slowly End-of-Life’ing Ruby Enterprise Edition.

We have no plans to create a Ruby 1.9-based version of Ruby Enterprise Edition for the following reasons:

A copy-on-write patch has recently been checked into Ruby 2.0.
Many of the patches in Ruby Enterprise Edition are simply not necessary in 1.9.
We wish to focus our efforts on Phusion Passenger and other products. Instead of doing many things poorly, we want to do a few things, but do them very very well.

We plan on providing minor fixes and updates for the time being, but users are recommended to slowly migrate to Ruby 1.9. Phusion Passenger <= 3.1 users can use reverse proxy setups to run multiple Ruby versions on the same server. Starting from Phusion Passenger 3.2 it will support multiple Ruby versions natively without the need for reverse proxy setups.

Looking for new maintainers

We believe that Ruby Enterprise Edition has served its purpose. That said, we understand that many people still rely on Ruby 1.8 for the time being because of compatibility issues. Therefore we would like to ask for volunteers who want to take over maintenance of Ruby Enterprise Edition. Please contact us if you are interested!

Download & upgrade

To install Ruby Enterprise Edition, please visit the download page. To upgrade from a previous version, simply install into the same prefix that you installed to last time. Please also refer to the documentation for upgrade instructions.

Bundler and public applications

I think Bundler is a great tool. Its strength lies not in its ability to install all the gems that you’ve specified, but in automatically figuring out a correct dependency graph so that nothing conflicts with each other, and in the fact that it gives you rock-solid guarantees that whatever gems you’re using in development is exactly what you get in production. No more weird gem version conflict errors.

This is awesome for most Ruby web apps that are meant to be used internally, e.g. things like Twitter, Basecamp, Union Station. Unfortunately, this strength also turns in a kind of weakness when it comes to public apps like Redmine and Juvia. These apps typically allow the user to choose their database driver through config/database.yml. However the driver must also be specified inside Gemfile, otherwise the app cannot load it. The result is that the user has to edit both database.yml and Gemfile, which introduces the following problems:

The user may not necessarily be a Ruby programmer. The Gemfile will confuse him.
The user is not able to use the Gemfile.lock that the developer has provided. This makes installing in deployment mode with the developer-provided Gemfile.lock impossible.

This can be worked around in a very messy form with groups. For example:

group :driver_sqlite do
  gem 'sqlite3'
end

group :driver_mysql do
  gem 'msyql'
end

group :driver_postgresql do
  gem 'pg'
end

And then, if the user chose to use MySQL:

bundle install --without='driver_postgresql driver_sqlite'

This is messy because you have to exclude all the things you don’t want. If the app supports 10 database drivers then the user has to put 9 drivers on the exclusion list.

How can we make this better? I propose supporting conditionals in the Gemfile language. For example:

condition :driver => 'sqlite' do
  gem 'sqlite3'
end

condition :driver => 'mysql' do
  gem 'mysql'
end

condition :driver => 'postgresql' do
  gem 'pg'
end

condition :driver => ['mysql', 'sqlite'] do
  gem 'foobar'
end

The following command would install the mysql and the foobar gems:

bundle install --condition driver=mysql

Bundler should enforce that the driver condition is set: if it’s not set then it should raise an error. To allow for the driver condition to not be set, the developer must explicitly define that the condition may be nil:

condition :driver => nil do
  gem 'null-database-driver'
end

Here, bundle install will install null-database-driver.

With this proposal, user installation instructions can be reduced to these steps:

Edit database.yml and specify a driver.
Run bundle install --condition driver=(driver name)

I’ve opened a ticket for this proposal. What do you think?

XCode 4 ships a buggy compiler

You may have heard of LLVM, a compiler infrastructure library. You may have heard of GCC, the GNU C and C++ compiler. Those two are completely separate software products, but there exists llvm-gcc which is a GCC frontend that utilizes LLVM. All OS X versions <= Snow Leopard originally shipped with regular gcc, but as of Xcode 4 (which is also the default on OS X Lion) Apple has switched to llvm-gcc as their default compiler. “Hurray”, some people may think, as they heard that LLVM generates better code and/or is faster than default GCC. Unfortunately, the reality is not that great.

llvm-gcc is unmaintained and is considered deprecated by its developers. It has many bugs that will never be fixed. Unfortunately these are not obscure bugs that few people will notice: today, while working on Ruby Enterprise Edition, we encountered several! Calling alloca(0) should return a pointer to the top of the stack. On llvm-gcc however, that code returns NULL but only when compiled with -O2! llvm-gcc also generates subtly different code that causes the MBARI patch set to fail completely.

To make matters worse, consider this program:

#include <alloca.h>

int
main() {
    if (alloca(0) != NULL) {
        return 0;
    } else {
        return 1;
    }
}

Question: when compiled with “llvm-gcc -O2″, does this program exit with 0 or with 1?

That was a trick question. It always returns with 0, even when alloca(0) is broken and actually returns NULL. Turns out the optimizer in -O2 thinks that alloca(0) doesn’t return NULL (even though it does) and replaces if (alloca(0) != NULL) with if (true).

It is unfortunate that we have to declare OS X’s default compiler as being broken. We do not recommend its use. Instead, we recommend people to use /usr/bin/gcc-4.2 (which is the non-LLVM GCC compiler) instead of /usr/bin/gcc (which is a symlink to /usr/bin/llvm-gcc). You can enable this on most build systems by setting these two environment variables:

export CC=gcc-4.2
export CXX=g++-4.2

On a side note, Ruby Enterprise Edition 2011.12 is being released. The official announcement hasn’t been written yet, but we’ve made haste because of the recent Ruby security vulnerability. You can already get the files on the Ruby Enterprise Edition website. An announcement will follow soon.

Update

some people have pointed out that they consider alloca(0) undefined. I guess it depends on the way you interpret the documentation. It would seem pretty clear to me what alloca(0) means, just like what memcpy(x, y, 0) means, but it’s understandable when some people would interpret it as undefined.
Still, alloca(0) failing is only one of the problems that prevented Ruby from compiling properly.

Why alloca(0)?

alloca(0) is used in Ruby Enterprise Edition’s conservative garbage collector, as a way to detect the end of the stack. There’s a bunch of fallback #ifdefs in the code: on platforms that don’t have alloca, it detects the end of the stack by calling a forced-non-inline function which returns the address of its sole local variable, but that assumes that the compiler supports the ‘noinline’ keyword. In any case, all of versions depend on highly platform-specific behavior.