Ruby Enterprise Edition 1.8.6-20081205 released, thank you sponsors

Ruby Enterprise Edition (REE) is a branch of the official Ruby interpreter which is capable of reducing your Rails applications’ memory usage by 33% on average, as well as improving your applications’ performance. This is possible because REE includes copy-on-write enhancements for the garbage collector, as well as an improved memory allocator (tcmalloc). REE has been out for several months now and is already used by many high-profile websites and organizations, such as New York Times, Shopify and 37signals.

“We switched to enterprise ruby to get the full benefit of the [copy-on-write] memory characteristics and we can absolutely confirm the memory savings of 30% some others have reported. This is many thousand dollars of savings even at today’s hardware prices.” – Tobias Lütke (Shopify)

And just like Phusion Passenger, Ruby Enterprise Edition is 100% open source.

Recent developments

REE has just become better. We had been talking with DHH from 37signals about a possible sponsorship campaign for supporting REE development. The campaign has recently ended, and so we’re presenting the world with Ruby Enterprise Edition version 1.8.6-20081205.

The sponsored improvements are:

Integration with the RailsBench garbage collector patches

These patches allow one to tweak various garbage collector settings. 37signals uses this to greatly improve server performance. The RailsBench GC patches did not work on REE in previous releases, but now we’ve properly integrated them into REE.

Better Mac OS X support

Previous versions sort-of support Mac OS X, but some people have reported problems. We’ve improved Mac OS X support as follows:

Installation should now work out-of-the-box on Mac OS X.
The tcmalloc memory allocator has been properly integrated with Mac OS X. Tcmalloc is what makes REE faster than the standard Ruby interpreter. In previous versions, tcmalloc was disabled when installing on OS X because it didn’t work properly on OS X. This not only meant that there were no performance gains compared to standard Ruby, it also meant that the copy-on-write optimizations were a bit less effective than they could be. By properly integrating tcmalloc on OS X, which was a tough task, OS X users can now fully enjoy the improved performance and copy-on-write optimizations.
Various OS X-related crashes have been fixed.

Special thanks to Stephen Heuer from Arux Software Inc. for providing access to an OS X testing environment.

Better 64-bit support

Even though previous versions support 64-bit platforms, many people experienced problems. We’ve spent some time making sure that installation works out-of-the-box on 64-bit platforms. We’ve tested against 64-bit FreeBSD 7 and 64-bit Ubuntu 8.10 Server.

Better Solaris support

We’ve made sure that installation works out-of-the-box.
Special thanks goes out to our friends at Sun for providing us with access to a Solaris testing environment.

In addition, this release also comes with various non-sponsored improvements and changes:

caller_for_all_threads

Philippe Hanrigou’s caller_for_all_threads patches have been integrated into REE. This feature allows one to dump the stack trace of all running threads in a Ruby application. This feature is a must-have for debugging multithreaded web applications.

Phusion Passenger’s development version has already been updated to take advantage of this feature, when available. It will be available starting from Phusion Passenger version 2.1.0 (which hasn’t been released yet at the moment of writing).

RubyGems updated to version 1.3.1.

The previous REE release ships with RubyGems 1.2. Version 1.3.1 is now included. FYI: Ruby on Rails 2.2 requires RubyGems 1.3.1.

REE’s RubyGems no longer makes use of the existing gems

One of the most often requested features is that REE’s RubyGems should be able to use the gems that are already installed (i.e. the gems that have been installed for the system’s Ruby interpreter). The previous REE release implemented this feature.

However, it turns out that this can cause problems with native extensions, e.g. RMagick and the Ruby MySQL library. If one uses REE to load a native extension that was originally compiled for the system’s Ruby, then REE might crash. So this feature has been removed. You should install all gems that you need with REE’s RubyGems.

ree-version

An ‘ree-version’ command is now provided. This command prints out the REE version number. This makes it easier for people to track REE updates.

The sponsors

Many thanks to the following people and organizations for sponsoring this release (list sorted in alphabetical order):

Officially tested platforms

We’ve tested this release of Ruby Enterprise Edition on the following platforms:

Ubuntu Linux 8.04 Desktop (x86)
Ubuntu Linux 8.10 Server (x86_64)
Mac OS X 10.5.5 Server (x86)
FreeBSD 5.0 (x86)
FreeBSD 7.0 (x86_64)
Sun Solaris (Sparc)

Download & upgrade

To install Ruby Enterprise Edition, please visit the download page. To upgrade from a previous version, simply install into the same prefix that you installed to last time.

Written by Hongli Lai on December 5, 2008 Ruby Enterprise Edition Comments (33)

33 Comments »

We had a MySQL problem earlier so we lost some comments. But to answer the people who asked why they get a 404 Not Found error on the download page: it’s because RubyForge is still updating their mirrors.

Comment by hongli — December 5, 2008 @ 11:30 am

You mention better 64-bit support. does that mean the memory allocator now works on 64-bit platforms?

http://www.rubyenterpriseedition.com/faq.html#thirty_three_percent_mem_reduction
“this change the FAQ answer of “this memory allocator does not work on 64-bit platforms. As a result, on 64-bit platforms, Ruby Enterprise Edition is slightly slower than standard Ruby (by a few percent). “

Comment by Aaron Batalion — December 5, 2008 @ 11:41 am

No, tcmalloc doesn’t work on 64-bit. But the copy-on-write patches still do, they just aren’t as effective without tcmalloc. So on 32-bit systems (with tcmalloc) you get about 33% memory savings, while on 64-bit systems (without tcmalloc) you get about 25% memory savings.

Comment by hongli — December 5, 2008 @ 11:43 am

Irony: you posted this literally 20 minutes after I finished installing passenger and the previous ruby-enterprise. Guess I should have taken the time to drink my coffee this morning after all Thanks for your efforts!

Comment by Keith — December 5, 2008 @ 11:46 am

hongli, 25% is great…. but thats memory savings.. but the FAQ mentions performance… “slower”. Can you elaborate? Is that still the case? Beyond tcmalloc, what makes the 64bit slower?

Comment by Aaron Batalion — December 5, 2008 @ 12:07 pm

Yes. The copy-on-write patch makes the GC about 5% slower, but this is negated by the fact that tcmalloc makes everything about 25% faster. So in summary:

On 32-bit:
- 5% slowdown because of GC patches
- 25% speedup because of tcmalloc
- Net result: 20% speedup.

On 64-bit:
- 5% slowdown because of GC patches
- No tcmalloc available
- Net result: 5% slowdown.

Comment by hongli — December 5, 2008 @ 12:21 pm

[...] customers the highest performance virtual machines and access to large memory spaces when needed. Just today Phusion has introduced a version that should fix these issues. Before *today*, Engine Yard *could not* support this combination, it’s just that simple, no [...]

Pingback by Apache + Passenger -vs- nginx + mongrel « Tom Mornini — December 5, 2008 @ 2:55 pm

This sounds like you are saying stay away if you using 64 bit linux.

Comment by bryanl — December 5, 2008 @ 3:59 pm

Awesome.. Thanks!

Quick Q: are there 64bit ubuntu/deb packages planned somewhere in the future?

Comment by Joerg Battermann — December 5, 2008 @ 4:05 pm

[...] new 1.8.6-20081205 version has some interesting changes. First of all, the tcmalloc memory allocator now works with Mac OS X. And second of all, it has [...]

Pingback by How REE and GC tuning reduced spec suite runtime to one third « Enjoying Rails — December 5, 2008 @ 4:51 pm

@bryanl: Well no, it just means that the tradeoff is different. If you’re on 64-bit, and you value memory usage more than raw CPU usage, then REE is the way to go.

Comment by hongli — December 5, 2008 @ 5:18 pm

Awesome release, I am installing it right now And actually – I may be wrong about this – but it seems that Apache performance is worse on 64-bits. It seems related to the Linux kernel’s scheduler. For highly demanded web servers, it is better to use Linux 32-bits and if you need more RAM, use just PAE. Apache + Linux will perform better in 32-bit settings.

Comment by AkitaOnRails — December 5, 2008 @ 6:53 pm

If you value memory consumption in general, you won’t use 64-bit linux for your app servers anyways. Ruby consumes much more memory on 64-bit platforms, and there isn’t any advantage to running on 64-bit.

Comment by Wesley Moxam — December 5, 2008 @ 11:48 pm

Hi, ruby enterprise edition does not compile on Ubuntu Jaunty, using g++-4.3 or g++-4.2. Any info on that?

Comment by Jonathan — December 6, 2008 @ 12:20 am

[...] Ruby Enterprise Edition 1.8.6-20081205 released [...]

Pingback by Joakim Andersson » Blog Archive » Ruby Enterprise Edition 1.8.6-20081205 released — December 6, 2008 @ 8:56 am

I’d suggest submitting the ‘gc tweakability’ patches upstream–other projects would surely benefit, and I’d +1 them
Or maybe run time tweakable would be better…hmm.

Comment by roger — December 6, 2008 @ 10:35 am

@roger: Those patches had in fact been submitted upstream, but the Ruby core team didn’t accept them.

Comment by hongli — December 6, 2008 @ 10:49 am

I can confirm that Ruby-Enterprise & Passenger works well on:

CentOS 5.2 x86

Comment by Gregg — December 6, 2008 @ 6:30 pm

But it does not work on

CentOS 5.2 x86_64 *exits after make during installation*

I’m surprised that there are no Red Hat flavored distros on that test list

Comment by Gregg — December 6, 2008 @ 6:33 pm

[...] numerous posts about Ruby Enterprise Edition and tcmalloc on a 64-bit platform, but Phusion have said that it still won’t build. Well, I made some simple changes to the installer and Ruby [...]

Pingback by Ruby Enterprise Edition on Ubuntu 8.10 (x86_64) - sethbc’s (infrequent) blog… — December 7, 2008 @ 9:23 pm

Have the all thread backtraces been posted upstream? That might be nice. Also do you have a link to the GC patches’ upstream post?
Thanks!
-=R

Comment by roger — December 8, 2008 @ 2:35 am

At line #73 in lib/passenger/spawn_manager.rb:
# Commonly used libraries. ['mysql', 'sqlite3'].each do |lib| require lib end

WTF????
Can you explain the presence of this piece of code? I had to manually remove it since I use Postgres and I couldn’t install
the Mysql gem!

Also:
$ sudo passenger-memory-stats /opt/ruby-enterprise-1.8.6-20081205/lib/ruby/gems/1.8/gems/passenger-2.0.5/bin/passenger-memory-stats:150:in `list_processes': Invalid options. (ArgumentError) from /opt/ruby-enterprise-1.8.6-20081205/lib/ruby/gems/1.8/gems/passenger-2.0.5/bin/passenger-memory-stats:112:in `start' from /opt/ruby-enterprise-1.8.6-20081205/lib/ruby/gems/1.8/gems/passenger-2.0.5/bin/passenger-memory-stats:236 from /opt/ruby-ee/bin/passenger-memory-stats:19:in `load' from /opt/ruby-ee/bin/passenger-memory-stats:19

$ uname -a
Linux **** 2.6.18-xen #1 SMP Tue Feb 12 06:40:50 UTC 2008 x86_64 x86_64 x86_64 GNU/Linux

Comment by ste — December 9, 2008 @ 3:24 pm

ste: Please calm down and stay polite, there’s no reason to get upset.

The code you pointed out is meant for improving copy-on-write efficiency, as the comment a few lines above your code piece explains:

# Preload libraries for copy-on-write semantics.

It doesn’t take into account that the library may be missing, but this has already been fixed.

Comment by hongli — December 9, 2008 @ 3:31 pm

Sorry for sounding impolite (and I assure you that I’m *always* calm , but this thing really caught me by surprise. Knowing the quality of your work, I find this “bug” really surprising, in a VERY bad way.

Comment by ste — December 9, 2008 @ 4:04 pm

Per the instructions in “REE’s RubyGems no longer makes use of the existing gems”, I installed the MySQL gem with:

/opt/ruby-enterprise-1.8.6-20081205/bin/ruby /opt/ruby-enterprise-1.8.6-20081205/bin/gem install mysql — –with-mysql-dir=/usr/local/mysql

But I am getting the following error in /var/log/apache2/error_log:

[Tue Dec 09 22:14:28 2008] [notice] Apache/2.2.9 (Unix) mod_ssl/2.2.9 OpenSSL/0.9.7l DAV/2 Phusion_Passenger/2.0.5 configured — resuming normal operations
dyld: NSLinkModule() error
dyld: Library not loaded: /usr/local/mysql/lib/mysql/libmysqlclient.15.dylib
Referenced from: /opt/ruby-enterprise-1.8.6-20081205/lib/ruby/gems/1.8/gems/mysql-2.7/lib/mysql.bundle
Reason: image not found

I am running Leopard 10.5.5 and a local build of MySQL 5.0.45.

Thanks for your help. This is the first time I have tried to get this running.

Comment by RyanR> — December 10, 2008 @ 1:47 am

[...] versão 1.8.6-20081205 do REE acaba de ser lançada, com melhor suporte ao Mac OS, Solaris e plataformas de 64 bits. Você pode pegar a sua cópia [...]

Pingback by Ruby Enterprise Edition versão 1.8.6-20081205 | Ruby Brasil — December 10, 2008 @ 8:22 am

Update to previous comment: I tried correcting the reference in mysql.bundle to reflect the correct location of the mysqlclient.15.dylib file, and I got seg faults instead.

Comment by RyanR> — December 10, 2008 @ 11:19 am

@ryanr — could try using it –with_mysql_config or the like
GL!
-=R

Comment by roger — December 11, 2008 @ 8:29 am

Wondering–which part(s) of the gcpatch were applied?
Thanks!
-=R

Comment by roger — December 11, 2008 @ 12:08 pm

I’m just wondering, why is it assumed by REE developers that reinstalling all Ruby Gems on a system is ok? I have over 50 gems used across 4 Rails applications, and this needs to be done on 5 servers.

I was hoping to use GEM_PATH to continue using the already-installed gems.

Comment by Just Wondering — December 12, 2008 @ 1:16 pm

@Just Wondering: Because it is not possible to use your existing gems reliably. We tried this in the past and it turns out that on some platforms, native extensions compiled against the existing Ruby installation can cause REE to crash, and the only way to prevent this is by reinstalling said gems. If there is a silver bullet we would have used it a long time ago.

Comment by hongli — December 12, 2008 @ 4:14 pm

still wondering…Wondering–which part(s) of the gcpatch were applied?

Thanks!
-=R

Comment by roger — December 17, 2008 @ 1:31 pm

@roger: Everything.

Comment by hongli — December 17, 2008 @ 2:32 pm

We had a MySQL problem earlier so we lost some comments. But to answer the people who asked why they get a 404 Not Found error on the download page: it’s because RubyForge is still updating their mirrors.

Comment by hongli — December 5, 2008 @ 11:30 am
You mention better 64-bit support. does that mean the memory allocator now works on 64-bit platforms?

http://www.rubyenterpriseedition.com/faq.html#thirty_three_percent_mem_reduction
“this change the FAQ answer of “this memory allocator does not work on 64-bit platforms. As a result, on 64-bit platforms, Ruby Enterprise Edition is slightly slower than standard Ruby (by a few percent). “

Comment by Aaron Batalion — December 5, 2008 @ 11:41 am
No, tcmalloc doesn’t work on 64-bit. But the copy-on-write patches still do, they just aren’t as effective without tcmalloc. So on 32-bit systems (with tcmalloc) you get about 33% memory savings, while on 64-bit systems (without tcmalloc) you get about 25% memory savings.

Comment by hongli — December 5, 2008 @ 11:43 am
Irony: you posted this literally 20 minutes after I finished installing passenger and the previous ruby-enterprise. Guess I should have taken the time to drink my coffee this morning after all Thanks for your efforts!

Comment by Keith — December 5, 2008 @ 11:46 am
hongli, 25% is great…. but thats memory savings.. but the FAQ mentions performance… “slower”. Can you elaborate? Is that still the case? Beyond tcmalloc, what makes the 64bit slower?

Comment by Aaron Batalion — December 5, 2008 @ 12:07 pm
Yes. The copy-on-write patch makes the GC about 5% slower, but this is negated by the fact that tcmalloc makes everything about 25% faster. So in summary:

On 32-bit:
- 5% slowdown because of GC patches
- 25% speedup because of tcmalloc
- Net result: 20% speedup.

On 64-bit:
- 5% slowdown because of GC patches
- No tcmalloc available
- Net result: 5% slowdown.

Comment by hongli — December 5, 2008 @ 12:21 pm
[...] customers the highest performance virtual machines and access to large memory spaces when needed. Just today Phusion has introduced a version that should fix these issues. Before *today*, Engine Yard *could not* support this combination, it’s just that simple, no [...]

Pingback by Apache + Passenger -vs- nginx + mongrel « Tom Mornini — December 5, 2008 @ 2:55 pm
This sounds like you are saying stay away if you using 64 bit linux.

Comment by bryanl — December 5, 2008 @ 3:59 pm
Awesome.. Thanks!

Quick Q: are there 64bit ubuntu/deb packages planned somewhere in the future?

Comment by Joerg Battermann — December 5, 2008 @ 4:05 pm
[...] new 1.8.6-20081205 version has some interesting changes. First of all, the tcmalloc memory allocator now works with Mac OS X. And second of all, it has [...]

Pingback by How REE and GC tuning reduced spec suite runtime to one third « Enjoying Rails — December 5, 2008 @ 4:51 pm
@bryanl: Well no, it just means that the tradeoff is different. If you’re on 64-bit, and you value memory usage more than raw CPU usage, then REE is the way to go.

Comment by hongli — December 5, 2008 @ 5:18 pm
Awesome release, I am installing it right now And actually – I may be wrong about this – but it seems that Apache performance is worse on 64-bits. It seems related to the Linux kernel’s scheduler. For highly demanded web servers, it is better to use Linux 32-bits and if you need more RAM, use just PAE. Apache + Linux will perform better in 32-bit settings.

Comment by AkitaOnRails — December 5, 2008 @ 6:53 pm
If you value memory consumption in general, you won’t use 64-bit linux for your app servers anyways. Ruby consumes much more memory on 64-bit platforms, and there isn’t any advantage to running on 64-bit.

Comment by Wesley Moxam — December 5, 2008 @ 11:48 pm
Hi, ruby enterprise edition does not compile on Ubuntu Jaunty, using g++-4.3 or g++-4.2. Any info on that?

Comment by Jonathan — December 6, 2008 @ 12:20 am
[...] Ruby Enterprise Edition 1.8.6-20081205 released [...]

Pingback by Joakim Andersson » Blog Archive » Ruby Enterprise Edition 1.8.6-20081205 released — December 6, 2008 @ 8:56 am
I’d suggest submitting the ‘gc tweakability’ patches upstream–other projects would surely benefit, and I’d +1 them
Or maybe run time tweakable would be better…hmm.

Comment by roger — December 6, 2008 @ 10:35 am
@roger: Those patches had in fact been submitted upstream, but the Ruby core team didn’t accept them.

Comment by hongli — December 6, 2008 @ 10:49 am
I can confirm that Ruby-Enterprise & Passenger works well on:

CentOS 5.2 x86

Comment by Gregg — December 6, 2008 @ 6:30 pm
But it does not work on

CentOS 5.2 x86_64 *exits after make during installation*

I’m surprised that there are no Red Hat flavored distros on that test list

Comment by Gregg — December 6, 2008 @ 6:33 pm
[...] numerous posts about Ruby Enterprise Edition and tcmalloc on a 64-bit platform, but Phusion have said that it still won’t build. Well, I made some simple changes to the installer and Ruby [...]

Pingback by Ruby Enterprise Edition on Ubuntu 8.10 (x86_64) - sethbc’s (infrequent) blog… — December 7, 2008 @ 9:23 pm
Have the all thread backtraces been posted upstream? That might be nice. Also do you have a link to the GC patches’ upstream post?
Thanks!
-=R

Comment by roger — December 8, 2008 @ 2:35 am
At line #73 in lib/passenger/spawn_manager.rb:
# Commonly used libraries. ['mysql', 'sqlite3'].each do |lib| require lib end

WTF????
Can you explain the presence of this piece of code? I had to manually remove it since I use Postgres and I couldn’t install
the Mysql gem!

Also:
$ sudo passenger-memory-stats /opt/ruby-enterprise-1.8.6-20081205/lib/ruby/gems/1.8/gems/passenger-2.0.5/bin/passenger-memory-stats:150:in `list_processes': Invalid options. (ArgumentError) from /opt/ruby-enterprise-1.8.6-20081205/lib/ruby/gems/1.8/gems/passenger-2.0.5/bin/passenger-memory-stats:112:in `start' from /opt/ruby-enterprise-1.8.6-20081205/lib/ruby/gems/1.8/gems/passenger-2.0.5/bin/passenger-memory-stats:236 from /opt/ruby-ee/bin/passenger-memory-stats:19:in `load' from /opt/ruby-ee/bin/passenger-memory-stats:19

$ uname -a
Linux **** 2.6.18-xen #1 SMP Tue Feb 12 06:40:50 UTC 2008 x86_64 x86_64 x86_64 GNU/Linux

Comment by ste — December 9, 2008 @ 3:24 pm
ste: Please calm down and stay polite, there’s no reason to get upset.

The code you pointed out is meant for improving copy-on-write efficiency, as the comment a few lines above your code piece explains:
```
# Preload libraries for copy-on-write semantics.
```
It doesn’t take into account that the library may be missing, but this has already been fixed.

Comment by hongli — December 9, 2008 @ 3:31 pm
Sorry for sounding impolite (and I assure you that I’m *always* calm , but this thing really caught me by surprise. Knowing the quality of your work, I find this “bug” really surprising, in a VERY bad way.

Comment by ste — December 9, 2008 @ 4:04 pm
Per the instructions in “REE’s RubyGems no longer makes use of the existing gems”, I installed the MySQL gem with:

/opt/ruby-enterprise-1.8.6-20081205/bin/ruby /opt/ruby-enterprise-1.8.6-20081205/bin/gem install mysql — –with-mysql-dir=/usr/local/mysql

But I am getting the following error in /var/log/apache2/error_log:

[Tue Dec 09 22:14:28 2008] [notice] Apache/2.2.9 (Unix) mod_ssl/2.2.9 OpenSSL/0.9.7l DAV/2 Phusion_Passenger/2.0.5 configured — resuming normal operations
dyld: NSLinkModule() error
dyld: Library not loaded: /usr/local/mysql/lib/mysql/libmysqlclient.15.dylib
Referenced from: /opt/ruby-enterprise-1.8.6-20081205/lib/ruby/gems/1.8/gems/mysql-2.7/lib/mysql.bundle
Reason: image not found

I am running Leopard 10.5.5 and a local build of MySQL 5.0.45.

Thanks for your help. This is the first time I have tried to get this running.

Comment by RyanR> — December 10, 2008 @ 1:47 am
[...] versão 1.8.6-20081205 do REE acaba de ser lançada, com melhor suporte ao Mac OS, Solaris e plataformas de 64 bits. Você pode pegar a sua cópia [...]

Pingback by Ruby Enterprise Edition versão 1.8.6-20081205 | Ruby Brasil — December 10, 2008 @ 8:22 am
Update to previous comment: I tried correcting the reference in mysql.bundle to reflect the correct location of the mysqlclient.15.dylib file, and I got seg faults instead.

Comment by RyanR> — December 10, 2008 @ 11:19 am
@ryanr — could try using it –with_mysql_config or the like
GL!
-=R

Comment by roger — December 11, 2008 @ 8:29 am
Wondering–which part(s) of the gcpatch were applied?
Thanks!
-=R

Comment by roger — December 11, 2008 @ 12:08 pm
I’m just wondering, why is it assumed by REE developers that reinstalling all Ruby Gems on a system is ok? I have over 50 gems used across 4 Rails applications, and this needs to be done on 5 servers.

I was hoping to use GEM_PATH to continue using the already-installed gems.

Comment by Just Wondering — December 12, 2008 @ 1:16 pm
@Just Wondering: Because it is not possible to use your existing gems reliably. We tried this in the past and it turns out that on some platforms, native extensions compiled against the existing Ruby installation can cause REE to crash, and the only way to prevent this is by reinstalling said gems. If there is a silver bullet we would have used it a long time ago.

Comment by hongli — December 12, 2008 @ 4:14 pm
still wondering…Wondering–which part(s) of the gcpatch were applied?

Thanks!
-=R

Comment by roger — December 17, 2008 @ 1:31 pm
@roger: Everything.

Comment by hongli — December 17, 2008 @ 2:32 pm

RSS feed for comments on this post.

TrackBack URL

Phusion, The Computer Science Company