Does Rails Performance Need an Overhaul?
Igvita.com has recently published the article Rails Performance Needs an Overhaul. Rails performance… no, Ruby performance… no Rails scalability… well something is being criticized here. From my experience, talking about scalability and performance can be a bit confusing because the terms can mean different things to different people and/or in different situations, yet the meanings are used interchangeably all the time. In this post I will take a closer look at Igvita’s article.
Performance vs scalability
Let us first define performance and scalability. I define performance as throughput; number of requests per second. I define scalability as the amount of users a system can concurrently handle. There is a correlation between performance and scalability. Higher performance means each request takes less time, and so is more scalable, right? Sometimes yes, but not necessarily. It is entirely possible for a system to be scalable, yet manages to have a lower throughput than a system that’s not as scalable, or for a system to be uber-fast yet not very scalable. Throughout this blog post I will show several examples that highlight the difference.
“Scalability” is an extremely loaded word and people often confuse it with “being able to handle tons and tons of traffic”. Let’s use a different term that better reflects what Igvita’s actually criticizing: concurrency. Igvita claims that concurrency in Ruby is pathetic while referring to database drivers, Ruby application servers, etc. Some practical examples that demonstrate what he means are as follows.
Limited concurrency at the app server level
Mongrel, Phusion Passenger and Unicorn all use a “traditional” multi-process model in which multiple Ruby processes are spawned, each process handling a single request per second. Thus, concurrency is (assuming that the load balancer has infinite concurrency) limited by the number of Ruby processes: having 5 processes allow you to handle 5 users concurrently.
Threaded servers, where the server spawns multiple threads, each handling 1 connection concurrently, allow more concurrency because because it’s possible to spawn a whole lot more threads than processes. In the context of Ruby, each Ruby process needs to load its own copy of the application code and other resources, so memory increases very quickly as you spawn additional processes. Phusion Passenger with Ruby Enterprise Edition solves this problem somewhat by using copy-on-write optimizations which save memory, so you can spawn a bit more processes, but not significantly (as in 10x) more. In contrast, a multi-threaded app server does not need as much memory because all threads share application code with each other so you can comfortably spawn tens or hundreds of threads. At least, this is the theory. I will later explain why this does not necessarily hold for Ruby.
When it comes to performance however, there’s no difference between processes and threads. If you compare a well-written multi-threaded app server with 5 threads to a well-written multi-process app server with 5 processes, you won’t find either being more performant than the other. Context switch overhead between processes and threads are roughly the same. Each process can use a different CPU core, as can each thread, so there’s no difference in multi-core utilization either. This reflects back on the difference between scalability/concurrency and performance.
Multi-process Rails app servers have a concurrency level that can be counted with a single hand, or if you have very beefy hardware, a concurrency level in the range of a couple of tens, thanks to the fact that Rails needs about 25 MB per process. Multi-threaded Rails app servers can in theory spawn a couple of hundred of threads. After that it’s also game over: an operating system thread needs a couple MB of stack space, so after a couple hundreds of threads you’ll run out of virtual memory address on 32-bit systems even if you don’t actually use that much memory.
There is another class of servers, the evented ones. These servers are actually single-threaded, but they use a reactor style I/O dispatch architecture for handling I/O concurrency. Examples include Node.js, Thin (built on EventMachine) and Tornado. These servers can easily have a concurrency level of a couple of thousand. But due to their single-threaded nature they cannot effectively utilize multiple CPU cores, so you need to run a couple of processes, one per CPU core, to fully utilize your CPU.
The limits of Ruby threads
Ruby 1.8 uses userspace threads, not operating system threads. This means that Ruby 1.8 can only utilize a single CPU core no matter how many Ruby threads you create. This is why one typically needs multiple Ruby processes to fully utilize one’s CPU cores. Ruby 1.9 finally uses operating system threads, but it has a global interpreter lock, which means that each time a Ruby 1.9 thread is running it will prevent other Ruby threads from running, effectively making it the same multicore-wise as 1.8. This is also explained in an earlier Igvita article, Concurrency is a Myth in Ruby.
On the bright side, not all is bad. Ruby 1.8 internally uses non-blocking I/O while Ruby 1.9 unlocks the global interpreter lock while doing I/O. So if one Ruby thread is blocked on I/O, another Ruby thread can continue execution. Likewise, Ruby is smart enough to cause things like sleep() and even waitpid() to preempt to other threads.
On the dark side however, Ruby internally uses the select() system call for multiplexing I/O. select() can only handle 1024 file descriptors on most systems so Ruby cannot handle more than this number of sockets per Ruby process, even if you are somehow able to spawn thousands of Ruby threads. EventMachine works around this problem by bypassing Ruby’s I/O code completely.
Naive native extensions and third party libraries
So just run a couple of multi-threaded Ruby processes, one process per core and multiple threads per process, and all is fine and we should be able to have a concurrency level of up to a couple hundred, right? Well not quite, there are a number of issues hindering this approach:
- Some third party libraries and Rails plugins are not thread-safe. Some aren’t even reentrant. For example Rails < 2.2 suffered from this problem. The app itself might not be thread-safe.
- Although Ruby is smart enough not to let I/O block all threads, the same cannot be said of all native extensions. The MySQL extension is the most infamous example: when executing queries, other threads cannot run.
Mongrel is actually multi-threaded but in practice everybody uses in multi-process mode (mongrel_cluster) exactly because of these problems. It is also the reason why Phusion Passenger has also gone the multi-process route.
And even though Thin is evented, a typical Ruby web application running on Thin cannot handle thousands of concurrent users. This is because evented servers typically require a special evented programming style, such as the one seen in Node.js and EventMachine. A Ruby web app that is written in an evented style running on Thin can definitely handle a large number of concurrent users.
When is limited application server concurrency actually a problem?
Igvita is clearly disappointed at all all the issues that hinder Ruby web apps from achieving high concurrency. For many web applications I would however argue that limited concurrency is not a problem.
- Web applications that are slow, as in CPU-heavy, max out CPU resources pretty quickly so increasing concurrency won’t help you.
- Web applications that are fast are typically quick enough at handling the load so that even large number of users won’t notice the limited concurrency of the server.
Having a concurrency of 5 does not mean not mean that the app server can only handle 5 requests per second; it’s not hard to serve hundreds of requests per second with only a couple of single-threaded processes.
The problem becomes most evident for web applications that have to wait a lot for I/O (besides its own HTTP request/response cycle). Examples include:
- Apps that have to spend a lot of time waiting on the database.
- Apps that perform a lot of external HTTP calls that respond slowly.
- Chat apps. These apps typically have thousands of users, most of them doing nothing most of the time, but they all require a connection (unless your app uses polling, but that’s a whole different discussion).
We at Phusion have developed a number of web applications for clients that fall in the second category, the most recent one being a Hyves gadget. Hyves is the most popular social network in the Netherlands and they get thousands of concurrent visitors during the day. The gadget that we’ve developed has to query external HTTP servers very often, and these servers can take 10 seconds to respond in extreme cases. The servers are running Phusion Passenger with maybe a couple tens of processes. If every request to our gadget also causes us to wait 10 seconds for the external HTTP call then we’d soon run out of concurrency.
But even suppose that our app and Phusion Passenger can have a concurrency level of a couple of thousand, all of those visitors will still have to wait 10 seconds for the external HTTP calls, which is obviously unacceptable. This is another example that illustrates the difference between scalability and performance. We had solved this problem by aggressively caching the results of the HTTP calls, minimizing the number of external HTTP calls that are necessary. The result is that even though the application’s concurrency is fairly limited, it can still comfortably serve many concurrent users with a reasonable response time.
This anecdote should explain why I believe that web apps can get very far despite having a limited concurrency level. That said, as Internet usage continues to increase and websites get more and more users, we may at some time come to a point where much a larger concurrency level is required than most of our current Ruby tools allow us to (assuming server capacity doesn’t scale quickly enough).
What was Igvita.com criticizing?
Igvita.com does not appear to be criticizing Ruby or Rails for being slow. It doesn’t even appear to be criticizing the lack of Ruby tools for achieving high concurrency. It appears to be criticizing these things:
- Rails and most Ruby web application servers don’t allow high concurrency by default.
- Many database drivers and libraries hinder concurrency.
- Although alternatives exist that allow concurrency, you have to go out of your way to find them.
- There appears to be little motivation in the Ruby community for making the entire stack of web frame work + web app server + database drivers etc scalable by default.
This is in contrast to Node.js where everything is scalable by default.
Do I understand Igvita’s frustration? Absolutely. Do I agree with it? Not entirely. The same thing that makes Node.js so scalable is also what makes it relatively hard to program for. Node.js enforces a callback style of programming and this can eventually make your code look a lot more complicated and harder to read than regular code that uses blocking calls. Furthermore, Node.js is relatively young – of course you won’t find any Node.js libraries that don’t scale! But if people ever use Node.js for things other than high-concurrency servers apps, then non-scalable libraries will at some time pop up. And then you will have to look harder to avoid these libraries. There is no silver bullet.
That said, all would be well if at least the preferred default stack can handle high concurrency by default. This means e.g. fixing the MySQL extension and have the fix published by upstream. The mysqlplus extension fixes this but for some reason their changes aren’t accepted and published by the original author, and so people end up with a multi-thread-killing database driver by default.
Is Node.js innovative? Is Ruby lacking innovation?
A minor gripe that I have with the article is that Igvita calls Node.js innovative while seemingly implying that the Ruby stack isn’t innovating. Evented servers like Node.js actually have been around for years and the evented pattern is well-known long before Ruby or Javascript have become popular. Thin is also evented and predates Node.js by several years. Thin and EventMachine also allow Node.js-style evented programming. The only innovation that Node.js brings, in my opinion, is the fact that it’s Javascript. The other “innovation” is the lack of non-scalable libraries.
Conclusion
Igvita appears to be criticizing something other than Rails performance, as his article’s title would imply.
I don’t think the concurrency levels that the Rails stack provides by default is that bad in practice. But as a fellow programmer, it does intuitively bother me that our laptops, which are a million times more powerful than supercomputers from two decades ago, cannot comfortably handle a couple of thousand concurrent users. We can definitely work towards something better, but in the mean time let’s not forget that the current stack is more than capable of Getting Work Done(tm).
Announcing EncryptedCookieStore plugin for Rails 2.3
EncryptedCookieStore is similar to Ruby on Rails’s CookieStore (it saves session data in a cookie), but it uses encryption so that people can’t read what’s in the session data. This makes it possible to store sensitive data in the session.
EncryptedCookieStore is written for Rails 2.3. Other versions of Rails have not been tested.
Note: This is not ThinkRelevance’s EncryptedCookieStore. In the Rails 2.0 days they wrote an EncryptedCookieStore, but it seems their repository had gone defunct and their source code lost. This EncryptedCookieStore is written from scratch by Phusion.
Source code at http://github.com/FooBarWidget/encrypted_cookie_store
Installation and usage
First, install it:
./script/plugin install git://github.com/FooBarWidget/encrypted_cookie_store.git
Then edit config/initializers/session_store.rb and set your session store to EncryptedCookieStore:
ActionController::Base.session_store = EncryptedCookieStore
You need to set a few session options before EncryptedCookieStore is usable. You must set all options that CookieStore needs, plus an encryption key that EncryptedCookieStore needs. In session_store.rb:
ActionController::Base.session = {
# CookieStore options...
:key => '_session', # Name of the cookie which contains the session data.
:secret => 'b4589cc9...', # A secret string used to generate the checksum for
# the session data. Must be longer than 64 characters
# and be completely random.
# EncryptedCookieStore options...
:encryption_key => 'c306779f3...', # The encryption key. See below for notes.
}
The encryption key must be a hexadecimal string of exactly 32 bytes. It should be entirely random, because otherwise it can make the encryption weak.
You can generate a new encryption key by running rake secret:encryption_key. This command will output a random encryption key that you can then copy and paste into your environment.rb.
Operational details
Upon generating cookie data, EncryptedCookieStore generates a new, random initialization vector for encrypting the session data. This initialization vector is then encrypted with 128-bit AES in ECB mode. The session data is first protected with an HMAC to prevent tampering. The session data, along with the HMAC, are then encrypted using 256-bit AES in CFB mode with the generated initialization vector. This encrypted session data + HMAC are then stored, along with the encrypted initialization vector, into the cookie.
Upon unmarshalling the cookie data, EncryptedCookieStore decrypts the encrypted initialization vector and use that to decrypt the encrypted session data + HMAC. The decrypted session data is then verified against the HMAC.
The reason why HMAC verification occurs after decryption instead of before decryption is because we want to be able to detect changes to the encryption key and changes to the HMAC secret key, as well as migrations from CookieStore. Verifying after decryption allows us to automatically invalidate such old session cookies.
EncryptedCookieStore is quite fast: it is able to marshal and unmarshal a simple session object 5000 times in 8.7 seconds on a MacBook Pro with a 2.4 Ghz Intel Core 2 Duo (in battery mode). This is about 0.174 ms per marshal+unmarshal action. See rake benchmark in the EncryptedCookieStore sources for details.
EncryptedCookieStore vs other session stores
EncryptedCookieStore inherits all the benefits of CookieStore:
- It works out of the box without the need to setup a seperate data store (e.g. database table, daemon, etc).
- It does not require any maintenance. Old, stale sessions do not need to be manually cleaned up, as is the case with PStore and ActiveRecordStore.
- Compared to MemCacheStore, EncryptedCookieStore can “hold” an infinite number of sessions at any time.
- It can be scaled across multiple servers without any additional setup.
- It is fast.
- It is more secure than CookieStore because it allows you to store sensitive data in the session.
There are of course drawbacks as well:
- It is prone to session replay attacks. These kind of attacks are explained in the Ruby on Rails Security Guide. Therefore you should never store anything along the lines of is_admin in the session.
- You can store at most a little less than 4 KB of data in the session because that’s the size limit of a cookie. “A little less” because EncryptedCookieStore also stores a small amount of bookkeeping data in the cookie.
- Although encryption makes it more secure than CookieStore, there’s still a chance that a bug in EncryptedCookieStore renders it insecure. We welcome everyone to audit this code. There’s also a chance that weaknesses in AES are found in the near future which render it insecure. If you are storing *really* sensitive information in the session, e.g. social security numbers, or plans for world domination, then you should consider using ActiveRecordStore or some other server-side store.
JRuby: Illegal Key Size error
If you get this error (and your code works with MRI)…
Illegal key size
[...]/vendor/plugins/encrypted_cookie_store/lib/encrypted_cookie_store.rb:62:in `marshal'
…then it probably means you don’t have the “unlimited strength” policy files installed for your JVM. Download and install them. You probably have the “strong” version if they are already there.
As a workaround, you can change the cipher type from 256-bit AES to 128-bit by
inserting the following in config/initializer/session_store.rb:
EncryptedCookieStore.data_cipher_type = 'aes-128-cfb'.freeze # was 256
Please note that after changing to 128-bit AES, EncryptedCookieStore still requires a 32 bytes hexadecimal encryption key, although only half of the key is actually used.
default_value_for Rails plugin: declaratively define default values for ActiveRecord models
Introduction
The default_value_for plugin allows one to define default values for ActiveRecord models in a declarative manner. For example:
class User < ActiveRecord::Base
default_value_for :name, "(no name)"
default_value_for :last_seen do
Time.now
end
end
u = User.new
u.name # => "(no name)"
u.last_seen # => Mon Sep 22 17:28:38 +0200 2008
We at Phusion use it for generating UUIDs for models.
Note: critics might be interested in the “When (not) to use default_value_for?” section. Please read on.
Installation
Install with:
./script/plugin install git://github.com/FooBarWidget/default_value_for.git
See also the AgileWebDevelopment Plugins entry.
If you like this plugin, then please consider donating and/or recommending us:
Hongli Lai
|
Ninh Bui
|
The default_value_for method
The default_value_for method is available in all ActiveRecord model classes.
The first argument is the name of the attribute for which a default value should be set. This may either be a Symbol or a String.
The default value itself may either be passed as the second argument:
default_value_for :age, 20
…or it may be passed as the return value of a block:
default_value_for :age do
if today_is_sunday?
20
else
30
end
end
If you pass a value argument, then the default value is static and never changes. However, if you pass a block, then the default value is retrieved by calling the block. This block is called not once, but every time a new record is instantiated and default values need to be filled in.
The latter form is especially useful if your model has a UUID column. One can generate a new, random UUID for every newly instantiated record:
class User < ActiveRecord::Base
default_value_for :uuid do
UuidGenerator.new.generate_uuid
end
end
User.new.uuid # => "51d6d6846f1d1b5c9a...."
User.new.uuid # => "ede292289e3484cb88...."
Note that record is passed to the block as an argument, in case you need it for whatever reason:
class User < ActiveRecord::Base
default_value_for :uuid do |x|
x # <--- a User object
UuidGenerator.new.generate_uuid
end
end
Rules
Instantiation of new record
Upon instantiating a new record, the declared default values are filled into the record. You’ve already seen this in the above examples.
Retrieval of existing record
Upon retrieving an existing record, the declared default values are not filled into the record. Consider the example with the UUID:
user = User.create user.uuid # => "529c91b8bbd3e..." user = User.find(user.id) # UUID remains unchanged because it's retrieved from the database! user.uuid # => "529c91b8bbd3e..."
Mass-assignment
If a certain attribute is being assigned via the model constructor’s mass-assignment argument, that the default value for that attribute will not be filled in:
user = User.new(:uuid => "hello") user.uuid # => "hello"
However, if that attribute is protected by attr_protected or attr_accessible, then it will be filled in:
class User < ActiveRecord::Base default_value_for :name, 'Joe' attr_protected :name end user = User.new(:name => "Jane") user.name # => "Joe"
Inheritance
Inheritance works as expected. All default values are inherited by the child
class:
class User < ActiveRecord::Base default_value_for :name, 'Joe' end class SuperUser < User end SuperUser.new.name # => "Joe"
Attributes that aren’t database columns
default_value_for also works with attributes that aren’t database columns. It works with anything for which there’s an assignment method:
# Suppose that your 'users' table only has a 'name' column.
class User < ActiveRecord::Base
default_value_for :name, 'Joe'
default_value_for :age, 20
default_value_for :registering, true
attr_accessor :age
def registering=(value)
@registering = true
end
end
user = User.new
user.age # => 20
user.instance_variable_get('@registering') # => true
Caveats
A conflict can occur if your model class overrides the ‘initialize’ method, because this plugin overrides ‘initialize’ as well to do its job.
class User < ActiveRecord::Base
def initialize # <-- this constructor causes problems
super(:name => 'Name cannot be changed in constructor')
end
end
We recommend you to alias chain your initialize method in models where you use default_value_for:
class User < ActiveRecord::Base
default_value_for :age, 20
def initialize_with_my_app
initialize_without_my_app(:name => 'Name cannot be changed in constructor')
end
alias_method_chain :initialize, :my_app
end
Also, stick with the following rules:
- There is no need to
alias_method_chainyour initialize method in models that don’t usedefault_value_for. - Make sure that
alias_method_chainis called after the lastdefault_value_foroccurance.
When (not) to use default_value_for?
You can also specify default values in the database schema. For example, you can specify a default value in a migration as follows:
create_table :users do |t| t.string :username, :null => false, :default => 'default username' t.integer :age, :null => false, :default => 20 t.timestamp :last_seen, :null => false, :default => Time.now end
This has the same effect as passing the default value as the second argument to default_value_for:
user = User.new user.username # => 'default username' user.age # => 20 user.timestamp # => Mon Sep 22 18:31:47 +0200 2008
It’s recommended that you use this over default_value_for whenever possible.
However, it’s not possible to specify a schema default for serialized columns. With default_value_for, you can:
class User < ActiveRecord::Base serialize :color default_value_for :color, [255, 0, 0] end
And if schema defaults don’t provide the flexibility that you need, then default_value_for is the perfect choice. For example, with default_value_for you could specify a per-environment default:
class User < ActiveRecord::Base
if RAILS_ENV == "development"
default_value_for :is_admin, true
end
end
Or, as you’ve seen in an earlier example, you can use default_value_for to generate a default random UUID:
class User < ActiveRecord::Base
default_value_for :uuid do
UuidGenerator.new.generate_uuid
end
end
Or you could use it to generate a timestamp that’s relative to the time at which the record is instantiated:
class User < ActiveRecord::Base
default_value_for :account_expires_at do
3.years.from_now
end
end
User.new.account_expires_at # => Mon Sep 22 18:43:42 +0200 2008
sleep(2)
User.new.account_expires_at # => Mon Sep 22 18:43:44 +0200 2008
Finally, it’s also possible to specify a default via an association:
# Has columns: 'name' and 'default_price'
class SuperMarket < ActiveRecord::Base
has_many :products
end
# Has columns: 'name' and 'price'
class Product < ActiveRecord::Base
belongs_to :super_market
default_value_for :price do |product|
product.super_market.default_price
end
end
super_market = SuperMarket.create(:name => 'Albert Zwijn', :default_price => 100)
soap = super_market.products.create(:name => 'Soap')
soap.price # => 100
What about before_validate/before_save?
True, before_validate and before_save does what we want if we’re only interested in filling in a default before saving. However, if one wants to be able to access the default value even before saving, then be prepared to write a lot of code. Suppose that we want to be able to access a new record’s UUID, even before it’s saved. We could end up with the following code:
# In the controller
def create
@user = User.new(params[:user])
@user.generate_uuid
email_report_to_admin("#{@user.username} with UUID #{@user.uuid} created.")
@user.save!
end
# Model
class User < ActiveRecord::Base
before_save :generate_uuid_if_necessary
def generate_uuid
self.uuid = ...
end
private
def generate_uuid_if_necessary
if uuid.blank?
generate_uuid
end
end
end
The need to manually call generate_uuid here is ugly, and one can easily forget to do that. Can we do better? Let’s see:
# Controller
def create
@user = User.new(params[:user])
email_report_to_admin("#{@user.username} with UUID #{@user.uuid} created.")
@user.save!
end
# Model
class User < ActiveRecord::Base
before_save :generate_uuid_if_necessary
def uuid
value = read_attribute('uuid')
if !value
value = generate_uuid
write_attribute('uuid', value)
end
value
end
# We need to override this too, otherwise User.new.attributes won't return
# a default UUID value. I've never tested with User.create() so maybe we
# need to override even more things.
def attributes
uuid
super
end
private
def generate_uuid_if_necessary
uuid # Reader method automatically generates UUID if it doesn't exist
end
end
That’s an awful lot of code. Using default_value_for is easier, don’t you think?
What about other plugins?
I’ve only been able to find 2 similar plugins:
- Default Value: http://agilewebdevelopment.com/plugins/default_value
- ActiveRecord Defaults: http://agilewebdevelopment.com/plugins/activerecord_defaults
Default Value appears to be unmaintained; its SVN link is broken. This leaves only ActiveRecord Defaults. However, it is semantically dubious, which leaves it wide open for corner cases. For example, it is not clearly specified what ActiveRecord Defaults will do when attributes are protected by attr_protected or attr_accessible. It is also not clearly specified what one is supposed to do if one needs a custom initialize method in the model.
I’ve taken my time to thoroughly document default_value_for’s behavior.
Credits
I’ve wanted such functionality for a while now and it baffled me that ActiveRecord doesn’t provide a clean way for me to specify default values. After reading http://groups.google.com/group/rubyonrails-core/browse_thread/thread/b509a2fe2b62ac5/3e8243fa1954a935, it became clear that someone needs to write a plugin. This is the result.
Thanks to Pratik Naik for providing the initial code snippet on which this plugin is based on: http://m.onkey.org/2007/7/24/how-to-set-default-values-in-your-model
If you like this plugin, then please consider donating and/or recommending us:
Hongli Lai
|
Ninh Bui
|
Internationalization features in Rails edge
The development version of Ruby on Rails has cool new internationalization features. Although the framework itself doesn’t provide a lot of I18N functionality, it does provide the necessary hooks for plugins to implement I18N however they see fit. Simon Tokumine has written an I18N demo application to show you what Rails is capable of, when used in combination with the localized_dates plugin.
We’ve deployed the demo application at http://i18n-demo.phusion.nl/. Check it out.
Ruby Enterprise Edition website “launched”
Well hello there.
Since you guys must probably be bored out of your ***** waiting for Ruby Enterprise Edition to be released, let’s play a little play to let the time pass by quicker mmm’kay?
A riddle actually, see http://www.RubyEnterpriseEdition.com for more information
Be sure to scroll down, the next hint will be revealed on May 20th, 2008.
We’re looking forward to reading your submissions
Cheers,
Hongli Lai
Ninh Bui
Passenger (mod_rails) 1.0.5 released
Passenger version 1.0.5 has just been released. Because of time constraints we haven’t had time to write announcements for 1.0.3 and 1.0.4, so here’s a summary of changes in 1.0.3, 1.0.4 and 1.0.5:
- Fixed MacOS X support, this time for real
-
1.0.2 fixed compatibility for MacOS X. Unfortunately, the fix broke 64-bit Linux. 1.0.4 fixed support for 64-bit Linux, but that fix broke MacOS X again.
1.0.5 fixes support for MacOS X *and* 64-bit Linux. It hadn’t been easy to get this right, and we kindly thank all the testers who have helped us with this.
Note to C/C++ developers: if you’re looking for cross-platform file descriptor passing code, please feel free to copy & paste it from Passenger’s source code. Every single code example about file descriptor passing that we could find on the Internet, has portability problems (= works on 64-bit Linux but not 64-bit OS X, or vice versa). We believe that we’ve solved all those portability issues.
- Fixed support for Ruby on Rails 1.2
- Version 1.0.2 broke support for Ruby on Rails 1.2, because it used a feature that only 2.0 and up has. Version 1.0.3 fixed this problem.
How do I upgrade?
Just install it like you did the first time:
gem install passenger
and
passenger-install-apache2-module
Please don’t forget to copy & paste the Apache config snippet that the installer gives you.
At the time of writing, RubyForge is still busy updating all their gem mirror servers. So if gem install passenger didn’t work for you, then please download it from HTTP at: http://rubyforge.org/frs/?group_id=5873
Phusion Passenger (mod_rails) version 1.0.2 released, and more
It has been two weeks since the release of Passenger version 1.0.1. More and more people are switching to Passenger, and most are very pleased with the quality of our initial release.
But in these past 2 weeks, we’ve continued to make improvements to Passenger. So today, we’re happy to announce the release of Passenger version 1.0.2.
Hongli Lai
|
Ninh Bui
|
Featured improvements and changes
- 100% support for MacOS X’s default Apache
-
Passenger has always supported MacOS X. This fact is demonstrated in our screencast, created by Ryan Bates on a Mac. However, there was an inconvenience: Passenger was incompatible with the default Apache installation, as provided by OS X. The installer warned about that. As a result, OS X users had to install Apache via MacPorts or by hand.
But no more. Thanks for the help of Weyert de Boer and the people at Fingertips, we’ve been able to track down the problem. Passenger now fully supports MacOS X’s default Apache! There is no need to install Apache via MacPorts anymore.
- RubyGems-related fixes: Rails < 2.0 is now supported
-
The Passenger gem specifies Rails 2.0 as a dependency. This seemed to be a good idea at the time: Passenger is to be used in combination with Rails, and we figured that by specifying Rails as a dependency, the user will have one command less to type.
But it turned out that some RubyGems versions will load Rails 2.0 during Passenger’s startup, even though Passenger didn’t explicitly tell RubyGems to do that. As a result, some people were having trouble with using Passenger with Rails < version 2.0. This issue has been fixed.
- Memory statistics tool
-
Some people have attempted to analyze Passenger’s memory usage. But standard tools such as ‘top’ and ‘ps’ don’t always report the correct memory usage.
We’ve provided a tool, passenger-memory-stats, which allows people to easily analyze Passenger’s and Apache’s real memory usage. For example:
$ sudo ./bin/passenger-memory-stats ------------- Apache processes -------------- PID PPID Threads VMSize Private Name --------------------------------------------- 5947 1 9 90.6 MB 0.5 MB /usr/sbin/apache2 -k start 5948 5947 1 18.9 MB 0.7 MB /usr/sbin/fcgi-pm -k start 6029 5947 1 42.7 MB 0.5 MB /usr/sbin/apache2 -k start 6030 5947 1 42.7 MB 0.5 MB /usr/sbin/apache2 -k start 6031 5947 1 42.5 MB 0.3 MB /usr/sbin/apache2 -k start 6033 5947 1 42.5 MB 0.4 MB /usr/sbin/apache2 -k start 6034 5947 1 50.5 MB 0.4 MB /usr/sbin/apache2 -k start 23482 5947 1 82.6 MB 0.4 MB /usr/sbin/apache2 -k start ### Processes: 8 ### Total private dirty RSS: 3.50 MB --------- Passenger processes --------- PID Threads VMSize Private Name --------------------------------------- 6026 1 10.9 MB 4.7 MB Passenger spawn server 23481 1 26.7 MB 3.0 MB Passenger FrameworkSpawner: 2.0.2 23791 1 26.8 MB 2.9 MB Passenger ApplicationSpawner: /var/www/projects/app1-foobar 23793 1 26.9 MB 17.1 MB Rails: /var/www/projects/app1-foobar ### Processes: 4 ### Total private dirty RSS: 27.76 MB
The private dirty RSS field shows the *real* memory usage of processes. Here, we see that all the Apache worker processes only take less than 1 MB memory each. This is a lot less than the 50 MB-ish memory usage as shown in the “VMSize” column (which is what a lot of people think is the real memory usage, but is actually not).
Please note that this tool only works on Linux. Unfortunately other operating systems don’t provide facilities for determining processes’ private dirty RSS.
- Improved stability
-
If the framework spawner server or application spawner crashes, then Passenger 1.0.1 will keep showing error messages until one restarts Apache. Passenger 1.0.2 will automatically restart spawner servers when they crash, thus lowering maintenance burden even more.
- Setting ENV['RAILS_ENV'] in environment.rb now works
- A bug caused ENV['RAILS_ENV'] in environment.rb to be ignored. This has now been fixed.
- Support for custom page caching directories
-
Page caching was supported by Passenger, but setting a custom (non-standard) page caching directory did not work. This has now been fixed. But please note that Passenger won’t be able to accelerate page cache files in non-standard page caching directories.
- Usability and documentation improvements
-
The community has provided a lot more insight on things that can go wrong. We’ve done our best to document all troubleshooting-related issue into our Users guide. We’ve also adapted some error messages so that users can solve the problem without reading the manual.
Thanks for all the feedback people!
- Fixed conflicts with system-provided Boost library
-
Passenger makes use of the Boost C++ library. Its sources are included into the Passenger sources. But if the system already has a different Boost version installed, then the two Boost libraries would conflict with each other, and Passenger would fail to install. We’ve made sure that this doesn’t happen: now, installation will succeed even if there’s already another Boost version installed.
- Improved SSL compatibility
-
There was a problem with SSL hosts, which would only be triggered if “SSLOptions +ExportCertData” is set. This issue has now been fixed.
- Improved support for graceful restarts
-
If you installed Passenger for the first time, then the first graceful Apache restart would not properly initialize Passenger. This issue has now been solved.
There are also a few small improvements and changes that aren’t worth mentioning.
How do I upgrade?
Just install it like you did the first time:
gem install passenger
and
passenger-install-apache2-module
Please don’t forget to copy & paste the Apache config snippet that the installer gives you.
Enterprise Licenses, donations and t-shirts
In many ways, Phusion Passenger (mod_rails) has been an overwhelming success to us, and we’re very grateful for the community support you guys have given us. Also, a lot of companies and individuals have been more than generous in purchasing an Enterprise License for Phusion Passenger (mod_rails). In particular, we’d like to thank all the people who have donated over a certain amount and thought it would only be fitting to send them something concrete as a reminder of this generous act. After giving it a lot of thought, we came with something really shabby (or at least we ‘part-time fashion connoisseurs’ think so
).
To celebrate our first successful open source product launch here at Phusion, we’ve decided to silkscreen-print 100 limited edition Phusion t-shirts, each hand-numbered from 1 to 100. A few of these will go to our friends at Apple, Sun Microsystems and 37 signals, and the remainder of the shirts will go to those who have donated over 200 USD in total (we’ll take care of the shipping fees). Needless to say, these shirts are going to be hot as heck at IT conferences such as Railsconf, as they have been silkscreen-printed by the same people who are responsible for printing the shirts for the uberhip brands Rockwell, Freshcotton and Top Notch. The shirts themselves are super premium t’s which weigh 205gr/m2. To emphasize this even more, we’ve arranged for a photo shoot with a few professional lady models
and just like you, we can’t wait to see the result of this. Hopefully, you’ll be able to see the result soon!
An artist’s impression of the Phusion t-shirt
People who haven’t donated yet, or donated less than 200 USD but who want a piece of the t-shirt action as well will get the opportunity to “set this right” in the second (current) and third batch of enterprise licenses by donating the remainder amount to us under the same PayPal account. We’ll try to sort this out then as soon as possible. Needless to say, first come, first serve will be maintained, so if you want a shirt, be sure to act fast as supplies are bound to not last for very long! You probably don’t want to be figuring out that you actually wanted a shirt like this when it’s too late right?
Also, we’ve only got a limited amount in each size (especially the sizes small and XXL are likely to run out fast, and not to mention the girlie sized shirts for the ladies).
Lastly, we’re very grateful for all donations, and it is for this reason that we’ll also occasionaly randomly pick a few people from the donation list that haven’t donated over 200 USD for a Phusion t-shirt as well
. So in short, whatever amount you decide to donate, be sure to include your shirt size as well from now on as it might be your lucky day
RailsConf
Not only community wise, but also commercial wise, Phusion Passenger has opened up a lot of doors for Phusion that would otherwise likely have remained closed. For starters, we’ll be talking at Railsconf in a little more than a month about Phusion Passenger and the highly anticipated Ruby Enterprise Edition. It seems that the latter has already generated a lot of buzz and that this for the greater part, is because of its name. We actually think this is a good thing since we don’t believe that there is such a thing as bad publicity.
Don’t worry too much about it though, Railsconf will provide us with the perfect opportunity to dive into this subject a little bit deeper and hopefully, you’ll agree with us on that it’ll make a lot of sense to call it Ruby Enterprise Edition. We’ll also do something that is probably unprecedented with regards to talks so be sure to check us out over there, even if it’s just for the meet and greet / casual chat.
We’re on the RailsConf speakers list!
Side notes
We’re also still hard at work on writing a series of articles on both Phusion Passenger as well as Ruby Enterprise Edition from which we’ll also distill a scientific paper to be published on eeprints at the University of Twente (rocking!
). Needless to say, these articles will be published for your reading pleasure as well.
As you may have already noticed, Phusion recently consisted of mainly Hongli Lai and Ninh Bui. Even though we two make up for one hell of a team, we both definitely started feeling the growth pains of a healthy growing startup company. A little while ago we posted some job openings in the hopes of increasing Phusion’s capacity, but unfortunately, most of these applications were from outside of the Netherlands.
Today however, we’re pleased to announce that our good friend Tinco Andringa has decided to join the fray by joining Phusion. He’s not alone in this though, since our other good friend Maurits Dijkstra has also decided to do the same. And yes, the latter of the two IS related to the famous Edsger Dijkstra, which you may already know from Dijkstra’s shortest path algorithm
(but that wasn’t the main reason why we wanted him on board at Phusion per se
) Just like with Hongli and I, Maurits’ and Tinco’s computer science education find their origin at the Universiteit Twente and both have built a nice career on the side as software engineers as well: with this configuration, we hope to be able to even deliver better on our services and products!
Well, that wraps it up for today! Stay tuned though, as we’ve only started to ‘bring it on’!
With kind regards, your friends at Phusion,
Hongli Lai
|
Ninh Bui
|
- Tinco Andringa
- Maurits Dijkstra
Hongli Lai
Phusion. All rights reserved.