Phusion white papers Phusion overview

The Road to Passenger 3: Technology Preview 3 – Closing the gap between development and production & rethinking the word “easy”

By Hongli Lai on July 1st, 2010

Update: “Phusion Passenger Lite” has been renamed to “Phusion Passenger Standalone” nowadays. The mass deployment feature has also been removed but will find its way back in the near future in a commercial version of Phusion Passenger.

Before Phusion Passenger came along, the most widely used Ruby app servers all implemented the same model which we refer to as the reverse proxy model. In this model, the user had to manually setup a bunch of app server processes and had to configure the web server to proxy requests to the app server processes. The technically inclined understand this model, but it is confusing to e.g. newcomers and to people who in general don’t have a lot of system administration skills or a reasonable understanding of HTTP. Most people were and still are much more familiar with PHP’s model, where you tell the web server where your app is and then have the web server take care of the rest for you. It was this confusion that caused all the uproar about sucky Rails deployment back in 2008.

While developing Phusion Passenger for Apache, we decided to follow a PHP-like model because ease of use was one of our main goals. No manual setups of app servers. No manual proxy configuration. Upload and go. For Phusion Passenger for Nginx, we continued to follow this model. Let’s call this the automatic model. As of 2010, Phusion Passenger appears to be the only widely-used Ruby app server that implements this model; the other widely-used Ruby app servers implement the reverse proxy model.

Reverse proxy vs automatic model: which one is better?

Ever since Phusion Passenger was first released, debates popped up about which one is superior. We believe that no model is inherently superior to the other. They are just different, meaning that both models have their own pros and cons. Which one is better for you depends a lot on your server infrastructure and your system administrators’ preferences.

Phusion Passenger’s automatic model:

  • Integrated into the web server. Processes are managed along with the web server itself, and configuration happens in the web server config file.
  • Easier to comprehend for most people. Appears more “standard stack” to system administrators who are not familiar with Ruby specifically.
  • Can spawn and shutdown processes dynamically according to traffic patterns.
  • Processes are automatically monitored: if they crash they are automatically restarted.
  • Less manual control over individual processes because they can come and go at any time.

Reverse proxy model as implemented by most other Ruby app servers:

  • App server is a separate entity. Processes are managed distinctly from the web server itself. Configuration happens outside the web server.
  • Many people have a hard time comprehending this and they generally find setups like this cumbersome, but to experts this model can be seen as simple, elegant and sensible.
  • Most app servers do not automatically restart crashed processes and one needs to monitor processes separately with things like Monit.
  • One needs to specify the number of processes up front: no dynamic process count scaling according to traffic.
  • Allows fine-grained manual control over individual processes.

We are not commenting on which points are supposed to be pros and which points are supposed to be cons because they are highly subjective. For us, integration into the web server is a strong plus because we host dozens of apps on our server(s) and we don’t like to spend time managing app server processes for each app, but other people are uncomfortable with having the web server manage things automatically and would prefer to keep a close eye on everything.

The automatic model can also be problematic to people who were on the reverse proxy model because they already had their web servers and infrastructures configured in a certain way. Switching to Phusion Passenger could mean changing a lot of web server configuration.

The hidden but unutilized potential

Reverse proxy model app servers can potentially have an extra advantage, but for some reason this hasn’t been implemented to its full potential so far:
Reverse proxy app servers are just easier to get started with. When you’ve just created a new Rails app, you can start it with script/server and you’re ready to go.

This works great in development but totally blows up in production. Reverse proxy model app servers must be put behind a reverse proxy e.g. Nginx or HAProxy for a variety of reasons such as security, load balancing between processes, handling of slow clients, etc. In production environments nobody exposes Mongrel or Thin directly to the Internet. Unicorn even explicitly documents that it is designed to be put behind a reverse proxy and that it doesn’t bother with slow clients at all.

In contrast, Phusion Passenger 2.x requires one to configure the web server, meaning the user must first install a web server. This is cumbersome when you’re in development and just want to get started. It is also cumbersome if you’re a newcomer and aren’t familiar with Apache or Nginx, and you just want to get your app running on your server.

Do you type script/server in development instead of creating a virtual host in the Apache or Nginx? Well you’re not the only one: we also do this until we eventually get sick of it, but there’s always a mental blockade that tells us that editing the web server configuration file is too much work to bother with.

Well, until Phusion Passenger 3 comes along.

Phusion Passenger Lite: fusion between the reverse proxy and the automatic model

In addition to Phusion Passenger for Apache and Phusion Passenger for Nginx, Phusion Passenger 3 introduces a new component to the existing lineup: Phusion Passenger Lite.

When it comes to usage, its interface is almost identical to that of Mongrel and Thin. To run your Ruby web app, just type this in the terminal and you’re ready to go:

passenger start

Closing the gap between development and production

Phusion Passenger Lite consists of an Nginx core. Nginx is known to be extremely scalable, high-performance and lightweight. You do not need to have Nginx already installed; this is automatically taken care of. You also do not need to have any Nginx experience: Nginx is hidden from the user but its power is automatically utilized.

Unlike Mongrel, Thin and Unicorn, Phusion Passenger Lite can be directly exposed to the Internet. It can serve static files at blazing speeds thanks to the Nginx core. Mongrel and Thin can serve static files but they aren’t very good at it. Unicorn doesn’t even try.

Easy migration from existing reverse proxy app servers

Because the interface is so similar, you can easily swap Mongrel, Thin or Unicorn in your existing reverse proxy setup and replace it with Phusion Passenger Lite. Unlike Mongrel and Thin, Phusion Passenger Lite only has to listen on a single socket instead of multiple, vastly simplifying your reverse proxy configurations. Phusion Passenger Lite can listen on a Unix domain socket instead of a TCP socket, just like Thin and Unicorn. In reverse proxy setups this can yield much higher performance than TCP sockets.

Advantages over existing reverse proxy app servers

Unlike Mongrel, Thin and Unicorn, Phusion Passenger Lite can dynamically spawn and shutdown processes according to traffic. However you can also configure it to use a static number of processes! In fact you can configure a minimum and a maximum and have Phusion Passenger Lite automatically figure out the number of processes to use for the current traffic.

Like Phusion Passenger for Apache/Nginx and Unicorn, worker processes that have crashed are automatically restarted.

That said, bear in mind that this advantage can be a disadvantage to some people. At its heart, Phusion Passenger Lite still manages processes for you, so you don’t have as much fine-grained control over the processes as you do with other reverse proxy app servers.

Advantage over Phusion Passenger for Apache and Phusion Passenger for Nginx

Another unintended advantage of Phusion Passenger Lite is that it runs as the same user as the shell and respects environment variables that are defined for your shell, e.g. things like PATH, LD_LIBRARY_PATH, RUBYOPT, GEM_PATH and GEM_HOME.

  • Some people find that their app cannot load a certain library when the app is started in Phusion passenger, but can when the app is started with e.g. Mongrel or Thin. This is almost always caused by some environment variable that’s set in the shell but not in the web server: everything you set in /etc/bashrc and friends don’t have effect on processes started from the web server.
  • Some people say that their application does not start on Phusion Passenger, but does under Mongrel/Thin. Very often this turns out to be just a permission issue or some web server configuration issue.

With Phusion Passenger Lite, even confusion like this will be a thing of the past.

Rethinking the word “easy”: automatic mass deployment

Phusion Passenger is considered by many people the easiest Ruby app server out there. But can it be easier? After some heavy thinking outside the box, we believe the answer is yes, it certainly can!

Imagine having a directory full of Ruby web apps, e.g. ~/Sites. To deploy an app, just drop your application’s directory into ~/Sites. To undeploy it, remove the application’s directory. The application directory’s name is used as the domain name. No manually signaling the web server for a restart.

This is exactly what we’ve done with Phusion Passenger Lite. We call this feature automatic mass deployment. Check out this demo.

So from now on, if you have a bunch of Ruby web apps in the same directory, just run this command in that directory

sudo passenger start -p 80 -u (some_unprivileged_username)

and you’ve immediately deployed every single app!

Conclusion

Phusion Passenger Lite does not replace Phusion Passenger for Apache or Phusion Passenger for Nginx. Rather, it is a complement to our existing lineup of Phusion Passenger products, optimized for different use cases. Phusion Passenger Lite closes the gap between development and production and can be used comfortably and easily in both. It can act as a drop-in replacement for your existing reverse proxy based setup. It makes Ruby web app deployment even easier than before: now you don’t even need a separate web server. On the other hand, if you need integration into the web server, then Phusion Passenger for Apache/Nginx is for you.

We hope you’ve enjoyed this Technology Preview. Please stay tuned for the next one because we have more exciting news for you!

  • Martijn Storck

    I’ve never had anything work in my development environment (with Webrick, Thin, whatever) and fail on Passenger, but it’s nevertheless very cool to be able to use Passenger in development. The install process seems as easy as ever, great job :) The mass deployment is also very cool.

    I’m just thinking aloud here, but would it be useful if Passenger for Apache/Nginx had a way to automatically start an instance of Passenger Lite and proxy to that if a user requested it (by means of a conf/passenger.conf file or something)? This would allow a user to apply slightly more tweaking to nginx and Passenger settings in a shared hosting environment.

  • http://www.eet.nu/ Tom-Eric

    Can automatic mass deployment work with multiple rubies/gemsets/etc.?

    I really love using passenger in development, because I can easily add a new host and use it. Unfortunately, a lot of apps use different ruby versions, so I find myself switching passenger’s ruby and restarting apache a lot.

  • http://www.phusion.nl/ Hongli Lai

    @Martijn: I think you’ve just described how Phusion Passenger for Apache works. You can customize some stuff with .htaccess as long as the proper AllowOverride permissions are set.

    @Tom-Eric: No but it’s planned for the future. Alternatively you can just run multiple Passenger Lite instances, each with a different Ruby.

  • Pingback: July 1, 2010: Screencasts and Road Maps « Rails Test Prescriptions Blog

  • Derek Hall

    I’ve been using passenger in development for over a year now. With the help of ‘passenger pane’ on my mac, configuration is minimal — but I can’t wait for this lite version. It looks amazing! Thanks for all your hard work on this. Do you have any plans for the PassengerPoolIdleTime to be on a per app basis in Passenger 3?

  • http://saimonmoore.net Saimon Moore

    You guys are certainly only a roll…

  • http://jeremy.wordpress.com Jeremy Lecour

    A seamless integration with RVM would be a HUGE progress. Passenger would detect if there is an .rvmrc file at the app’s root directory, and use the specified ruby and gemset.

    Until such magic is possible, some description on how to do this with multiple Passenger Lite instances would be very useful.

    Thanks for your hard work. I can’t wait to put my hands on this.

  • http://www.phusion.nl/ Hongli Lai

    Basically just like you would with Mongrel or Thin etc:

    $ rvm ruby-1.8.7
    $ gem install passenger
    $ cd /path/to/app-1
    $ passenger start -p 3000
    
    $ rvm ruby-1.9.2
    $ gem install passenger
    $ cd /path/to/app-2
    $ passenger start -p 3001
  • http://flocktravel.com cbmeeks

    Good gravy! That’s the way app deployment SHOULD be!

    I can’t WAIT to use this. But I’m still confused….so, we still need to install Passenger/NGINX like we did before and then install passenger lite?

    Do you have a tutorial from no web-server at all to a Passenger Lite install?

    Thanks for the hard work!

  • zdzolton

    No, @cbmeeks, sounds like the whole point of Passenger Lite is that it comes with nginx and you don’t have to worry about it.

    Hooray for easy development environments!

  • http://cientifico.net Guillermo

    You can mass deployment with nginx a passanger current version. Just follow the nginx and passenger documentation.

    I explain how in this post:

    http://cientifico.net/post/2010/07/01/nginx-rails-passenger-mass-deployment-actually

    But is as simple as:


    http {
    # [...]
    passenger_root /usr/local/lib/ruby/gems/1.8/gems/passenger-2.2.15;
    passenger_ruby /usr/local/bin/ruby;

    server {
    listen 0.0.0.0:80;
    root /var/www/servers/$host/public;
    access_log /var/log/nginx/servers.access.log;
    error_log /var/log/nginx/servers.error.log;

    location / {
    index index.html index.php;
    passenger_enabled on;
    autoindex on;
    }
    }
    # [...]
    }

  • Brian Hogan

    One question about this automatic deployment/discovery…. is it gonna look for “current” folders in there so we can still use capistrano? Or are we gonna just have to make symlinks from /var/apps/foo.com/current to /var/foo.com so Passenger Lite can see them?

    Just curious. Very cool stuff here, and we’re really excited.

  • http://alexle.net Alex Le

    How does Passenger Lite handle memory in a small VPS with little RAM, say 512MB? This will obviously boost the use of Rails for long tail users, people who just want to have the ease of Rails web development instead of resorting to PHP for smaller sites. Especially now with Linode offer dirt cheap plans, it makes sense to grab a small VPS to deploy Rails apps with Passenger Lite.

  • http://madebykiwi.com Zef Houssney

    Is there a way to point multiple domains/subdomains at the same app, or would you have to use the full version for that kind of thing?

    Very nice, I’m looking forward to using this!

  • JohnE

    Awesome! Does it work with ruby-debug?

  • http://glenngillen.com/ Glenn Gillen

    Does passenger lite allow multiple concurrent requests? That’s the only reason I ever bother to put an app behind nginx/apache so I don’t get stuck with webrick serving a page with a lot of assets one at a time.

  • http://www.phusion.nl/ Hongli Lai

    @Glenn: Yes.

  • Me

    This sounds fantastic! But still no windows. :(

  • Mark

    If passenger lite respects rvm for ruby binaries now, then what is stopping rvm gemsets from being respected?

  • Surjit

    i hate to say this but nothin’ more annoying than a video without sound. other than that, great work!

  • http://blog.angelbob.com Noah Gibbs

    Automatic mass deployment sounds like *exactly* what I wanted!

    Question: when running with automatic mass deployment, will it be possible to drop a “restart.txt” in the root directory and restart all the web servers? I’d love to put a big “git pull” on a cron job every hour and pull down the latest code, then have it touch a file to restart all the Rails servers…

    I *could* write a little shell script to just touch “tmp/restart.txt” in every directory under the root, of course. And I will if you guys don’t add that feature. But for maximum wonderfulness, it should be possible to just touch a single file in the root :-)

  • http://cs8.my Fadhli Rahim

    Wow. Again you manage to amazed people with your products. Passenger Lite seems to look a little bit like Apache Tomcat where the server picks up automatically whatever you put in the public folder. I was wondering whether it also shares the same session across all the apps? Meaning a user from app Sites/app1 can also access Sites/app2 and use the same session between both?

  • http://hanskristiankoren.com Hans-Kristian

    Have to agree with Surjit, sound should be added to the videos. Apart from that I have to say that Passenger Lite looks amazing. Well done.

  • sandro d.

    Great Work guys…

    I’ve been using Passenger for development for a while, and works great.

    I was wondering… will there be an option to run Pass. Lite in backgroud as a service, so I don’t have to manually start it everytime?

  • Bryan

    I already do something similar in develpment with the current passenger and the following in my nginx.conf:

    server {
    listen 127.0.0.1:80;
    server_name *.local;

    if ($http_host ~* (.*)\.local) {
    set $http_host_base $1;
    }

    root /Users/bryan/Sites/$http_host_base/public;

    rack_env development;
    passenger_enabled on;
    }

  • http://nepalonrails.com Millisami

    Passenger Lite with rvm, its the perfect app server.
    Great work guys! Can’t wait to use it??

  • John Bachir

    Great stuff. Glenn’s question was the first thing I wanted to know. Great to hear that the single instance handle’s multiple requests.

    Can you elaborate on that a bit more? And also (or at the same time) compare Lite to apache/nginx deployment? What do you see as pros and cons?

    Can the embedded ngnix be configured with all the regular nginx configuration options?

  • Pingback: Link dump for July 1st | The Queue Blog

  • http://k776.tumblr.com Kieran P

    Looks interesting and useful.

    * As Brian asked, will this work with Capistrano deployments? i.e. Will it see a current directory and go further into that, or if not, does it work nicely with symlinks?

    * Will this use a system installed Nginx? i.e. if I ‘brew install nginx’, Passenger Lite should not redownload and install another nginx.

    * Is there a –daemonize (-d) option, to have the process return so ssh sessions can be closed? You won’t get the log output, but presumbly things still get logged to the Rails log folder…

    * Does ‘automatic mass deployment’ work properly with Bundler? i.e. gems from one app aren’t available in the other… ?

    Keep up the great work. Looking forward to the post that describes this image: http://twitpic.com/1pn0y0

    :-D

  • http://jakubpawlowicz.com Jakub Pawłowicz

    The idea with embedding nginx is pretty neat!

    One question: would it be possible to alter Passenger Lite nginx configuration manually and/or install 3rd party modules for embedded nginx?

  • http://blog.marcchung.com Marc Chung

    Really nice job guys.

    Being able to also configure a static number of processes means that developers can work with an environment that closely resembles a typical production environment, which is definitely a good thing.

  • SB

    Very very nice. I can’t wait to deploy my apps with this, especially considering all my deployments these days are Ruby-only.

    What would be interesting is support for different Ruby versions per web-app, otherwise I suppose another passenger-lite process on a secondary IP remains a viable workaround.
    Few of my active deployments require Ruby 1.8.7, while the rest sails happily on 1.9.1.

  • http://twitter.com/jrmey Jeremy

    Very nice! If passenger lite was evented like thin, i would seriously consider…

  • http://blog.codefront.net/ Chu Yeow

    This sounds awesome (the previous 2 tech previews were more awesome) – is there a way I can get in on the beta test? I’ve a mid-sized Rails site that I’d like to test out with Passenger 3.

  • http://rwsleep.blogspot.com Jan

    great work, can’t wait to try it in development!

  • Richard

    Looks fantastic!

    In automatic-mass-deployment mode :

    May I specify RAILS_ENV on a per-app basis?

    Can I daemonize and write to each app’s own logs/#{RAILS_ENV}.log?

  • http://ryansobol.com Ryan Sobol

    @Hong Lai: Very cool! So, essentially, Passenger Lite is a wrapper around the Passenger for Nginx spawner and monitor that can also:

    * Auto install and configure a flexible, OS-independent Nginx environment
    * Configure the Nginx and Rack environments at run-time via command line options
    * Auto install and remove Rack applications from the Nginx environment given a filesystem path to monitor

    Am I missing anything?

    So the big question is… when?

  • Gimi

    Any plans for eventmachine support? I guess Passenger Lite + EM will be awesome.

  • http://www.cientifico.net Guillermo
  • http://coverallcrew.com Michael Wood

    What are you going to charge for something this amazing? I’d buy licences right away and I’m sure many other people would too. I love free software as much as the next guy, but you guys deserve some payment for your hard work.

  • Pingback: Thinking In Rails » Blog Archive » Recent Rails Links

  • adam

    it’s like a pavlov conditioned reflex …. I’m checking this blog every day to see if there are any new information to digest about passenger 3 … hope soon we will get more news ;-)

    have a great day,

  • Pingback: Delicious Bookmarks for July 20th from 13:42 to 14:00 « Lâmôlabs

  • http://www.technologieeninnovatie.hu.nl/Data/Lectoren/Wiebe%20Wiersema.aspx Wiebe Wiersema

    Nice work guys!!

    I am writing a paper on layering architectures and would like to include an example on Rails with Passenger in it.

    I am looking for architecture description/picture on the physical deployment with Passenger. All the senario’s I find seem to be about having one physical machine in which several forks are running. I am curious about a setup in which one phusical machine is not enough. So basically a loadbalancer distributing calls over several physical app servers how would this setup look like with passenger?

    Help greatly appreciated!

    Wiebe.

  • http://www.phusion.nl/ Ninh Bui

    @Wiebe:
    Thanks for the kind words and we’d love to help you out with your paper. So Phusion Passenger can be incorporated with a webserver e.g. nginx / apache. Because it now speaks HTTP because of that, you can put it behind any loadbalancer or your choice ;-) It would look pretty much no different than with other technologies in that regard. Hope that answers your question.

  • tvw

    Hi,

    wow, great thoughts.

    Would it be possible to make automatic mass deployment available not only through subdomains but also through sub-uris?

    I think of something like http://www.example.com/projects/app1

    where “projects” serves the same way as “sites”.

    In the moment I have to reconfigure and restart the webserver when doing this with passenger.

    I am working on an webapp, which allows to add little apps (with very short lifetime) which I would like to run in separate processes and I do not want DNS to be involved with this setup.

    Before your video, I thought, I would do this with jruby and tomcat, but since I normally use passenger, well it would be great, to use it for this project.

  • http://betterlogic.com/roger roger

    now if it justed worked on windows…or maybe the lite version does?

  • http://www.phusion.nl/ Ninh Bui

    @Roger:
    No, as long as windows POSIX state/semantics isn’t really fully compliant to the *nix one, windows won’t be supported any time in the near future.

  • Eduardo Rocha

    “As of 2010, Phusion Passenger appears to be the only widely-used Ruby app server that implements this [automatic] model”

    LiteSpeed has always implemented this model, even before Passenger. Maybe you are referring to those completely free app servers?

  • http://www.phusion.nl/ Hongli Lai

    @Eduardo: “widely-used” is the keyword.

  • Pingback: Phusion Passenger 3.0.0 final released – Phusion Corporate Blog

  • http://dawanda.com/ Steffen

    Would be great to have SSL in Passenger Standalone as well, like:

    rvmsudo passenger start -p 80 -pssl 443 –user=username

  • http://nathany.com Nathan Youngman

    Yah, SSL would be nice. Had to use the full-blown Passenger for Nginx in development to be able to test with self-signed certs.

  • Pingback: Phusion Passenger Standalone für die Entwicklung | Andreas Richter