Earlier this week Eileen M. Uchitelle (@eileencodes), Senior Systems Engineer at GitHub and Rails core team member, spilled the beans and let the attendees of the Amsterdam Ruby Meetup in on the future of Rails. Fotos Georgiadis, Senior Software Engineer at Catawiki, elaborated on the journey of splitting up the auction site's monolith into microservices.

Rails does(n't) scale

Rails getting a bad rep for not being scalable is ironic as it has taken many companies to a scale they could have never dreamed of. If Twitter had build their app to handle the scale it is today, they probably would have been bankrupt before they had a proof of concept.

Rails does scale, it just doesn’t easily (yet). Historically it hasn’t prioritised scaling. A framework is scalable when over time an application’s growth, code, data and traffic don’t affect the way it performs. In the context of Eileen’s talk at Amsterdam.rb, it comes down to the following:

• Your test suite shouldn’t block development,
• Your application should be able to handle data and traffic increases,
• It should be easy for engineers to scale their apps without spending hours writing a customer setup or using external tools.

A new major version always allows the core team to reconsider what’s possible. While 5.2 just came out, Eileen shared her thoughts on the future of Rails (6).

$ rails test, or: deploying code instead of waiting for CI to finish

Parallel testing is new, allowing you to run tests concurrently using forked tests or threads. The more code you add the more tests you’re adding (presumably). Parallel testing will significantly reduce the time it takes as compared to running your tests linearly. Some companies have written an implementation of this concept, but never shared it upstream in Rails. Rails 6' Parallel Testing is based on the parallel testing infrastructure of GitHub, and supports using multiple databases, out-of-the-box.

Parallelization is invoked through the test helper using the parallelize method. Parallelization with forking lives in ActiveSupport and is backed by Minitest’s existing parallel testing feature. Rails will automatically fork into two processes and create a database for each process to run your tests against.

For newly generated Rails 6 applications, parallelization will be automatically included in the test helper. For upgraded apps you’ll need to add a call to parallelize in your test helper.

Eileen M. Uchitelle, Senior Systems Engineer at GitHub and Rails core team member, at the Amsterdam Ruby meetup.

Threaded testing with Minitest is what you'll want to use if you're on Windows, or if you're a JRuby user. Obviously, your code will have to be thread safe in order for you to use the threaded parallelizer.

Interested in the code that made this all possible? Check out the PR!

Justin Bieber started using your app and you're DOOMED

Too many reads and too many writes will cause your database to fail and take your application down with it. Your single primary database won’t cut it anymore. While it is already possible to support multiple databases in Rails, it isn’t standard behaviour. Nor is it documented or intuitive.

Splitting tables off into separate clusters or adding read replicas to relief pressure off of your databases were features introduced in Rails 5. It doesn’t however support migrations for multiple connections. None of your Rake tasks will work for your secondary database. Rails only has an understanding of ActiveRecord::Base. Database tasks for your secondary base effectively don’t exist. Your app is responsible for telling Rails where the migration paths live, cause Rails can’t ask where they are. Rails 6 stores the migration paths not at the class level but on the connection (in database.yml).

For the Migration Paths refactoring PR, clickety-click. It’s backwards compatible too.

To make database tasks like rails db:drop, db:create and db:migrate work across all databases, not just the primary one, you'll need to specify the multiple databases using configs_for. The PR for this improvement can also be found in the Rails project on GitHub.

Since her Railsconf and Parisrb keynotes, Eileen fixed connection management for Rails 6 and multiple dbs support.

Rails isn't dying, it's maturing

Eileen isn't concerned that Rails will become bloated with all the new scaling code. While it's true that most apps won’t meet GitHub’s scale, looking at the ways that GitHub has forced Rails to scale over the years, Eileen can’t help but wonder what Rails would look like if they’d upstreamed these features 5 years ago.

Eileen upgraded GitHub from Rails 3.2 to 5.2 and got to see all the parts of the application, all of the decisions made over the last 10 years. "Scaling problems aren’t unique and various companies have developed their own implementations of parallel testing, and multiple databases management, but didn’t share. We tend to look inward for the solutions, instead of pushing a patch upstream, further benefitting from community improvement." Indeed, an 'upstream-first' approach forces you to write generic code (as opposed to rigid, inflexible code). Eileen encourages everyone to look at what we have done to force Rails to scale and what we can give back to the community.

cw-api to tame the monolith

Catawiki was founded in 2008, has seen 3 pivots, as well as a ~45.000% revenue growth over 4 year. Sixty developers have made 40.000 commits and jointly wrote 1 million lines of code.

If that sounds like a monolith, that's because it is. Some of the code relied on super old versions of Rails and Ruby, some of those versions went EOL years ago. Migrating to Rails 5.2 took 2 years, and that didn't do anything to change the fact that they we're still fighting a monolith.

Fotos Georgiadis, Senior Software Engineer at Catawiki.

With code smells and other issues through the roof, and in an attempt to improve their code score, Catawiki first wrapped a core API around the monolith. Then they started creating microservices that introduced UIs. cw-api was the glue to put it all together, and once Fotos stripped the API of all the Catawiki-specific stuff, it will be available open source.

Catawiki now counts 15 microservices, all with a single responsibility, that are easy to roll back when one fails, without affecting the rest of the system. Catawiki went from a low 'C', to a proud 'B', cutting code smells and issues by half and are seeing excellent test coverage scores for their microservices.

November 20th, Amsterdam.rb will host its last meetup of the year with talks by Wander Hillen from WeTransfer, and David Halasz from RedHat.