Okay, so it turns out that while I thought I’d posted this article successfully several weeks ago, in fact, I hadn’t. Apparently in Mephisto, you have to publish your articles, or they won’t show up on the homepage. By publish, I mean assign it to a “section”. That’s what I get for never checking the front-page of my blog.

So, just when you thought the deluge of post-RailsConf posts was finally over, here’s another one: my obligatory RailsConf wrap-up.

+++

Along with about 1,600 fellow Rails developers from around the world, I’m at RailsConf in Portland Oregon. Traveling here was rather hellish: in a vain effort to save money, we had a flight out of Buffalo on Wednesday instead of YYZ (Toronto). The flight was cancelled due to the fact that bad weather had grounded our connecting flight from JFK, so we ended up staying the night in Buffalo, the armpit of western New York and possibly, the entire USA.

A Holiday Inn Express contained us for the night, and after drinking four of the hotel-restaurant’s finest bottles of mediocre wine, we made use of their luke-warm, heavily chlorinated hotub before calling it a night.

Our flight was rescheduled for 2:45PM the next day (almost 24 hours after the original departure time), and luckily, it was on time. After dining at a nearby Denny’s, we checked our bags and got on board. It was a short, hour-long trip to JFK where we were to wait two hours for our connecting flight. After four hours or so, we boarded the plane, got seated, and watched as the flight attendants went through the motions of the safety instructions before being told that due to a technical malfunction, the plane was being grounded.

We were herded out of our seats and back into the terminal where we waited another hour and a half for a fully functional plane. After boarding, we waited on the tarmac for another hour before takeoff. As a small penance, JetBlue awarded us each a $25 discount on our next JetBlue flight.

Finally, after leaving Unspace at 11AM Wednesday, we got to Portland at 4AM Friday.

So, here’s my write-up of the first day at RailsConf 2007. Most of the talks I attended, with the exception of the keynotes, were centered around testing. As most of my friends and readers know, I’m pretty big into testing. It makes sense, then, that I was drawn to this topic. First, though, came the keynotes.

Keynote: DHH

DHH had the first keynote of the conference. He started by talking about the accomplishments we’ve already made and how far Rails has come, especially in the last year. The code, the community, the plugins, and the books. He made special mention of the books, pointing out the plethora of Rails-specific texts that now exist (there were a mere three last year). His slide featured a collage of Rails book covers, including mine (yay!), even though it’s not published (quite) yet.

He did a quick demo of a resource scaffold (which now replaces the standard scaffold), and a console-based demo of ActiveResource, the latest addition to the Rails stack.

If you’re already using edge Rails and are paying attention to the commit logs, there wasn’t much that was new. Most (all?) of the features he spoke of are already implemented in edge, and if you’re like me, you’re already using them today. I was actually a bit surprised at the reaction of the audience for whom these features where brand spanking new. Apparently most people are using stable versions of Rails.

DHH finished off his presentation with a rundown of nine of his favorite things about Rails 2.0 (i.e., edge), summarized below.

9 Great Things in Rails 2.0

Breakpoints make a comeback courtesy of ruby-debug
HTTP performance improvements: stylesheet_include_tag :all, :cache => true
QueryCache: cache SQL queries that have have already been seen
Template extensions: html.erb and html.builder
Initializers: files in config/initializers to clean up environment.rb
Sexy migrations
HTTP authentication plugin now in core
The MIT Assumption: plugin generator creates the MIT license by default.
Spring cleaning: currently deprecated methods are slated for permanent removal

Clean Code, by Robert Martin

The first talk I went to was Clean Code by Robert Martin. Seriously, this was a great talk. Robert’s a good speaker, and was full of enthusiasm. He talked about how keeping your code clean is a matter of professional ethics; that being a good programmer means being a craftsman. It reminded me a lot of Dave Thomas and Andy Hunt’s The Pragmatic Programmer.

He walked the audience through a refactoring example, talking about the importance of using sound design principals and the need for test-driven feedback. There was a big emphasis on testing, something that I feel too many programmers don’t pay enough attention to. This is especially true of a lot of Rails developers who’ve come to Rails from paradigms where there isn’t a lot of emphasis on testing (PHP, I’m looking at you). He even went so far as to say that if you’re not testing your code, you can’t rightly call yourself a professional programmer. I tend to agree.

It was a great start to the morning and really got me pumped. In keeping withe the testing theme, the next talk I went to was Jay Fields’ on maturing your test suite.

Taking Rails Tests to the Next Level, by Jay Fields

Jay Fields is one of my favorite Ruby bloggers. I’ve been reading him for several months, and I really like his ideas. So, it goes without saying that I was excited for his talk.

He’s a ThoughtWorker, and his experience is on large teams, with up to 16 programmers(!), working 10 hours a day, all pairing, all the time. He admits that this kind of work is indeed stressful, but he feels it produces better software and is therefore worth it. Commendable.

His talk was mostly about testing tips, strategies, and advice. One recurring theme was the speed and ease with which you can execute your test suite. When you’re making frequent commits on largish teams like he is, it can be very time consuming to run large test suites before every checkin. The faster the tests run, and the fewer dependencies they have, the better.

He had a number of noteworthy tips and tricks, here are the ones I managed to jot down.

General Testing Tips

Testing one thing per test
One assertion per test
If you have a lot of setup code for your test, you have some other problems, perhaps some Law of Demeter violations, or other setup issues
Don’t cross boundaries with your tests
Don’t couple your tests to your implementation
Oftentimes you don’t need to hit the database to test something
Unit tests should not hit the database
Functional tests should hit the database
Classes should be tested in isolation
Testing in isolation often implies the need for mocks and stubs
Tests aren’t OO, so don’t bother trying to make them OO
If your tests depend on each other, its a Bad Thing
Tests should be able to run independently, on their own
Tests should be explicit; they should convey their intent clearly
Avoid creating abstract classes or helper methods that encapsulate magic
Readability, readability, readability

Decoupling Controller Tests

Decoupling models from the database is fairly straightforward. But what about decoupling controller tests?

You can call methods on your controllers directly, just like regular classes
Try to isolate controller test: decouple the rendering of the view

Private and Protected: YAGNI?

how do you test private or protected methods?
is protected and private even necessary?
in Rails controllers, public methods are accessible via the URL, but outside this context, marking methods as non-public has questionable value
zentest addresses the idea of testing private methods

Interesting Tidbit

There’s talk of “unit tests” becoming “model tests” and “functional tests” becoming “controller tests” in Rails. I like this. It would help to clear up some of the confusion that exists as a result of the terminology.

After lunch, I went to Rabble’s talk on adding tests to legacy (yeah, legacy) Rails apps.

Testing Legacy Apps, by Rabble

Rabble (Even Henshaw-Plath) talked about adding tests to ‘legacy’ Rails apps. How many times has this happened to you: you inherit a Rails project and discover much to your horror that it lacks tests. Sadly, a lot of Rails projects lack tests. So, should you sit down and write a whack of tests right away? Not really. According to Rabble, there are two basic strategies when it comes to testing apps that lack tests.

write tests when you find bugs
write tests when you refactor

Tips

Don’t do it all at once
Do BDT: Bug Driven Testing
Test before refactoring
Don’t write tests for code you’re not editing
Don’t write tests for tests sake
Baby steps: test one thing at a time
Build tests from logs
Use rcov
Use Autotest
Use Zentest
Use Heckle

What to test? (when you’ve not done TDD)

assert that calling methods doesn’t raise exceptions (assert_nothing_raised { })
assert that the page action returns :ok
assert responses, renders, assigns

Heckle

Heckle, by Kevin Clark and others, is a mutation tester (fuzzer). It screws with your code.

A lot of people use rcov to measure code coverage. This is absolutely a good thing. However, rcov isn’t enough: just because the line was run, doesn’t mean the line was useful.

If testing is your safety belt; rcov is your airbag; Heckle is your helmet, neck brace and fireproof suit.

Heckle is…

a measurement tool
an authoritative measurement of test quality
to be used alongside rcov, not in place of it
great for inherited code with no tests

Heckle can…

tell you which permutations survived the tests
let you know which things “don’t matter” to your tests

Interesting tidbit: S-Expressions

I’d never heard of S-Expressions before. It turns out, an S-Expression is an in-code expression of what the parse-tree looks like. Heckle uses the ParseTree library and ruby2ruby to rewrite the mutated code on the fly.

Fixtures: Friend or Foe by Tom Preston-Werner

In most cases, Rails makes testing a veritable cake walk. That is, with the exception of fixtures. As projects increase in size, fixtures tend to become brittle and interdependent. When your fixtures become a web of complex relationships, seemingly benign changes to one fixture can cause many test cases to fail. This is a Bad Thing.

There are a few solutions to this problem, none of which are very good.

Solutions

add lots and lots of documentation to your fixtures (ugh…)
use helpers that leverage AR to create fixture data (lots of code…)
use an in-memory database
use mocks and stubs

Tom proposes a solution I like very much: fixture scenarios. Here’s the gist:

All of the fixtures placed into your scenario/ directory will be loaded when you invoke the scenario method with your scenario name. In addition, any Ruby files you place in the scenario directory will be run after the fixtures. You can use a Ruby file to create non-database model instances, set up relationships between fixtures, or replace fixtures entirely by creating your database items with Ruby code.

[RAILS_ROOT]
+-test/
  +-fixtures/
    +-someting.yml # gets loaded for all scenarios
    +-brand_new_user/
      +-users.yml
      +-books.rb
      +-experienced_user/
        +-articles.yml

In your tests, you simply replace fixture with scenario:

class UserTest < Test::Unit::TestCase
  scenario: brand_new_user
end

class UserTest < Test::Unit::TestCase
  scenario: brand_new_user, :root => false
end

rake test:scenarios # run just the scenario tests

Fixture scenarios sovlessolves the following test grievances:

namespacing
brittleness
validation
contamination
performance

Creating scenarios with pure Ruby

You can create a rb file called scenarios.rb to define your scenario yaml files in pure Ruby. Observe:

+-fixtures
  +-scenarios.rb

build_scenario :banned_users do
  %w(Tom Chris Kevin).each_with_index do |user, index|
    User.create(:name => user, :banned => index.odd?)
  end
end

Creates the scenario directory, banned_users, with a users.yml fixture inside it

+-test/
  +-fixtures/
    +-banned_users/
      +-users.yml

Awesome. I’ll definitely be using this.

Zen and the Art of Rails Deployment

Ezra is easily the foremost authority on Rails deployment. It makes sense, then, that he’s a founder of EngineYard. He started his talk by pointing out the fact that the Rails deployment scene is in constant flux. Although different options have come and gone, the basics remain unchanged.

The new recipe? A gateway server (Apache, Nginx, Lighty, etc.) that fronts Mongrel. As if you don’t know this already, Mongrel is Zed Shaw’s HTTP server. It’s fast and flexible, and has removed the need for any of the more clunky solutions we used to use: CGI, mod_fastcgi, and mod_fcgid.

There are many options for http tools that front mongrel

pen, pound, balance, haproxy
lightspeed—can serve static files and proxy to mongrel
apache2.2.x/mod_proxy_balancer

There are many problems with the http tools that front mongrel

Pen—no ssl, no connection rate limiting
pound—falls down under high load
haproxy—supports connection rate limits, very high performance, no static files, many moving parts
lightspeed—free version is crippled
apache 2.2.x—works, but bloated

Enter: Nginx

seriously high performance
insanely fast
super small resource footprint
stands up under the heaviest loads without leaking memory
killer rewrite and proxy modules
approachable author and growing community
memcached module
ssl support baked in

Nginx + Mongrel

According to Ezra, Nginx + Mongrel is the stack to be on. The only reason to keep Apache around anymore is for mod_dav_svn (if you need it). Nginx’s configuration is incredibly flexible, allowing for the serving of static files and rails caches, addition to proxying dynamic request to mongrel. And Nginx is ridiculously fast.

Nginx Gotchas

There aren’t many gotchas when it comes to Nginx. Ezra pointed out a few.

niginx buffers file uploads
no connection rate limiting

Future of Nginx

As far as the future of Nginx goes, it’s looking bright.

mod_rewrite is going away
to be replaced with http_script_module
embed nekoVM directly in niginx

The Perfect Simple Stack

I love it when experts make recommendations. It takes out the guesswork. Here’s Ezra’s recommendation for the perfect (simple) stack.

Linux
Nginx
Mongrel (cluster)
Monit

Swiftiply

Swiftiply is a new entrant onto the Rails deployment scene.

It hot patches Mongrel, removing ruby’s thread and socket handling from mongrel core replacing it with EventMachine event loop. So, we get Evented Mongrel: single threaded and event driven. This provides a noticeable speed and io throughput increase that stands up much better under higher concurrent load without starting to slow down or leak memory.

This really makes a difference when Mongrel’s being hit with hundreds of concurrent users. In this scenario, Evented Mongrel can offer up to a 5x increase.

Swiftiply Proxy

Swiftiply Proxy is an event-driven proxy with a small memory footprint. According to Ezra, it’s even faster than Haproxy. It differs from a normal proxy in that, while a standard proxy must know about the ports of all backend, Swiftiply Proxy works by having the backends connect to it. So, all mongrels get started on the same port and then they open a persistent connection to the proxy. This alleviates the need to tell the server which backends are available, meaning you can start and stop as many mongrels as you want and they get auto configured in the proxy. This opens the door for scaling the number of mongrels automatically in response to the increased load on the fly. Sweet.

The Swiftiply Stack

Nginx
Swiftiply Proxy
Mongrels

Yep, so that’s my writeup. Hope you enjoyed it and all that.

Quoted-Printable

a multipart alternative

Obligatory RailsConf Wrap-up