snailgun: speeding up process startup

snailgun is an attempt to speed up Ruby command-line startup for applications which require large number of libraries (like Rails). It preloads the libraries into a process, then forks that process each time a new instance is required.

The code, and a couple of simple usage examples, are at

I can demonstrate script/runner starting in about one-quarter of the normal time, for an empty rails app.

However the code is far from usable at the moment - in particular, I don't know why it doesn't work properly with `rake test` where the benefit could be felt most. It also only works under Linux/BSD. I'm releasing this prototype in case anyone thinks there is mileage in this approach and might want to take it further, or tune it to work better with Rails.

At the moment, I'm only preloading the Active* and Action* libraries. I can't even use the logic from Rails::Initializer to do this, because config/boot.rb checks for defined?(Rails::Initializer) to decide whether to continue or not. There is still a lot more work done by Rails::Initializer which I think could be re-used.

So, to get maximum benefit from this approach, it may make sense to refactor Rails::Initializer into two parts:

(1) everything that's safe to preload - i.e. is independent of ENV and is very unlikely to be changed between requests (e.g. RAILS_GEM_VERSION, configured frameworks, gems, plugins)

(2) everything that depends on ENV (for example, the conditional requiring of 'active_support/whiny_nil'), and all the user's application code

In this case, everything in (1) can be preloaded into the master image, and then after each fork stage (2) can be run.

An alternative approach would be to start separate processes for 'test' and 'development' environments, so they can both run the initializer to completion. This would give the maximum speedup, and as long as you set config.cache_classes = false in the test environment, I think it should be safe. The awkward bit is then making all the tools dispatch to the correct process (e.g. script/console, script/ runner, rake test etc). That is: if the environment says there is an preinitialized Ruby process then use that, rather than loading config/ boot.rb. A third process with just 'rake' preloaded could also be available.

Regards,

Brian.

P.S. The name "railgun" would have been more obvious, but unfortunately it has already been taken by another project :slight_smile:

An alternative approach would be to start separate processes for 'test' and 'development' environments, so they can both run the initializer to completion. This would give the maximum speedup, and as long as you set config.cache_classes = false in the test environment,

I decided to go with this approach, and it seems to work rather well. For now, you need to specify the environment to use up-front: e.g.

    RAILS_ENV=test frake test:units

    RAILS_ENV=development fruby script/runner 'p Foo.first'

The reduction in startup time is quite startling.