How do you improve the speed of Factory_girl? Not use it? Some tips for those using Factory_girl and still need some help.

rubyonrails-talk
1

I am on a quest to make my Rails tests faster. I only have 520 tests, but they take 62 seconds to run in bash, and 82 seconds to run in Rubymine.

As an example of a typical controller test, I was using this code to sign_in as a @user and create the basic @comment in a CommentsController for my RSpec controller tests:

    before(:each) do       @user = Factory.create(:user)       sign_in @user

      @comment = Factory.create(:comment)     end

As you might realize... this is slow. It builds a `@user`, but also builds the associations for that user. Same for the `@comment`.

So I thought calling `Factory.build(:user)` would solve it... but I get weird errors. For example, `current_user` returns `nil`.

So... I decided to use `Factory.build()` and stub out all the before filters in my parent controller. However, my rspec log still says a TON of inserts are hitting the database when I inspect the RSPec log afterwards (we are talking hundreds of lines of code for just 3 tests!)

      before(:each) do         @user = Factory.build(:user)         #sign_in @user

        controller.stub(:authenticate_user!) #before_filter         controller.stub(:add_secure_model_data) #before_filter         controller.stub(:current_user).and_return(@user)

        @comment = Factory.build(:comment)       end

The sad fact is, the above `before(:each)` block has ZERO effect on test performance. As I discovered, calling `Factory.build()` will still internally call `Factory.create()` on the child associations.

Here is a `before(:each)` block that effectively removes the junk produced in the RSpec log. It gave me a 35-40% test performance boost

      before(:each) do         @user = Factory.build(:user, :role => Factory.build(:role))         #sign_in @user

        controller.stub(:authenticate_user!)         controller.stub(:add_secure_model_data)         controller.stub(:current_user).and_return(@user)

        @site_update = Factory.build(:site_update, :id => 5, :author => Factory.build(:user, :role => Factory.build(:role)))

        @comment = Factory.build(:comment,                                  :author => Factory.build(:user, :role => Factory.build(:role)),                                  :commentable => @site_update)       end

This makes the tests run faster, but it's also ugly as sin. We can't seriously write this for every test... do we? That's nuts. I'm not doing it.

I also want to point out that any one of these `Factory.build()` lines still takes about .15 seconds even though they are NOT hitting the database!

Running only 3 tests still results in about .3 to .35 seconds of time taken up by factory_girl PER test! I think that is totally unacceptable. If you remove the `Factory.build()` lines, the tests run in 0.00001 seconds.

I think the jury is in: factory_girl is one really slow library. Is the only solution to not use it?

2

I am on a quest to make my Rails tests faster. I only have 520 tests, but they take 62 seconds to run in bash, and 82 seconds to run in Rubymine.

As an example of a typical controller test, I was using this code to sign_in as a @user and create the basic @comment in a CommentsController for my RSpec controller tests:

   before(:each) do      @user = Factory.create(:user)      sign_in @user

     @comment = Factory.create(:comment)    end

As you might realize... this is slow. It builds a `@user`, but also builds the associations for that user. Same for the `@comment`.

So I thought calling `Factory.build(:user)` would solve it... but I get weird errors. For example, `current_user` returns `nil`.

So... I decided to use `Factory.build()` and stub out all the before filters in my parent controller. However, my rspec log still says a TON of inserts are hitting the database when I inspect the RSPec log afterwards (we are talking hundreds of lines of code for just 3 tests!)

     before(:each) do        @user = Factory.build(:user)        #sign_in @user

       controller.stub(:authenticate_user!) #before_filter        controller.stub(:add_secure_model_data) #before_filter        controller.stub(:current_user).and_return(@user)

       @comment = Factory.build(:comment)      end

The sad fact is, the above `before(:each)` block has ZERO effect on test performance. As I discovered, calling `Factory.build()` will still internally call `Factory.create()` on the child associations.

Here is a `before(:each)` block that effectively removes the junk produced in the RSpec log. It gave me a 35-40% test performance boost

     before(:each) do        @user = Factory.build(:user, :role => Factory.build(:role))        #sign_in @user

       controller.stub(:authenticate_user!)        controller.stub(:add_secure_model_data)        controller.stub(:current_user).and_return(@user)

       @site_update = Factory.build(:site_update, :id => 5, :author => Factory.build(:user, :role => Factory.build(:role)))

       @comment = Factory.build(:comment,                                 :author => Factory.build(:user, :role => Factory.build(:role)),                                 :commentable => @site_update)      end

This makes the tests run faster, but it's also ugly as sin. We can't seriously write this for every test... do we? That's nuts. I'm not doing it.

I also want to point out that any one of these `Factory.build()` lines still takes about .15 seconds even though they are NOT hitting the database!

Running only 3 tests still results in about .3 to .35 seconds of time taken up by factory_girl PER test! I think that is totally unacceptable. If you remove the `Factory.build()` lines, the tests run in 0.00001 seconds.

Isolate out factory girl and make sure that's the culprit... unless I'm doing my math wrong my :user factories take 0.039 seconds each. This is on a macbook pro, rails 3.0.7, postgresql db.

$ rails runner 'Benchmark.bm {|x| x.report { 100.times { Factory.create(:user) } } }'       user system total real   3.840000 0.100000 3.940000 ( 4.591107)

My factory...

Factory.define :user do |f|   f.sequence(:username) {|n| "user#{n}" }   f.sequence(:email) {|n| "user#{n}@example.com" }   f.sequence(:first_name) {|n| "First#{n}" }   f.sequence(:last_name) {|n| "Last#{n}" }   f.sequence(:url) {|n| "http://user#\{n\}\.example\.com" }   f.password 'secret'   f.password_confirmation 'secret'   f.is_confirmed true   f.sequence(:born_on) {|n| (10 + n).years.ago - 10.days } end

Anyway... for what it's worth...

-philip

3

Thanks for the response philip:

Here is my results:

x.report { 100.times { Factory.create(:user) } } }'       user system total real   9.940000 0.080000 10.020000 ( 14.872736)

That's about .1 to .14 seconds, which is exactly what I stated. My controllers usually make 2-3 calls to Factory.create(), so that accounts for the .3 to .35 seconds PER test I spoke of.

What do I do? This is too slow for me. I will have 2500+ tests by the time I am done my application probably. I already have 520 and I just barely began. I am only 2 weeks in... not even.

4

I think the culprit in my case is f.association. That probably accounts for why mine is 2-3x longer than yours. I am on an intel quadcore machine... there's no reason for this.

But even having said that, I still submit that 4.591107 seconds to make 100 objects + dependencies is ridiculously slow - even for Ruby. Factory_girl is really just a poorly performing library.

I think there's one other library worth considering... so I will see if that fairs any better. I think Factory_girl (or something like it) is worth using when you need to test validations that also depend on associations. I think Factory girl is pretty good for making mini- databases to test your queries/scopes.

But I think Factory_girl is clearly NOT good for testing other things besides these. For controllers, it is too expensive to use.

I will investigate others.

5

It turns out, if I make the before(:each) block say this:

  before(:each) do     @user = User.new     controller.stub(:authenticate_user!)     controller.stub(:current_user).and_return(@user)     controller.stub(:add_secure_model_data)

    @site_update = SiteUpdate.new     @comment = Comment.new   end

All 3 tests now run in 0.07 seconds! That is a 2000% improvement.

I guess the solution is not to use factories at all for controller tests.

6

It turns out, if I make the before(:each) block say this:

before(:each) do @user = User.new controller.stub(:authenticate_user!) controller.stub(:current_user).and_return(@user) controller.stub(:add_secure_model_data)

@site\_update = SiteUpdate\.new
@comment = Comment\.new

end

All 3 tests now run in 0.07 seconds! That is a 2000% improvement.

not hitting the database helps! In the apps I work on now I've written a little test helper that allows you do do stuff like

common_context do   @user = Factory.create :user   @product = Factory.create :product end

unlike before(:each) this will only run once for a given context, but the @user and @product instance variables will get resurrected at the start of each examples. Database savepoints are used so that if an example modifies anything in the database it is rolled back before the next example runs. Nested contexts can each have their own common_context block. This only works for active record objects and can't deal with unsaved objects but that covers most of my needs (and of course you can still have a regular before(:each) for your other setup stuff) I've been meaning to write this up as a blog post for a while but haven't got round to this.

Fred