Failed TravisCI builds because test run exceeds 50 minutes

I raised this issue via Github (here) and got informed by rafaelfranca that it’s better to do this via the mailing list. Sorry, Rafael; and thank you as well :-).

A few builds have been marked as “Error” because (most of the time) their GEM=railties, ruby=1.9.3 test suites exceed 50 minutes.

My suggestion is we split this particular test suite into two parallel builds (by giving each build a number of test folders) so that hopefully each build runs for about 25 minutes, which goes along with its ruby=2.0.0, ruby=2.1, ruby=ruby-head counterparts.

I’m still keen on this suggestion though. It:

  • reduces build time

  • prevents repetitive (and expensive) builds, caused by slow Rubygems or Bundler API

WDYT?

Sounds like a great idea, just make sure it doesn’t break the build locally.

For bonus points make the number of parallel builds configurable. For extra bonus points, make the parallelization work locally as well as Travis.

– Chad

In fact we already have parallelization, but it doesn’t work on Travis because we only have one core in our machines.

To run railties tests in parallel you only need to specify the CORES environment variable.

CORES=4 rake test

I don’t think it is worth to split this build, it doesn’t fail often, and even if it does we don’t take in consideration before merging. We prefer to fix the build after merging than wait the build to run and ask contributors to fix the build.

In the case of failed builds because of exceeding runtime, it means there are still tests left not run. Do we still merge the PR? What if one of the leftover tests is the one related to the proposed code change?

Yes we still merge the pull request. We don’t usually wait the build finish to merge a pull request. If it is related with the code, it will fail locally for someone.

Just to be clear we do use the CI but it is not a blocker to merge something.