One of the main purposes of continuous integration is to know when your build fails. As soon as a problem is detected you want to be alarmed. But in certain situations you can accept a failed build. Today I will show you how to use a build matrix in Travis CI to allow failures for certain builds – without causing the entire build to be shown as failed.
Why?
If you use Ruby you can run it on MRI, JRuby or Rubinius. Most likely you have your favourite Ruby implementation and use Travis CI to build against it. If you don’t care about the other implementations you keep everything as it is and don’t need to use a build matrix.
A build matrix is a great help if you want to know if your code runs on JRuby or Rubinius but don’t bother if it doesn’t. Should your build fail on those implementations but succeed on MRI then your build still would be a success.
Configuration
The configuration is quite simple but if you don’t have a template it is easy to mess it up. As soon as you use a build matrix successfully Travis CI will show you a nice explanation on your build page – but unfortunately not when you would need it to get the configuration right.
Let’s start with this .travis.yml file to build for Ruby 1.9.3, 2.0 and 2.1:
|
1 2 3 4 5 |
language: ruby rvm: - 2.1.0 - 2.0.0 - 1.9.3 |
If you add Rubinius and JRuby as you did with MRI your entire build will fail as soon as any of your builds fail:
|
1 2 3 4 5 6 7 8 |
language: ruby rvm: - 2.1.0 - 2.0.0 - 1.9.3 - jruby-head - jruby-19mode - rbx-2.2.3 |
To allow Rubinius or JRuby to fail you must add them a second time as allow_failures inside the build matrix:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
language: ruby rvm: - 2.1.0 - 2.0.0 - 1.9.3 - jruby-head - jruby-19mode - rbx-2.2.3 matrix: allow_failures: - rvm: jruby-head - rvm: jruby-19mode - rvm: rbx-2.2.3 |
Now you can push to GitHub and your code is built on Rubinius and JRuby as well. Should any of those additional implementations fail your entire build is still a success and you can check the compatibility issues on your own time.
Conclusion
To allow failed builds in continuous integration seems wrong. But in the right situation this is a great help to slowly add support for more Ruby implementations. The biggest challenge in the configuration is to know that you must add the implementations you allow to fail twice. If you put them just in the allow_failures block they are ignored – and without an error message it’s hard to find the source of the problem.