To see our setup for testing JRubyFX applications, see our post on using Automaton.

The Docker container used for Brakeman Pro can be found here and the source is here.

Docker

Have you heard of Docker? I’m sure you have. It’s awesome. I love it. But it did take some time getting the setup to work. This post assumes you know your way around Docker.

In our post on using Automaton, I mentioned that Automaton commandeers your mouse and keyboard, essentially meaning you cannot use your computer while you test. What is this? 1999???? Unacceptable!

JavaFX can run “headless” using xvfb with relative ease. However, JRubyFX complicates things a little bit. We need to accomplish a few things:

1. Install libgtk2.0-0 and libxtst-dev. Some toolkit must be available for Java to understand how to manipulate the invisible UI.
2. Install Java (note Java 8u60 will NOT work out of box. See jruby/jrubyfx/issues #95).
3. Install JRuby. We chose not to use a ruby version manager for simplicity. Also, we have not had any success moving to JRuby 9000. I owe the JRuby and JRubyFX teams a couple of bug reports, but basically the jar that is produced from the jrubyfx-jarify command does not run under JRuby 9.0.0.0 or 9.0.1.0.
4. Ensure xvfb is available.

Unfortunately it takes about 40 lines to accomplish those things, but hey, IT WORKS. Now I can run the tests in the background without interrupting my workflow. Such heaven.

Parallelizing Tests

Running tests is great, but they can take time. In fact, our test suite took about 15 minutes. That is completely unacceptable!

Running tests in parallel can GREATLY reduce test time, but is also sometimes difficult to setup without each worker stepping on the toes of other workers. e.g. each worker needs their own database to keep the test data clean. Usually this is done by adding a bunch of conditional code that has to account for sharding, which is a pain in the ass. parallel_tests is one such tool that makes this easier.

However, we have docker containers. Isolated containers. Why not just shard the tests by using multiple docker containers at a time? Yeah, why not.

GNU Parallel is pretty sweet. I had no experience using it before but now I love it. Basically, we can create N containers and each up N cores. I found that sharding over 3 containers on my relatively new Macbook Pro was a good balance between the speed of tests and the resources eaten up while the tests are running. So:

docker build -t bmp . && parallel --gnu --halt 2 -j 3 docker run -t bmp ./tests.sh ::: 1 2 3

Voila, build and test times are down to 4 minutes.

CI Portability Win

Dockerizing our test also means that in theory, these tests will work for any environment (cough CI). So we were able to switch to Shippable without any pain, saving us tons of money in the process.