Today LMAX Exchange has released ElementSpecification, a very small library we built to make working with selectors in selenium/WebDriver tests easier. It has three main aims:

• Make it easier to understand selectors by using a very English-like syntax
• Avoid common pitfalls when writing selectors that lead to either brittle or intermittent tests
• Strongly discourage writing overly complicated selectors.

Essentially, we use ElementSpecification anywhere that we would have written CSS or XPath selectors by hand. ElementSpecification will automatically select the most appropriate format to use – choosing between a simple ID selector, CSS or XPath.

In programming courses one of the first thing you’re taught is to avoid “magic literals” – numbers or strings that are hardcoded in the middle of an algorithm. The recommended solution is to extract them into a constant. Sometimes this is great advice, for example:

if (amount > 1000) {
  checkAdditionalAuthorization();
}

would be much more readable if we extracted a ADDITIONAL_AUTHORIZATION_THRESHOLD variable – primarily so the magic 1000 gets a name.

Probably the best thing I’ve discovered with my recent playing is Travis CI. I’ve known about it for quite some time, even played with it for simple projects but never with anything with any real complexity. Given this project uses rails which wants a database for pretty much every test and that database has to be postgres because I’m using it’s jsonb support, plus capybara and phantomjs for good measure, this certainly isn’t a simple project to test.

I’ve been playing around with ruby on rails recently, partly to play around with rails and partly to take a run at a web app I’ve been considering (which I’ve open sourced because why not?).

It turns out the last time I played with it was back in 2005 and slightly amusingly my thoughts on it haven’t changed all that much. The lack of configuration is still good, but the amount of magic involved makes it hard to understand what’s going on. The ease of finding documentation has improved dramatically – 10 years of blog posts really help. I’m still using TextMate and it’s still annoying that I can’t click a method name to jump to it’s definition – I hear good things about RubyMine but I’m not keen to invest that kind of money in what may be a very short-lived experiment.

Given our DSL makes heavy use of aliases, we often have to provide a way to include the real name or ID as part of some string. For example, an audit record for a new account might be:

Created account 127322 with username someUser123.

But in our acceptance test we’d create the user with:

registrationAPI.createUser("someUser");

someUser is just an alias, the DSL creates a unique username to use and the system assigns a unique account ID that the DSL keeps track of for us. So to write a test that the new account audit record is written correctly, we need a way to build the expected audit string.

Miller is like sed, awk, cut, join, and sort for name-indexed data such as CSV. Leaving this here for the next time I have to work with CSV files on the command line.  It can do a lot more than just CSVs of course, but that’s likely to be the most useful to me.

I have a dual monitor setup with Linux where one display is connected via DisplayPort and the other via DVI, both from an Nvidia graphics card. When the screen is locked both displays go black, the DVI monitor says it’s entering power save mode and the DisplayPort monitor says “No input” at which point both monitors turn back on displaying the unlock screen.

Playing with xset dpms force standby|suspend|off gave a variety of effects, sometimes the DisplayPort stayed off but the DVI turned back on, sometimes the DisplayPort went black but didn’t turn off etc.

A little while back I said that LMAX ran around 11,000 end to end tests in around 50 minutes. Since then we’ve deployed some new hardware to run our continuous integration on, plus continued building new stuff and are now running about 11,500 tests in under 20 minutes.

A large part of the speed boost is extra VM instances but also the increased RAM allocation available to each VM has allowed us to increase a number of limits in the system and we can now run more tests concurrently against each VM instance.

Even after a magnum opus on the DSL LMAX uses for acceptance tests, there’s one crucial feature that I haven’t mentioned: the use of aliases to allow tests to use simple, meaningful names while ensuring that uniqueness constraints are met.

Creating a user with our DSL looks like:

registrationAPI.createUser("user");

You might expect this to create a user with the username ‘user’, but then we’d get conflicts between every test that wanted to call their user ‘user’ which would prevent tests from running safely against the same deployment of the exchange.

At LMAX, all our acceptance tests are written using a high level DSL. This gives us two key advantages:

• Tests can focus on what they’re testing with the details and mechanics hidden behind the DSL
• When things change we can update the DSL implementation to match, avoiding the need to change each test that happens to touch on the area.

The DSL that LMAX uses is probably not what most people think of when hearing the term DSL – it doesn’t attempt to read like plain English, just simplifies things down significantly. We’ve actually open sourced the simple little library that is the entrance-way to what we think of as the DSL – creatively named simple-dsl. It’s essentially the glue between what we write in an acceptance test and the plain-java implementation behind that.

The Google Testing Blog has an article “Just Say No to More End-to-End Tests” which winds up being a rather depressing evaluation of the testing capabilities, culture and infrastructure at Google. For example:

Let’s assume the team already has some fantastic test infrastructure in place. Every night:

1. The latest version of the service is built. 
2. This version is then deployed to the team's testing environment. 
3. All end-to-end tests then run against this testing environment. 
4. An email report summarizing the test results is sent to the team.

If your idea of fantastic test infrastructure starts with the words “every night” and ends with an email being sent you’re doomed. Bryan Pendleton does a good job of analysing and correcting the details so I won’t recover that ground. Instead, let me provide a view of what reasonable test infrastructure looks like.

It’s not uncommon to want to find a particular line in a file and print just a part of it.  For example, if we wanted to find the total memory on a machine we could use:

grep '^MemTotal:' /proc/meminfo | 
  sed -e 's/^MemTotal:\s*\([0-9]*\).*/\1/'

Which does the job but duplicates a bunch of the matching regex. If you’re using a reasonably recent gnu grep, you can use it’s support for perl regex syntax and the \K operator: