Test Driven Development And The Myth Of Better Code

Claim: TDD Generates High Quality, Maintainable Code

Proponents of TDD argue that by writing the tests first, you are forced to focus on how the component will be used and thus will come up with a better design. My experience has actually been the opposite; TDD encourages you to just do whatever needs to be done to make the test pass and not think about design at all. I’ve never copied and pasted code so much in my life and the number of times I’ve written code that’s so messy I feel like I need a shower is getting scary. That’s okay though, it’s meant to be that way. TDD encourages short-sightedness, short-cuts and general sloppiness and that’s a good thing.

Since the first code we wrote was so bad, any professional developer will feel compelled to invest time in cleaning it up and that’s where the refactoring step in TDD comes in. The tests pass and we’ve done all the grunt work and head scratching to get the code to work, now we can devote all our energy to making it clean, clear and maintainable. In the traditional approach, the developer tries to get the code to work while at the same time trying to keep it clean and maintainable – there’s a constant struggle between wanting to just try this approach quickly to see if it will work and wanting to avoid creating an unmaintainable mess. Often, this struggle results in mediocre code being developed that works and is just good enough for the programmer to get lazy and not clean it up. With the mess that TDD makes, your conscience won’t let you sleep at night if you don’t take the time to clean it up. The odd thing about refactoring is that the biggest hurdle is starting. Once you start refactoring code you get into it and don’t stop until the code is beautiful but before you start it just seems like hard work so having your conscience nagging at you is a definite advantage.

The other advantage of designing last is that you have complete knowledge of the code before you start designing. You won’t suddenly find a corner case that doesn’t fit into your plan and you won’t have the remnants of failed attempts to make it work hanging around in the code base. To an extent it’s true that the design just “falls out” of the tests – when you write the tests, you think of all the things you’d like in the public API and they get encoded in the test cases. The internal design is only minimally influenced by the test cases and so the process of refactoring is really the process of designing the internals of the component. It may be cleaner to convert the input passed into the public APIs to an internal format to work with or there may be opportunities to reuse code. It’s important to note here that you shouldn’t be limited by the refactoring options provided by your IDE – there are a huge range of ways to refactor code that no IDE will ever provide an automatic method for. With passing tests to back you up, it should be easy to refactor by hand without introducing errors. By focussing exclusively on the design during refactoring, developers are more likely to find the best design and the result is cleaner, more maintainable code; even if the first draft was plain ugly.

Working with Doug, I’ve found that the refactoring process works particularly well when one person writes the code and then flicks the keyboard to their partner to do the refactoring. This ensures that there’s a fresh pair of eyes doing the refactoring work and since the “hands-off” pair takes a higher-level view of the code they are more likely to see opportunities to refactor and clean up the code. It also provides a brief break for the person who had been coding while their partner sets themselves up to attack the refactoring.

Claim: Reduces The Number Of Bugs By Ensuring Complete Test Coverage

Since you write the tests first and only write the minimum code required to make the tests pass you should wind up with pretty much 100% code coverage. Everyone knows that 100% code coverage means that your product is free from bugs don’t they? Obviously TDD isn’t the panacea that everyone’s been looking for. Even if you have 100% code coverage, there’s no guarantee that your tests are right or that you didn’t miss a corner case. How can you miss a corner case if you’ve covered every line? There’s a couple of ways.

1. Your code coverage tool is counting lines, not paths. Every time there’s an if statement in your code there’s two paths – one where the if evaluates to true and one where it evaluates to false. Most code coverage tools work on lines and so they only check that you covered the true case and ignore the fact that you should also have a test that doesn’t execute the lines of code in the if block. Loops can create an infinite number of possible paths through your code – perhaps your algorithm to calculate pi breaks on the millionth decimal place but works for every other case. Usually it’s not worth executing every possible path through your code but you need to execute the major ones and code coverage reports won’t always help you identify them. That said, many code coverage tools have a mode to work with blocks instead of lines which can be more accurate in this regard.
2. You may have executed the code, but not checked all the possible results. Perhaps the code was executed in the setup phase of a test and so the result wasn’t checked. Maybe the method did what you expected and tested for but also did something completely unexpected that you would never have dreamed of writing a check for.

That said, this isn’t a failing of TDD, it’s a failing of testing in general. At least with TDD you have some tests and you get into the habit of writing them every time. As an added bonus TDD can make writing tests fun and rewarding. Having to sit down and write tests after you’ve gotten the code working is tedious and boring – you never think you’ll find any problems, so why bother doing it, and if you do find a problem it just means you’ve created more work for yourself.  With TDD writing the tests becomes an intellecual challange – you know the code isn’t right but how do get things into the exact scenario that causes it to break? Like all good games, TDD even provides levels – the first test you write is easy, it’s guaranteed to break. As you write more tests though you cover all of the obvious problems and starting getting into the esoteric situations that take real skills and strategy to write a test that fails. TDD proponents regularly talk about how seeing the greeen bar showing all tests passing is a feel-good moment and gives confidence but seeing the bar go red is just as big a reward when you’re in test writing mode. Then you get to switch modes and try and make the bar green again.

The testing game works particularly well when paring because you can write your failing test then flick the keyboard over to your pair with a smug look to let them try and make your test pass. Three of us wound up working on a difficult problem one day and it turned into a game of TDD pong – one person would write a test that failed, the second person had to make it pass then the third person wrote another failing test and flicked back to the first person. The challenge was not just about breaking or fixing the code but doing it in the simplest way possible.

The biggest danger with TDD in this area though is that it encourages you to not code defensively. You write the simplest code that could possibly work. Often this means that you just know the code you’re writing is wrong but it’s the simplest thing and it does make all the tests pass. Take this as a challenge to find the case that does fail and don’t stop until it breaks or you’re completely satisfied that it works in all cases. The time you invest into writing that test case will pay off in maintainability because someone else may not have known that the simpler way wouldn’t work and now you have a test to make sure noone falls into that trap. More importantly though, you gain a better understanding of the code and how it all works because you’ve had to analyze it carefully to find the fail-case or to identify the reason that it always works. Don’t let TDD cause you to doubt your intuition though, if you think that the simplest way it could possibly work isn’t the right way to do it, then make sure you take the time to either write the failing test case or understand why it will always work. If you don’t trust your intuition you’ll likely find out too late that you were right – most of the time you will be. The power of TDD isn’t that it avoids writing unessecary code (though it does that) but in the fact that the code you right is guaranteed to do what you meant it to. You need to use your experience and intuition to make sure that you know what it’s meant to do.

Claim: TDD Provides Confidence In Your Code

If you have confidence in your tests you’ll have confidence in your code. The trouble is that having confidence in your tests is really quite difficult. Acceptance tests are easier to deal with because you can talk with the client to sort out whether or not there’s a problem with the test. With unit tests though there’s always this nagging feeling that the test is wrong and the code is right. It can be very difficult to determine why a test was written even fairly recently after you wrote it. Refactoring your tests and ensuring that they are as clean and clear as the rest of your code base helps with this, as does using very descriptive method names for your tests but I’m yet to find a good solution for this. Even writing detailed comments about what the test was for doesn’t always help as the concepts can be very complex and describing them in English becomes confusing very quickly. I’m not yet sure how we’ll handle this – perhaps a policy of running the acceptance tests before checking in changes to existing unit test cases would help ensure that the change doesn’t have negative impacts.

The confidence that tests provide shouldn’t be abused though – it doesn’t allow you to just randomly change things until you get the tests to pass or you’ll wind up with a big mess of spaghetti code that even your tests won’t save you from. Before you make any change you have to understand why your making the change and what the impacts of the change will be – the tests are just your safety net to catch you when you screw up. If nothing else, understanding the code you write to pass this test may help you write the next test that will break your code. Unless you have that understanding, you won’t be able to find all the corner cases that your clients will complain about.

Overall

Overall I think TDD is going to provide huge advantages for us, mostly in code quality. We may see some productivity improvements but I don’t expect them to be significant and we have a lot of work to do before we will see them because of the large amount of legacy code we have that is under-tested. It will be interesting to see how long it takes us to get good code coverage of our entire code base. I’ll also be interested to see how long it takes us to effectively rewrite the entire application through various incremental improvements and maintenance – this doesn’t mean setting out to replace the code but just the general maintenance to avoid code rot gradually updating the entire code base as required.