Framing The XP Principles

A while back Ben Hyde wrote his thoughts on the key XP principles in What every you doing, it’s wrong! I’m not sure I fully comprehend exactly what Ben is trying to say but a lot of it seems opposed to the way I see XP in theory and to the experience I’ve had in implementing XP.

I’ll start with Ben’s rewording of the summary of Extreme Programming (see also James Mead’s original):

• integrate continiously assures you take only small steps
• writing tests first assumes the code and platform has nothing to say about the problem
• pairing programmers assures you leave a large swath of good talent for your competors to hire
• refactor mercilessly assumes you have large code bases rather large installed bases, poor you.

Firstly, integrating continuously, does ensure you take small steps and that’s a good thing. Ask Netscape about how successful throwing out an entire code base and rewriting it is. Sure Firefox is becoming popular now, but Netscape has all but gone out of business because of one bad technical decision to rewrite their entire browser instead of taking small steps and fixing the existing one.

There are more benefits to small steps – by taking small steps you get quicker feedback, you know sooner that something is broken and there are fewer changes that could have broken it. You’re also less likely to break things in the first place because small steps are easier to fully understand, so it’s less likely that you’ll miss something.

Most importantly though, taking small steps doesn’t mean that you can’t make large changes. You just make large changes one step at a time. You can achieve the effect of any large step you might want to take in small steps, with the added benefit that you know sooner if it’s not going to work and you may realize you don’t need to go the whole way because the benefits are achieved by just doing part of the work.

I’m not sure what the criticism of test first is meant to be – writing tests firsts mean you do a small piece of design up front (by writing a test) and then write the code to execute on that design. When you write your tests you should be considering the strengths and limitations of both the existing code base and the platform you’re working with. In addition, you get the ability to verify that your code matches your design and that it does what you think it does by running the tests. That doesn’t mean you’ll have bug free code, but you will have code that works in all the cases you thought to write a test for, considering you have a test for every branch and every side-effect (or at least you should if you’re actually doing TDD) is a pretty high level of quality and provides a lot of confidence in your code.

The pairing programmers rewrite is the biggest falsehood in this list. If you have a good team who actually have social abilities then pairing is fun. Our office comes alive with happy chatter and laughter as people pair because we’ve built a team that gets on well together and focussed on enjoying our work. Frankly, I don’t care how brilliant you are at writing code, if you don’t have the social skills to interact with your co-workers you are not a talented programmer, you’re a code monkey and I wouldn’t want you on my team.

It’s not XP that makes communication important, software engineering has always required good communication skills and it’s a shame that so many people in the field don’t realize that. You almost always have to work with a team in software development, you almost always have contact with other parts of the business and if you want to develop good software, your engineering team should be in regular contact with customers so they understand their problems. Besides which, wouldn’t it be nice to have confidence that when your biggest customer has problems you can send one of your developers out to help them fix the problem? You can’t do that unless you value social skills when you hire engineers, yet even really big companies value the ability to do just that.

Finally, the refactoring rewrite shows a complete lack of understanding about what refactoring is. Refactoring is not about changing public APIs, it’s not about constantly shifting the product under your user’s feet. It is about changing the implementation of APIs for the better, without affecting the end result. As Martin Fowler put it, refactoring is “a change made to the internal structure of software to make it easier to understand and cheaper to modify without changing its observable behavior” (Fowler, Martin. Refactoring: Improving the Design of Existing Code. P. 53).

If you are refactoring instead of redesigning, your users will never see the difference except perhaps for a smaller binary size (due to removed duplication in your code) and fewer bugs (due to the code being more maintainable). The functionality of your program, the public APIs and the user interface should all remain unchanged when refactoring.

As an example, Ephox has a very large installed base – not only do we have thousands of clients around the world, most of those clients deploy to a large number of end-users, add in the OEM deals we have and the number of people affected by changes to our product is pretty immense. Even more importantly, we regularly deal with very large enterprise development projects that don’t upgrade regularly and have very long test cycles to deal with any changes. If we were to break our backward compatibility we’d get huge numbers of complaints from our existing clients.

Despite that, we refactor mercilessly to keep our code maintainable. In our last major release we completely refactored out HTML list editing code – users would only have noticed that bugs were fixed. In the major release before that, we completely refactored the way we handle loading the editor interface based on the configuration file we’re given – none of our clients or end-users noticed, but we did because the new code was far more maintainable. In the next major release we’ll have done some refactoring to nearly every part of our code, but it won’t affect our public API at all. The only changes users will see are those that we deliberately decided to make because the product manager wanted to improve features or add new features, but all of it is 100% backwards compatible.

It’s worth noting that encouraging refactoring runs counter to the advice to integrate continously. Refactoring is fundimentally a choice to buy a bag of disintegration.

This is also false. You simply refactor in small steps and integrate continuously. You should definitely not be forking a project just to refactor it, you shouldn’t even create a separate branch in version control. Refactoring should happen continuously, every day you’ll find something that could be better be it because it wasn’t perfect when it was first written or because the new code you’ve added means there are extra demands on its flexibility. Refactoring should just be a routine day to day habit so that you ensure you keep your code base in top condition all the time.

All that said, if you happen to be working on a library instead of a product, refactoring can be more difficult, particularly if you’ve failed to clearly denote what APIs are intended for public use and which are just available because of limitations of the language you’re using. That pain only extends as far as the public APIs though, you should still refactor your internal code mercilessly.