QA in XP

I’ve seen a misconception a number of times where people believe that the regular release cycles and TDD practices in XP mean that you don’t need a standard QA process – that the developers are responsible for writing perfect code, first time every time. It’s true that XP practices can significantly improve quality, both of the code and the final product, but that doesn’t excuse the team from properly testing their work.

It’s important to note that TDD is a design process, not a testing process. It will only test the cases that the initial developer thought of. For changing code or when the developer can’t keep all the relevant cases in their head, the tests will eliminate regressions, but by definition they won’t cover any cases the developer didn’t expect. Somewhere along the line you need someone who’s not as close to the code to really test the code and cover all the cases – not just the ones the developer thought of. Pairing helps with this, but the pair is still too close to the code. To do it properly and ensure you’re releasing a quality product, you need an actual QA process.

Regular releases in XP make traditional QA processes impossible to work with – you can’t finish all the features, throw them over the wall to QA, fix all the bugs and repeat inside a one week iteration. Instead, QA has to be an ongoing process where the QA department works alongside the development team. Straight after the planning game, QA should assist the client in writing acceptance tests that provide in-depth testing of the story requirements to ensure all the corner cases are covered. Clients often don’t think of corner cases off the top of their heads so having someone with decent QA skills will make the acceptance tests much more detailed and ensure that when the story is signed off it actually does what was needed.

As the developers work on stories, QA needs to keep an eye out for any clarifications to requirements and extra cases that the user explains to the developers while they work and make sure they are reflected in acceptance tests.

Finally, once a story is completed by the developers, QA should become an advocate for the client and test out the feature for anything that looks wrong while the client goes through and checks that it meets their requirements. This is where anything that can’t be covered by an automated test gets tested and the testing requirements documented so those tests can be run manually regularly.

During their slack time, the QA department should research ways to automate the currently manual tests, go back and run the manual tests to make sure they still pass.

The QA department might be a group of specialists or they might be the same people as in the development team – it’s the process and the allocation of time that matters more than the people. However, QA requires a distinct set of skills to development and a lot of developers aren’t much good at doing QA. If your development team doesn’t have QA skills you need to either train them up or get in QA specialists. My general inclination is that it’s best to have QA specialists who have a real passion and talent for testing software, but that your regular development team should regularly pair with them on QA work (and them with developers on non-QA work) to make sure that the team values and understands the work they do. It’s also important to avoid a divide between developers and QA – both teams are there to deliver value to the client, not to fight about whether a behavior is a bug or a feature request etc. QA and development need to be one team that works together and sits together.

One other advantage of keeping QA and development working together, is that developers fix the bugs that QA finds straight away instead of building up a big backlog that then delays the release while all those bugs are fixed and verified. Keep on top of your bug list and QA processes and you can ship top quality software in predictable timelines without the death march of bug fixing that traditionally happens at the end of projects.