Tracking Estimate Accuracy

Over the past few weeks we’ve been working on tracking our velocity and trying to get a grip on how many points a week we can do. We haven’t really rolled out proper iterations yet, but we pick a set of stories to work on for the week and then set about doing them. At the end of the week we add up how many points we completed. This has been working quite well except that we are regularly doing more points than we expected but not nessecarily completing any extra stories.

The problem is that our estimates aren’t accurate enough. Sometimes this is because we estimate a story at a 1 or a 2 and it turns out to be a 3. Sometimes it’s because we estimate a story as a 3 when we should have broken it down into multiple stories. Often though, we’re seeing our estimates are out because there is a dependent story that we didn’t know about at the start of the week.

So over the next few weeks we’re going to try to get a handle on our estimates, see how accurate they are and try and improve on that accuracy. So far I see a three pronged approach:

1. Any story that is taking longer than expected should be brought up at the morning stand-up meeting so that any required help is made available. This also means we’ll know about potential problems early.
2. Track and graph the difference between our estimates from the start of the week and the final number of points we thought the story was worth. This will let us look back at things that seriously blow out but also to pick up if we’re just consistently underestimating things.
3. Add a bit more into our planning game so that the engineers actually go through the process of brainstorming required tasks for each story and then estimating them instead of just going on gut feel.

The last item is likely to be the most beneficial but it will take a while to ramp up and we only get a shot at it each week so it will take another week before we even get started on it. We haven’t done this previously because we’ve kind of snuck up on iterations and the planning game. Our whole approach to adopting XP has been like this – find the most pressing need and implement the part of XP that best solves it, then move on to the next need.

At first we just threw a huge number of stories up on the wall and started work on them, but that gave us no indication of our velocity. So we started planning what we’d do each week and tracking how much we got done – the first step towards doing proper planning games. Now we’re seeing a need to get more accurate estimates so we’ll start rolling out tools and techniques that help us estimate.