Velocity and Estimate Accuracy

In my previous post on tracking estimate accuracy, Stephen Thorne commented:

I thought the entire idea about estimates is that they should be self correcting? Estimates made in some arbitary numbering system, you sum up the estimates, divide by the time actually taken, then you have an accurate picture of how one developers estimates map to reality…

The ability to map arbitrary estimates to actual time that Stephen refers to is velocity. Essentially, how many points per time period can the team complete? This is a really effective technique if your estimates are consistent. That is, you may find that 3 points takes on average, a day to complete. So your velocity is 3 points per day1. However, if your estimates aren’t accurate, you may have some 3 point stories take 15 minutes, and some take a week.

Clearly, estimates that vary by that much make scheduling and planning extremely difficult and reduce the confidence you have in your measure of velocity – what if one week you happen to pick 5 week-long 3s? To avoid this, it’s important to try to make your estimates consistent. It doesn’t really matter if a 3 is half a day or two days, as long as it is a consistent measure of work.

One of the most common ways that this happens is that the original estimate didn’t take into account some prerequisite work that needed to be done. Addressing this problem is why we plan to start brainstorming tasks required when we estimate – hopefully by spending some more time thinking about what’s required, we’ll find less unexpected surprises.

Another common way that estimates turn out to be wrong is that the team has to pay back code debt as part of the story. This is significant refactoring that will pay itself back over time, but makes the task at hand involve more work than expected.

A less common, but particularly painful case is when unexpected or unplanned dependencies occur between stories. In an ideal world, all stories are independent of each other, however sometimes it doesn’t work out that way and it turns out that another story needs to be completed before this one can be. This case doesn’t really make for a bad estimate, as much as it throws out the planning for what can be completed. The estimates for each of the stories are probably right, but in the planning game, both stories should have been played in the same iteration instead of just one. As a result, the team needs to drop stories unless they get lucky and have a higher than expected velocity.

1 – Typically, velocity is tracked per iteration and is reported as just a single number rather than "per" some timeframe.↩