Sunday, April 15, 2007

More Work Ponderings

In a post last month, I mentioned some of my thoughts on Key Performance Indicators (KPIs). It received some great comments from someone named Alexey, who works for an Agile product management company (www.yoxel.com). I encourage anyone interested in Agile to read those comments, but I also thought it was time to re-energize the subject via a new blog post here.

As an aside: Alexey asks how to subscribe to this blog. I'm sure someone can provide the details (for an invaluable Blog Point) but since I've never subscribed to any blogs, it won't be me!

Just to wrap up the previous conversation, I do think a team can get some value out of comparing their estimates to the actual time spent, but in our environment that make take a lot of different forms. Some teams don't record the actual time spent at all, so in their case it's a question that would only really be asked if they failed to complete some of their committed work. In their Retrospective, they should ask, "Why didn't we complete everything?" and it's possible the answer will be, "Because we under-estimated the work." Most teams I've talked to would have a sense of what the handful of tasks were that ballooned the most, so they could then focus on figuring out how to identify those sorts of tasks in the future. Alternatively, they might conclude that they now know how to handle anything of that type, but what they need to recognize are those things they haven't done before. In other words, they'll know best how to adapt.

However, at least one team I know has taken to tracking their actual time spent, at least in the cases where it varies significantly from the estimate. This pinpoints those under-estimated tasks for them, for use in the Retrospective and subsequent planning sessions. One common pitfall of this approach that I've noticed is that almost no one ever reports a task taking less time than was estimated; when that situation happens, they're happy enough to get a little buffer and seemingly don't want to draw attention to it. Only in the case where they're not completing what they said they would will they admit that the work was mis-estimated. And even then, the information is delivered in a tone of excuse-making rather than lesson-learning, but hopefully that'll change over time. After all, that should be a good thing: we now know more than we did before!

In general, we (management) don't want to mandate that every team track its actual hours, because we're trying very hard to let the teams define their own processes, based on what works best for them. I guess in a case where a team consistently came up short, I might - as the Agile Manager - get in there and coach them on techniques for improvement, one of which would certainly be some sort of tracking of where their time was spent.

Which leads right into another point Alexey raised, which was the examination of planned work versus unplanned work, so as to have historical data to use in future planning. That's something that, again, some of our teams have started coming to on their own. I know that our VP has recently hit on this as a metric he'd like to have more information on, but I tend to look at it as simply being one means to an end. In other words, if a team isn't succeeding, one possible reason may be that they're getting hit with too many impromptu requests for their time. In that case, they may get value out of tracking those requests and using that to allocate their time in each Iteration. Another team, though, might find that it's left more-or-less alone to complete its planned work each Iteration, and so the amount of unplanned work they face is trivial and not worth tracking. One size shouldn't be expected to fit all, as it were.

A recurring theme within this blog entry, I think, is that KPIs should be decided by the team, based on what they get value out of. That sort of thinking is what lead me to recuse myself from the cross-company working group that was tasked with defining common KPIs between our three joint venture organizations. It became clear to me that the real mission of that group was to create a set of "progress metrics", that could be used by our parent companies to see how things were going. There's absolutely nothing wrong with that, and I can totally see why they'd want just such a set of statistics. But that's not where my interest lies, as I'm looking for measurements that the feature teams themselves can use, rather than reporting metrics. And as I've mentioned several times here, those KPIs may differ from team to team. What I've failed at so far, except in a few isolated cases, is in encouraging each team to think about this and start experimenting. That's going to have to be one of my missions over the next few months.

4 comments:

Anonymous said...

To subscribe to your blog, point your RSS aggregator to:

http://kimota94.blogspot.com/feeds/posts/default

cjguerra said...

For a long time, way before the company started going towards Agile, I've been bothered by the discontinuity between estimates and actual time taken. There were many occasions when we were asked to produce an estimate for some task. This has always been difficult, but the more I thought about it, I realized that I was not learning from each estimate. To become a better estimator, I would need to have both the estimate and the actual time it took to adjust my next estimate.

This is a crude description, but I think that the new estimate needs to be similar to the old one. Change is obvious, but the more information fed back into the estimate process, the better the estimate will become.

Many have commented that estimating software projects is difficult and it is very different from other occupations. Software development is difficult, but so are many other businesses where estimation is an essential component. What is different between these other occupations and software development is that the other businesses are dependent for the profit on these estimates, so the participants get very good at estimating very quickly.

Surprisingly, you've read to the end of this comment, and I'm trying to say that it would be prudent to record how long something took. An Agile workplace should use this as a non-judgmental piece of information. As stated above, the exact form of this information is something each Agile team needs to determine.

Anonymous said...

Just to follow up Croptop's comment, you can also subscribe to an individual post's comments; the link is right down there below "Post a comment". There's also a blog-wide comments feed.

Kimota94 aka Matt aka AgileMan said...

OK, I'm awarding Croptop and PeterJ a Blog Point each for taking the time to post their subscription suggestions. Thanks, guys.