More Work Ponderings

In a post last month, I mentioned some of my thoughts on Key Performance Indicators (KPIs). It received some great comments from someone named Alexey, who works for an Agile product management company (www.yoxel.com). I encourage anyone interested in Agile to read those comments, but I also thought it was time to re-energize the subject via a new blog post here.

As an aside: Alexey asks how to subscribe to this blog. I'm sure someone can provide the details (for an invaluable Blog Point) but since I've never subscribed to any blogs, it won't be me!

Just to wrap up the previous conversation, I do think a team can get some value out of comparing their estimates to the actual time spent, but in our environment that make take a lot of different forms. Some teams don't record the actual time spent at all, so in their case it's a question that would only really be asked if they failed to complete some of their committed work. In their Retrospective, they should ask, "Why didn't we complete everything?" and it's possible the answer will be, "Because we under-estimated the work." Most teams I've talked to would have a sense of what the handful of tasks were that ballooned the most, so they could then focus on figuring out how to identify those sorts of tasks in the future. Alternatively, they might conclude that they now know how to handle anything of that type, but what they need to recognize are those things they haven't done before. In other words, they'll know best how to adapt.

However, at least one team I know has taken to tracking their actual time spent, at least in the cases where it varies significantly from the estimate. This pinpoints those under-estimated tasks for them, for use in the Retrospective and subsequent planning sessions. One common pitfall of this approach that I've noticed is that almost no one ever reports a task taking less time than was estimated; when that situation happens, they're happy enough to get a little buffer and seemingly don't want to draw attention to it. Only in the case where they're not completing what they said they would will they admit that the work was mis-estimated. And even then, the information is delivered in a tone of excuse-making rather than lesson-learning, but hopefully that'll change over time. After all, that should be a good thing: we now know more than we did before!

In general, we (management) don't want to mandate that every team track its actual hours, because we're trying very hard to let the teams define their own processes, based on what works best for them. I guess in a case where a team consistently came up short, I might - as the Agile Manager - get in there and coach them on techniques for improvement, one of which would certainly be some sort of tracking of where their time was spent.

Which leads right into another point Alexey raised, which was the examination of planned work versus unplanned work, so as to have historical data to use in future planning. That's something that, again, some of our teams have started coming to on their own. I know that our VP has recently hit on this as a metric he'd like to have more information on, but I tend to look at it as simply being one means to an end. In other words, if a team isn't succeeding, one possible reason may be that they're getting hit with too many impromptu requests for their time. In that case, they may get value out of tracking those requests and using that to allocate their time in each Iteration. Another team, though, might find that it's left more-or-less alone to complete its planned work each Iteration, and so the amount of unplanned work they face is trivial and not worth tracking. One size shouldn't be expected to fit all, as it were.

A recurring theme within this blog entry, I think, is that KPIs should be decided by the team, based on what they get value out of. That sort of thinking is what lead me to recuse myself from the cross-company working group that was tasked with defining common KPIs between our three joint venture organizations. It became clear to me that the real mission of that group was to create a set of "progress metrics", that could be used by our parent companies to see how things were going. There's absolutely nothing wrong with that, and I can totally see why they'd want just such a set of statistics. But that's not where my interest lies, as I'm looking for measurements that the feature teams themselves can use, rather than reporting metrics. And as I've mentioned several times here, those KPIs may differ from team to team. What I've failed at so far, except in a few isolated cases, is in encouraging each team to think about this and start experimenting. That's going to have to be one of my missions over the next few months.