Tuesday, December 05, 2006

An Interesting Solution Is Proposed

During today's Fraternity of Feature Leads meeting, Kevin threw out a solution to one of our most nagging problems: how to get some of the manual testing done that used to be performed by our large QA department at the end of projects back in the Waterfall days. These include testcases that don't automatically fall into the domain of a single Feature Team, as one example; another would be heavily manual testing that can't easily be automated. The suggestion was to include an integration period (possibly an entire Iteration) prior to a milestone of some sort, such as a Release. This, on its own, is nothing new, as many of us talked about this back in the Spring. What was new today was the elaboration of having a list of difficult tests prepared, and then having the various Feature Teams execute them, as appropriate (by what they support). A difficult test, by definition, would be anything that both needs to be run in order to qualify our product as being releasable, and which can't easily be run each Iteration. This might mean it's a manual test, or it could even be an automated test that possibly requires a long duration or periodic manual intervention.

In our discussion, it was further suggested that the driving force behind this approach has to be the Product Team. First and foremost, that group has to buy into the need for an Integration Iteration (for example), since that's time during which the Feature Teams would not be producing new features. The Product Team also has to guide the prioritization of what tests to include. Simply taking all of the tests that QA used to run would be overwhelming, since many of them are now covered off by automated testing, while others may simply be deemed not worth the effort involved in running them. Those sorts of decisions are best handled by the Product Team, with input from the Feature Teams where applicable. While that unfortunately implies a certain aspect of "... and then a miracle occurs", due to the more general problem currently of Product Owners being stretched too thin, it's still the first solution to this problem that actually seems to have a good chance of success. At least that's my initial reaction.

Now it's time to start talking this one up at work and see what comes of it.

2 comments:

Jimmy said...

Would this take care of cases like:
- how does feature X deal with EAS?
- stress testing (i.e. do we still run as well after 300 DB updates)
- ad hoc crazy combinations like changing all my series recordings just after we detect a new DB update

Anonymous said...

Well, I must say that it's nice to see my topic finally reach the top of the backlog, too bad it was on a day that I could not attend the session :)

I'm glad to see that the results of the conversation were so positive, as I primarily left the topic on the table to help solve some potential problem areas I anticipate in the wake of "going Agile". The creation of so many "feature teams" while pillaging the vast majority of the support groups that glued the waterfall process together would almost certainly leave some gaps to be filled. Integration testing is but one such area...