Smashing The "Change Code, Break Code" Mindset

One of the mantras at work, for as long as I've been there (7 years now and counting), has been "change code, break code." What that expression espouses is the belief that, anytime you change software, you're likely to break some existing functionality in the process. It's such a deeply-help conviction by many of the old-timers in the office (like me) that it pretty much never gets challenged when it's uttered.

And yet we've got at least one Feature Team who've bought whole-heartedly into Agile, including the importance of automated tests and the practice of re-writing code to make it clearer/more testable/more maintainable. This development has caused me to begin to question "change code, break code." Today, I decided to put it to the test, and I spent most of my day analyzing bug trends among a handful of our teams (including the one in question). What I wanted to see was whether defect-discovery really does go down if you're living by the Agile principles. To accomplish that, I extracted a whole bunch of data from our bug reporting system and produced a spreadsheet showing the "new defect" counts, by Iteration, for a sample set of our Feature Teams.

The results were amazing! For the team that's focused on (and been allowed to focus on) quality, as Agile defines it, there's been a very clear and distinct downward trend to the number of new bugs being found every month. This, despite the fact that the team has re-written huge chunks of their product over that same period, and reduced their code size by about 25% in the process! So much for "change code, break code!"

For every other team that I looked at, the "number of new bugs" trend-line is either flat or moving upwards! In most cases, these teams have been pushed to release more and more features, and as a result either haven't invested in automated tests and code re-writes very much at all, or at best have been able to do so in fits and starts. The difference between the results was even more stark than I'd expected to see.

I couldn't imagine a better sales pitch for the value of placing importance on quality than what I got out of this data-mining! As our Master Coach commented, if we factor in the cost of each defect - in terms of customer dissatisfaction, support time, reporting time, and the allocation of resources to fix the bug instead of adding new features - you'd have to be pretty thick not to see that we're talking about a savings of tens or even hundreds of thousands of dollars implicit in this approach (considering the # of bugs we're talking about) as compared to what we've done in the past.

I'm hopeful that this data will help bring about some changes, but of course, I've been fooled in that regard before! This could certainly be a whole chapter in another Agile book, if I ever wrote one!