Defect Driven Testing: Your Ticket Out the Door at Five O'Clock

Test automation is not a controversial topic in most circles. Even developers who don't write automated tests agree it's a great idea. They just don't have time to work on it very often. The idea of having your code verified automatically sounds great, but it never rises high enough on their priorities to ever get done, and that's a shame. Effective test automation is a remarkably effective way to keep your code clean, which helps you avoid those late night debugging sessions.

It's my opinion that most developers don't get a good introduction to test automation. When most developers hear the phrase "test automation", they think only about Test Driven Development (TDD). Unfortunately, TDD is a difficult practice to learn. Don't misunderstand me though... TDD is a powerful technique. Developers that master it have put a valuable technique in their toolbox. It's just a difficult practice to pick up without help.

I like to start developers with a different introduction to automated testing. Defect driven testing, or DDT.

DDT is a fairly simple concept. When you find a bug, add a test.

Why take this approach?

First, no one can dispute the need for the test. If an issue was found, then it had been missed earlier. Perhaps the developer missed it and QA spotted it. Maybe it slipped past everyone and it was reported by your customer. Whenever it's reported, it needs to be fixed in a way that will prevent it from reappearing.

Secondly, test automation is very expensive. It takes an investment of time from your company's most valuable resource. You. So how should you invest that time? As effectively as possible. When you're first starting your test automation work, you won't know where to focus your work. It takes time to understand how test automation can most effectively be used. Instead of trying to figure this out for yourself, let the problems in your product guide your work. DDT provides an extremely focused set of tests.

Third, DDT is a gradual approach. TDD takes developers, who don't like being told where commas or brackets should be, and turns their entire development style upside down. While many, myself included, would argue that this needs to be done for many developers, I'm also very practical. If I lose your attention and you stop writing automated tests, then what have I accomplished? DDT lets you add another test every time you encounter a bug. This more gradual approach doesn't require an upfront investment. Instead you add to your test gradually.

There's an additional trick you can use when using DDT. I call it testing jazz. Consider jazz to be variations on a theme, and apply that to your tests. Bugs tend to cluster, so never write one test to cover a single bug, then moving on. Instead stop and devote a bit more time. Instead of writing one test, try to add a dozen. Don't create completely different tests, but try to add minor variations to your original. If the bug is exposed by passing in a string with a single space in it (like "hello world"), then try to pass in "helloworld", "hello wor ld", " hello world ", "h e l l o  w o r l d", and so on.

Over time you'll find that DDT creates an extremely effective test suite that targets the most problematic parts of your code base. Run your defect driven tests inside of a continuous integration system and you'll find your code running more cleanly every day. Six months from now you'll look back and wonder why you ever had to work so much overtime.