Subscribe / Unsubscribe Enewsletters | Login | Register

Pencil Banner

Are you over-testing your software?

Matthew Heusser | July 8, 2015
Release candidate testing takes too long.

How long regression cycles are born 

Software developer Abby Bangser was describing a project she recently worked on that build the capability to deploy continuously. For business reasons, the team wanted to deploy every iteration, which is fine. At the end of the iteration, one of the leaders asked Bangser to do a "five minute check," just to make sure everything was fine. 

She refused. Why? Because if managers are asking developers to spend five minutes exploring, it's because they don't have enough confidence in the system as defined either the quality of the code, the tooling, the ability to notice problems or, perhaps, the ability to roll back. Bangser wanted that confidence. 

Why? Why is five minutes a big deal?

Because that's how these legacy applications ended up with release-test windows that are a month long: They started with five minutes, and grew five minutes at a time. 

Another look at cadence 

Companies that have a large release-candidate test process got that way for a reason, and it is likely the reason included the reason the company still survives. Release candidate testing brought it success. Dropping it seems like foolishness. 

In some cases, it might be. If you switch out the Web server, integrate the entire login system with Google's user IDs, or any other sweeping change, you might batch the work up for months and hide it behind config flags, and even do some candidate testing before release. Even if you're an Etsy, Twitter, IMVU or another media darling. 

The key is to do just the right amount of release candidate testing, to trade risk for time in the smartest way possible. That might mean flexing the cadence up or down based on risk. 

Ask yourself the tough questions: How long does the process take now? What is your cadence? Are you subtracting five minutes from that every two weeks or adding? And what are you going to do about it?

 

Previous Page  1  2  3 

Sign up for CIO Asia eNewsletters.