The demands on quality are greater than ever, with so many software applications passing data through shared networks and cloud environments. With more critical systems being developed for healthcare, finance, government and defense, companies are rightly concerned with security and vulnerability issues. They are also faced with the difficulties of uncovering complex issues that would be expensive to set up and test in-house.
So what to do?
One avenue many organizations are pursuing is crowd-sourced bug detection. This is more than just complaining on Twitter or reporting issues to support. There are actual organized, and sometimes highly paid, methodologies behind this. Here are a few of the strategies companies are employing in order to tap into the larger, more varied world that real users represent.
Yawn. Not so new — we've been around the beta block a few thousand times as an industry. From managed betas to private betas to public betas... we've adopted them all. The benefit of a beta test is that it is typically time-bound and overseen by someone in the company. The downside is that is it time-bound and overseen by someone in the company. What do I mean by that? Well, we all bring our own biases to the table whether we mean to or not. When companies select beta customers based on some criteria and/or assign them tasks to perform, they often miss crucial issues because they limit customer actions and environments.
This is slowly becoming a lucrative business because many companies see the value of "in the wild" testing, especially in the mobile space. You can only create and re-create so many user scenarios in a lab environment unless you have unlimited time, money and resources. Using a crowd-testing service can provide you with much-needed insight into your application quality and uncover some difficult-to-find bugs in the process. But beware — if you want clean defect reports with steps to reproduce and analysis of user impact, this is not where you'll get that. I prefer to think of crowd testing as a great ongoing feedback loop outside the normal rigor of your in-house testing.
"Find that potential bug and bring it in — we'll pay you a bounty." That's the premise behind bug bounties, which are primarily aimed at researchers rather than testers. Having a security breach can be very costly, as any of those companies who have been hacked can tell you. But it's very difficult to unearth security flaws despite code reviews and security testing - so what companies are focusing on are the potential problems in their applications, otherwise known as vulnerabilities. It's a win-win for both the researcher, who has the potential to make a lot of guaranteed money from finding a security flaw, to the company, which would otherwise face enormous payouts to consumers and government for security breaches. Hiring the right people full-time to look for these flaws is also a costly proposition without a long-term value prop.
Sign up for CIO Asia eNewsletters.