Another hallmark of open source is the support available in community forums, particularly for the more mature or widely used systems. But choosing to rely on community support instead of signing a service contract can be risky.
"People can Google for 90 per cent of the problems they run into, but the last 10 per cent may be killer if it's a mission-critical system," says Gartner's Driver.
It's important to understand the business impact of a catastrophic failure and have contingency plans in place to remediate the problems, he says. Reducing your risk might mean limiting your use of an application based on its maturity and the level of community support available, or choosing to pay for vendor or third-party support.
"If you have no service-level agreement, contract or warranty, you have shouldered the burden of responsibility," Driver says. "If you're able to do self-support, it's an upside, but if you can't, you have created unforeseen risk."
Of all the open-source software NPC uses, Brisbin opted to pay for support only for SpringSource tc Server, which it uses to deploy Web-based applications in an internal cloud. He went that route because the application server deployment is pushing the envelope of common developer knowledge. "We can't go out to a mailing list of 150 developers and ask questions, because not many people are doing this the way we are," Brisbin says. But he says he's happy that the contract didn't require him to purchase a license, and that it cost just a couple thousand dollars.
Organizations serious about using open source are also advised to establish policies and governance practices to monitor and control its use. Driver estimates that only 20% of organizations using open source have such policies in place, and in the Computerworld survey, most respondents said they didn't measure ROI. Taking such a risk can lead to unforeseen costs; for instance, even if you think you're reaping benefits, with no benchmarking or cost comparison, that could be an illusion, he says.
"People can be getting a negative ROI and firmly believe it's positive because they've gone from a [capital] expense to an [operating] expense," he says. In other words, the savings on license fees could be outdone by the salaries of employees who must spend eight to 10 hours a week updating, testing and patching the software.
In some cases, companies are realizing savings but can't prove it. "The key to minimizing the potential downside and maximizing the upside is governance," Driver says. "Without that, you're shooting in the dark."
At the New York State Office of Temporary and Disability Assistance, Chan is creating a direct comparison between the cost and performance of the new IT environment and the older one. He cautions that it requires an investment of resources to run tests and create meaningful benchmarks.
Sign up for CIO Asia eNewsletters.