Even though there are technologies available that can enable the safe use of encryption when cloud servers are involved, getting everything right and ensuring that there are no errors in the overall implementation can require a lot of resources.
"It can be done, but it takes a lot of forethought, a lot of effort, and the use of true end-to-end encryption will increase your costs," Melancon said. "It may also require you to rewrite applications, or switch providers in order to handle all aspects of end-to-end encryption."
When considering that NSA's primary mission is the gathering of foreign intelligence, companies that are not based in the U.S. should probably be even more concerned about the recent revelations regarding the agency's surveillance efforts.
"If you're a European company dealing in sensitive corporate data, I think you'd be crazy to use a U.S. cloud service," Green said. However, that won't stop companies from doing it, he said.
"A big part of the political scandal in the USA right now is the fact that the NSA is spying on Americans," said Zooko Wilcox-O'Hearn, co-founder of the Tahoe-LAFS project, a distributed, fault-tolerant and encrypted cloud storage system. "However, absent evidence to the contrary, I would assume that the NSA is at least as effective at spying on data in European and other locales as in American locales."
That said, Wilcox-O'Hearn believes that companies should also be concerned about other actors spying on them. Those could include law enforcement, military and intelligence organizations from other countries, as well as organized crime gangs or corrupt employees of telecommunication companies and ISPs, he said.
Banks and other financial organizations, as well as companies from the telecommunications industry, that handle very sensitive data usually prefer to keep it on their servers, under their control, primarily because they need to meet regulatory compliance and can't perform security audits in the cloud, said Sergiu Zaharia, the chief operations officer at Romania-based security consultancy firm iSEC.
Such organizations use encryption to secure the traffic between their different branch offices or between customers and their publicly accessible services, but very few of them encrypt data as it travels through their internal networks, between their own servers, at least in Romania, he said.
Other companies, like small online retailers, that choose to use cloud servers to run applications and store customer data don't care too much about encryption or if they do encrypt the data, they don't care if the service provider has access to their encryption keys because they usually don't perform an advanced enough risk analysis, he said.
"All our customers have highlighted their concern with security issues, especially when it comes to services hosted in a third party location," said Dragos Manac, CEO of Appnor MSP, a provider of managed dedicated servers and cloud computing with infrastructure in both Europe and the U.S., via email. "The current Prism scandal is a major blow for governments, but it also hurts service providers."
Sign up for CIO Asia eNewsletters.