The room where it rained: Facebook's Prineville, Oregon data centre. Photo: Facebook/AP
In the early days of Facebook's first data centre, an actual indoor rain cloud brought down the company's web cloud, according to Jay Parikh, Facebook vice president of infrastructure engineering.
In 2011, Facebook's Prineville, Oregon, data centre suffered a problem in the building management system that caused the air used to cool the servers to reach temperatures of more than 26 degrees and the humidity to exceed 95 per cent. As a result, the air condensed and formed a rain cloud inside the data centre.
"I got a call, 'Jay, there's a cloud in the data centre,' " Parikh told the Register. "'What do you mean, outside?' 'No, inside.'
"It was raining in the data centre."
The problem occurred because Facebook was using outside air to cool its centre, rather than traditional cooling systems that use a great deal of electricity, the Register report said. After the building management system broke down, it began recirculating high-temperature and low-humidity air through a water-based evaporative cooling system. That eventually caused the air to essentially become liquid and cover the servers in water.
Parikh said some servers broke entirely and that for a few minutes it was possible to stand in the data centre and hear the company's servers pop and fizzle.
"This is one of those things," Parikh said. "When you are 100 per cent air-cooled, it's awesome from an efficiency perspective, but the range you have to operate in is much, much wider."
The incident hasn't occurred again, though. Parikh said Facebook now seals its servers' power supplies with a rubber raincoat to keep water from destroying them.
Sign up for CIO Asia eNewsletters.