Facebook's cold storage facility in Prineville, Oregon. Credit: Facebook
When it comes to storage, Facebook is learning to do more with less.
For backup copies of older content, the social network is building "cold storage" facilities that are designed to keep data available without some of the expensive, power-sucking features found in a traditional data center. Facebook says it's built in strong protection against data loss while reducing the overhead of additional storage.
These are data centers designed to hold more than an exabyte of data -- 1,000 petabytes -- with no redundant electrical systems, while consuming less than one-sixth as much power as a conventional facility. And they store all that data on cheap, consumer-grade media.
At the heart of the system is software, including a program that keep most disks idle most of the time and one that can reconstruct a lost or corrupted file without having a full duplicate stored in the data center, the company says.
When Facebook builds a user's News Feed or Timeline, it gets the photos, videos and other elements in it from so-called "hot" storage in data centers located around the world. If a data center with hot storage fails, another one can take over and deliver the bits without users noticing, said Kestutis Patiejunas, a Facebook software engineer.
Cold storage comes in when that failed data center has to be brought back online and all its data restored. In the past year, Facebook has opened two such facilities, in Prineville, Oregon, and Forest City, North Carolina. Facebook had a chance to design and build them with new technology from the ground up, and it implemented new ways of saving space and power.
The centers initially have been built around racks of 4TB hard disk drives, though Facebook could use the same techniques with Blu-Ray discs and cheap flash, two other kinds of media that it's exploring for cold storage, Patiejunas said.
Facebook designed the hardware using the Open Vault specification from its Open Compute Project, but modified it for the new centers.
For one thing, the company set up the systems so that in each tray, only one hard drive could be running at any given time. It wrote software to manage the order in which bits are accessed so that the activity in any given tray was only happening on one disk at a time, Patiejunas said. With fewer disks running, the system can get by on less power and stay cool with fewer fans, while the task set out for the cold storage -- replenishing data to the hot storage that feeds users directly -- still runs fast enough.
Sign up for CIO Asia eNewsletters.