Facebook found hotspots in its cache, where frequently consulted data could cluster in small areas, causing bottlenecks
Facebook continues to push the boundaries of storage and server technology in order to more quickly serve its billion users, and the results are being offered as open-source technology that can also benefit other companies.
Recently, Facebook updated its internally developed caching software, called Flashcache, to more efficiently use the thousands of solid-state drives (SSDs) that the social networking giant deploys to store frequently consulted data.
The newly released Flashcache 3.0 is able to make better decisions about what data to cache, while reducing the amount of wear and tear on expensive flash disks.
"With these improvements, Flashcache has become a building block in the Facebook stack," wrote Domas Mituzas, a Facebook database engineer who authored a blog post explaining the updates to the open-source software.
The work aims to improve overall Facebook performance without unduly driving up operating costs.
"While the cost per GB for flash is coming down, it's still not where it needs to be," Mituzas wrote. Given the premium prices commanded for SSDs, Facebook doesn't want to wear out these disks too quickly. "SSDs have limited write cycles, so we have to make sure that we're not writing too much."
Flashcache is one of a number of software projects that Facebook originally developed in house that the company has also released as open source. Earlier this year, for instance, the company also released a virtual machine, called HipHop, that speeds the processing of PHP code.
The company hopes that other organizations could reuse such programs as HipHop and Flashcache and eventually contribute to their further development. Like other open-source caching software such as memcache and Redis, Flashcache can be used to speed the responsiveness of a heavily visited website or popular Web application.
Facebook originally created Flashcache to boost the responsiveness of the MySQL databases that store user data. The software can be loaded onto the Linux kernel as a module without making any changes to the kernel itself.
The idea behind Flashcache is to use SSDs to hold the material that is most requested by users. SSDs tend to be faster than traditional rotating platter hard drives, though they are also more expensive by the GB when compared to hard drives. So it would not be cost-effective for Facebook to store all of its data on SSDs, especially if the vast majority of Facebook user data is rarely consulted.
Sign up for CIO Asia eNewsletters.