Subscribe / Unsubscribe Enewsletters | Login | Register

Pencil Banner

Open source projects created by Facebook, Google, Twitter and LinkedIn

Matthew Finnegan | June 28, 2016
Some of the largest and most innovative companies in the world have offered their technology to IT communities

Facebook recently began to build custom servers based around Nvidia GPUs - chips that were originally intended for rendering computer game images but have proved to be well suited to deep learning.

7. Facebook - Torch

Facebook - Torch

Facebook uses deep learning internally to, for example, filter information on Facebook feeds, and has open-sourced some of the modules it created as part of the Torch deep learning framework. The algorithms created by its Facebook AI Research (FAIR) team are claimed to be faster than those already available in Torch, which is also used by others such as Google and Twitter.

8. Facebook - Cassandra

Facebook - Cassandra

Non-relational database Cassandra was created by Facebook engineers Avinash Lakshman and Prashant Malik as a means to power its inbox search function.

"Cassandra is a distributed storage system for managing structured data that is designed to scale to a very large size across many commodity servers, with no single point of failure," said Lakshman upon its public release in 2008. 

Although Facebook no longer uses Cassandra itself, it is used within other large tech firms such as Twitter, Netflix and Apple, while five-year software firm DataStax is helping popularise the technology among more traditional enterprises.

9. Twitter - Aurora, Storm

Twitter - Aurora, Storm

The social media firm is a major open source software user, and has contributed back to the community in a number of ways.

Its Aurora framework was created by Google developer, Bill Farner, taking a lead from Google's Borg microservices architecture.

Aurora builds on top of Apache Mesos and provides common features that allow any site to run large-scale production applications. It is able to make scheduling decisions, such as moving a service onto a healthy machine in the event of a failure, ensuring greater reliability. 

"Aurora is software that keeps services running in the face of many types of failure, and provides engineers a convenient, automated way to create and update these services," the company said in a blog post. A number of other companies are now using the software. 

Other projects include Bootstrap and Storm, which is used to analyse large-scale data streams created by millions of Twitter feeds.

10. NetFlix - Chaos Monkey

NetFlix - Chaos Monkey
Image: NetFlix

As a major AWS user, NetFlix wanted a way to test resiliency of its applications running in the cloud. Chaos Monkey was born with the aim of artificially creating problems with virtual machines hosted by the public cloud provider - unleashing its Simian Army to test that its systems are able to react to random failures on the network.

 

Previous Page  1  2  3  4  Next Page 

Sign up for CIO Asia eNewsletters.