Subscribe / Unsubscribe Enewsletters | Login | Register

Pencil Banner

LinkedIn open sources its database change capture system

Mark Gibbs | Feb. 28, 2013
OK, lots of interesting stuff for you this week. First up, LinkedIn has open sourced a system called Databus, a real-time database change capture system that provides a "timeline-consistent stream of change capture events ... grouped in transactions, in source commit order."

On the other hand, if you're like Backblaze, an online backup company, you'll find yourself thinking, "Wow, that kind of pricing will break the bank," so you might choose to do what Backblaze did: Build your own high-density storage systems.

I had a very interesting talk with Backblaze's CEO Gleb Budman, and he told me they had, indeed, looked at the big boys and worked out that storage from NetApp or Drobo would cost them 10 times the cost of the raw hard disk drives. Given that Backblaze offers unlimited online storage (still priced at a measly $5 per month since they opened the doors!), that wasn't a formula that would work for them.

After some head scratching, Backblaze decided to "roll their own" and designed a workhorse storage platform from off-the-shelf components. They then did something remarkable: In 2009 they open sourced the design!

They initially thought that a handful of people would be interested but, to their surprise, there was a huge amount of interest (their blog announcing the open source design got 1 million hits) because, come on, who wouldn't get excited about 67TB of storage for less than $8,000? (OK, perhaps your family and friends might not be but you get it, right?)

Now, roll forward three years and Backblaze has just published its latest iteration of what they call the Backblaze Storage Pod: Storage Pod 3.0.

This latest version packs up to a total of 45 drives on nine five-port multiplier backplanes with a Supermicro MBD-X9SCL-F motherboard and an Intel Core i3 processor i3-2100 with 8GB of RAM into a 4U custom rackmount case with a 760 watt power supply, six fans, and two Gigabit Ethernet ports. Fully stocked with 4TB drives, which can be had for $195 each, a 180TB pod built by you will set you back about $11,000 or $59.54 per TB. Now, that's cheap!

Should you think that using 4TB drives is gilding the lily, Backblaze notes: "While it looks like the cost of a 4TB drive system is more expensive [than using 3TB drives], when you factor in rack space, electricity, installation labor, etc., the long-term cost for Backblaze leans towards using 4TB drives. Our monthly cost for a full rack of Storage Pods with 3TB drives is $0.63 per TB, while a full rack of Storage Pods with 4TB drives is $0.47 per TB. When you factor all the costs together, it takes about five months for us to recover the extra cost encountered when building 4 TB based Storage Pods."

It's up to you what you want to run on the processor to manage and deliver the storage, but you might go the same way Backblaze has: A customized version of Debian Linux with the Ext4 filesystem to which they have added their own proprietary storage management storage which allows new pods to be automatically integrated into their system (you'll have to figure that bit out for yourself).

 

Previous Page  1  2  3  Next Page 

Sign up for CIO Asia eNewsletters.