Also, commercial switches were expensive, given that they offered a lot of features to ensure the highest reliability.
Google engineers came to realize that if they designed their network differently than the industry norms, they wouldn’t have to rely on such expensive switches. This is where Clos came in.
Developed in 1952 by Engineer Charles Clos for the telephone industry, the Clos topology describes the best way to set up a network with lots of end points that may need to talk with one another.
“It’s a really beautiful idea,” Vahdat said, speaking of Clos. “It basically allows you to take very small commodity switching elements and, by arranging things appropriately, be able to scale out to an arbitrary size.”
“We certainly didn’t invent this, but we were able to rediscover it, leverage it and apply it to our setting,” Vahdat said.
The Clos designed also allowed Google to manage the entire network as a single entity.
Typically, Internet grade switches are geared towards making many of the decisions about the best place to send a data packet so that it gets to its destination. For an open-ended network like the Internet, this approach made sense.
Google didn’t need all that intelligence in its switches, because it already knew the topology of its data center network.
So instead, Google controlled the routes through a centralized operation, which issued instructions to each switch about where to send their packets.
In effect, Google was practicing network virtualization before the term was a buzzphrase, Vahdat said.
The lessons Google learned could be useful to other enterprises and Internet services companies. Many enterprises now have the infrastructure that Google itself had when it started investigating this problem 10 years ago, and may be facing similar limitations, Vahdat said.
Sign up for CIO Asia eNewsletters.