Given VMware's NSX product is designed to handle virtual switching in VMware virtual server environments, you might think the company would be a big proponent of the idea that servers will eventually subsume switches. But even though the server-as-a-ToR switch architectural model is being proposed for hyperscale environments, Guido Appenzeller, chief technology strategy officer in VMware's Network Security Business Unit, has never seen it used.
"If you want to get rid of ToR altogether, you need new silicon in servers," like packet sorting engines, Appenzeller says. "You probably need a mini switch in the server. It doesn't work with today's architecture."
That mini switch would be an Ethernet device enabling a direct server-to-server fabric. Another option would be a Layer 1 cross connect and multiplexer on the server motherboard, Appenzeller says.
Appenzeller gives the nod to the Ethernet mini switch implementation due to its familiarity in the server world, and its ability to do virtual LAN separation, something optical cross connects cannot do, he says. "I've never seen either deployed," Appenzeller says. And both may be impractical given the steadily dropping price of ToR switch ports. "ToR prices are coming down quickly."
Dell'Oro Group group agrees. The company reports that the average selling price of a 10G Ethernet port will drop from $715 to $212 between 2011 and 2016.
And the price/performance of network silicon from suppliers like Broadcom and Mellanox is outpacing that of general purpose CPUs, according to JR Rivers, CEO and cofounder of Cumulus Networks, a maker of network operating system software for bare metal switches. Also, bogging down the central CPU with networking features would sap its value. "When you start to put really beefy stuff in the middle of your CPU silicon, you diminish returns," Rivers says.
Optical interconnects and backplanes have also been evaluated before but never took off due to cost and complexity, Rivers says. Intel's RackScale architecture, which disaggregates and pools compute, networking and storage to make an IT rack more flexible and agile through software, proposes a photonic interconnect as a fabric weaving together those pooled resources.
But this may prove too complex to be practical, Rivers suggests.
"Optical backplanes are too complex, and that's why it hasn't played out," he says. "RackScale is too tightly coupled for today's data center environments and too much of a highly engineered system, as opposed to a loosely coupled system able to move at network scale and speeds. RackScale looks like a one type fits all, which inevitably it doesn't, and oftentimes customers don't get the benefit out of it."
He likens it to efforts to embed blade switches in blade servers, which users virtually ignored. Instead, they used pass thru modules to Cisco switch ports.
Sign up for CIO Asia eNewsletters.