It started off as an experiment, but Microsoft now wants to speed up and return more accurate Bing search results with the help of reconfigurable chips called FPGAs (field-programmable gate arrays) in data centers.
Next year, Microsoft will start installing FPGAs in its data centers, hoping to speed up and increase the accuracy and relevancy of search results.
"Increasing capabilities and reducing the cost of Microsoft cloud services relies on increasingly efficient hardware," said Andrew Putnam, a researcher at Microsoft, during the Hot Chips conference in Cupertino, California, on Wednesday.
The FPGAs are intended to help Microsoft deliver text and image search results much faster. Their use will also reduce the number of servers for processing search requests in Microsoft's data centers.
The FPGAs will be used in specially designed servers and will be connected through a specialized cable called Catapult.
"Hardware specialization is ... possibly the way to gain efficiency and performance we need to improve data center services. The Catapult reconfigurable fabric offers a flexible, elastic pool of resources where we can build these kind of accelerators," Putnam said.
Catapult started off as the name of an experiment in 2011 to put FPGAs in servers, and as a way to circumvent traditional data center designs. Microsoft was playing with the idea of specialized hardware to tackle processing for Bing, which would be much faster on FPGAs than through traditional CPUs.
"Technology has not been a friend ... in the last handful of years. It becomes more and more difficult to get those kind of efficiency gains and cost reductions by just waiting for technology to get better," Putnam said.
An experimental deployment of FPGAs in 1,632 servers in Microsoft's data center yielded promising results.
"The FPGAs provided two times improvement in throughput and 29 percent reduction in network latency. Half the servers were required to do all the processing," Putnam said. "If we need to improve the efficiency of the architecture, we can do it via hardware specialization."
But the decision to deploy FPGAs took a while. Microsoft initially looked at FPGAs and ASICs (application-specific integrated circuits). ASICs would be better on the power-efficiency scale than FPGAs, but they would be less flexible regarding software. But FPGAs, with their ability to be quickly reprogrammed, can be repurposed much more easily.
"As soon as the software changes, that FPGA accelerator can be updated along with the application change. But the ASIC is going to be a waste of power, or going to hold software development back," Putnam said.
The FPGAs run code to determine query results, which are then sent back to the CPU for delivery to the Web. The faster computing through FPGAs allows for more services to be separated, differentiated and applied.
Sign up for CIO Asia eNewsletters.