"What is now in a chip -- a single piece of silicon -- is the size of the systems we were trying to schedule 15 years ago," Kramer said. "We've assumed the operating system or the programmer will handle all that scheduling we were doing."
The old supercomputing metrics of throughput seem to be fraying. This year, Jack Dongarra, one of the creators of the Linpack benchmark used to compare computers on the SC500 list, called for additional metrics to better gauge a supercomputer's effectiveness.
Judging a system's true efficiency can be tricky, though.
"You want to measure the amount of work going through the system over a period of time," and not just a simplistic measure of how much each node is being utilized, Kramer said.
He noted that an organization can measure the utilization of a system by measuring the percentage of time each node is utilized. But this approach can be misleading in that a workload can be slowed to increase the utilization rate, but as a result, less work is going through the system overall.
John Hengeveld, Intel's director of HPC marketing, suggested the supercomputing community take a tip from manufacturers of airplane jet engines.
"At Rolls-Royce, you don't buy a jet engine any longer, you buy hours of propulsion in the air. They ensure you get that number of hours of propulsion for the amount of money you pay. Maybe that is the way we should be doing things now," Hengeveld said. "We shouldn't be buying chips, we should buy results."
Sign up for CIO Asia eNewsletters.