He plans to maintain the Linpack test alongside HPCG in part for the valuable trending information that Linpack provides, he said. But it will also continue to be used because it could take years before a significant number of supercomputers are tested against the new benchmark.
"I expect in November we'll just have a few entries based on this new benchmark. Populating the list with 500 entries is going to take some time, so I'd guess over the next five years we'd have a chance of seeing that list fully populated," he said.
Starting in November, "we're going to have a list of the Top500, and then we're going to have a second column, and that second column will be the new benchmark," Dongarra said.
"It may ultimately lead to a list that is based on this new benchmark, but certainly not right away," he said.
The dueling benchmarks could potentially lead to different supercomputing centers, which covet positions on the Top500, claiming leadership based on both the old and the new tests. That could make it hard to say definitively who has the fastest supercomputer, though it seems the Top500 will consider Linpack to be the primary ranking metric at least for now.
The new test could lead to some "big changes" in which systems show the greatest performance potential, Dongarra said. The HPCG benchmark stresses architectural features which might not be easy for systems that perform well on the Linpack test to optimize for, he said.
"I think individuals will have to then evaluate what number makes sense for their particular mix of problems. And over time I would hope that the new [benchmark] would carry more weight."
HPCG was developed partly at the behest of the U.S. Department of Energy, Dongarra said. "They're looking towards exascale now, and the concern is that if you build an exascale computer that will do this Linpack test well, it may not do well at other problems. So that's one of the issues here."
The University of Tennessee conducts joint projects with the DOE and Dongarra said he's familiar with their application requirements. But he said the new test will be a good indicator of how computers will run other types of applications as well, such as those used for oil and gas exploration or weather modeling.
"One of the problems with the Linpack is that is stresses only one component, that being the floating point potential of the computer," he said. It doesn't stress areas like system latency and memory hierarchy, and the new test will be able to expose weaknesses of systems as they relate to those areas.
Dongarra plans to distribute the software for the new test to computer vendors in the next few months, giving them a chance to begin optimizing their systems and to propose changes to HPCG before it's introduced formally at the SC13 supercomputing conference in Denver this November, where the next Top500 list will be announced.
Sign up for CIO Asia eNewsletters.