“We have a lot of respect for him,” Pool said. “I think he’s wrong in this case.”
Torvalds argues against the value of small code loops in measuring performance, but Poole said the future is mostly about smaller loops. Poole said moving a window around a screen or opening a window is mostly a solved problem for CPUs.
Poole said they’ve been very transparent with what the test measures and have provided extensive documentation as well. In order to measure the chip performance, Geek Bench tries to execute the same code on every platform, Poole said.
Poole claims the question of whether the A9X is faster than, say a Core m3, is beside the point. The software just isn’t there on the iPad Pro today that can do what you can on a laptop, rendering the iPad Pro mostly a curiosity until software changes that.
Still Poole said, he does wonder how the A9X would run OS X. I do too, as it would make comparisons far easier to measure.
To be fair to Geek Bench, benchmarks developers are often accused of serving internal political needs. BAPCo, the software and hardware consortium, has its roots in the PC industry and some feel it’s too PC-centric. Even among PC vendors, there was strife when AMD quit the group and accused it of being too closely aligned with Intel on its once-popular SYSMark.
BAPCo officials have always said their intentions are to create benchmarks that offer insight to the public. If the test is cooked to favor Intel or x86, it certainly didn’t show when the new version was released earlier this year and Intel’s older Atom chips were kicked down the stairs by Apple’s and Nvidia’s CPUs.
I spoke with BAPCo’s John Peterson about the philosophical differences between the two.
”TabletMark utilizes a wide variety of APIs provided by each platform to represent productivity/media app performance and battery life. The workloads are implemented in a way that’s meant to reflect the implementation choices app developers would make for each of the platforms,” Peterson said. “GeekBench generally utilizes its own libraries to perform tasks, while TabletMark utilizes the available platform APIs to perform equivalent functionality in a more platform-tailored way."
Sign up for CIO Asia eNewsletters.