Subscribe / Unsubscribe Enewsletters | Login | Register

Pencil Banner

AMD accuses BAPCo and Intel of cheating with Sysmark benchmarks

Gordon Mah Ung | Jan. 18, 2016
AMD comes out swinging in a video released Thursday.

AMD's FX chip logo
AMD's FX chip logo Credit: Advanced Micro Devices

AMD threw out a bombshell and accused its rival Intel and BAPCo, the benchmarking consortium, of cheating. 

In a video posted Thursday on Youtube, John Hampton, director of AMD’s client computing products, went so far as to refer obliquely to the recent Volkswagen scandal, where the German car manufacturer was accused of cheating on diesel emissions tests. “The recent debacle over a major auto maker provide the perfect illustration as to why the information provided by even the most established organizations can be misleading,” Hampton said.

Intel declined to comment on AMD’s accusation, and we’ve yet to hear a response from BAPCo.

Why this matters: Performance still matters to consumers and organizations. Third-party benchmarks hold heavy sway over purchasing decisions even if few understand what they measure. AMD asks reasonable questions, but the answers remain murky—even from AMD.

amd fx8150 chip 
Credit: Robert Cardin

AMD makes its case

Hampton laid out AMD's case in the video. “So truth or myth: is Sysmark a reliable, objective, unbiased benchmark to use in evaluating system performance?” Hampton asked. Hampton and AMD engineering manager Tony Salinas then ran two “similar” laptops running Sysmark 2014. The Core i5 laptop scored about 987, while the AMD FX laptop scored 659.

Salinas then ran the same laptop in Futuremark’s PCMark 8 Work Accelerated workload. While the AMD FX laptop is slower, it’s only about 7 percent slower.

One final test Salinas ran was an unidentified benchmark using Microsoft Office. The Core i5 finished in 61 seconds, while the FX chip finished in 64 seconds.

“What we concluded is that Sysmark does not use realistic every day workloads,” Hampton said. He encouraged viewers to read the FTC’s fine print, which dictated what Intel had to disclose on benchmarks.

The FTC ruling in 2010 bound Intel to say: “Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such as Sysmark and MobileMark, are measured using specific computer systems, components, software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated purchase, including the performance of that product when combined with other product.”

A longstanding feud

AMD’s problems actually go all the way back to 2000, when the company’s Athlon XP CPU was kicking Pentium 4 butt in Sysmark 2001. When Sysmark 2002 was released, however, the Pentium 4 was suddenly the leader. After that AMD decided to join BAPCo in an attempt to have more influence over what it tested.

 

1  2  3  Next Page 

Sign up for CIO Asia eNewsletters.