That bring us to the really contentious thing about this test - on numbers alone FireEye really didn't do that badly, detecting 95 percent of web malware, 96 percent of email malware and 93 percent of exploits, giving an overall detection rate of 94.5 percent and a zero percent false positive rate. Although this is below the roughly 98-99 percent scores achieved by most of its rivals, the real problem NSS Labs found with the FireEye systems was their cost-performance.
This plots the total cost per Mbps protected against security effectiveness, which in the case of FireEye left its product with a figure of $427.85 (£280) against the highest-rated Sourcefire Advanced Malware protection costed at $231.86. In NSS's assessment at least, Sourcefire simply offers more protection for every dollar spent than does FireEye.
Regardless of the arguments on either side of this judgement, it is clear that breach protection security comes at a premium more or less which company is looked at; these are all expensive systems and measuring value for money and effectiveness will remain a black art shrouded in technical complexity. It is also the case that working how good they are at living up to the claims made in the sales brochures is not going to be as easy in 2014 for any firm as it was a year or two ago.
Sign up for CIO Asia eNewsletters.