Brown said the way to eliminate as much bias as possible from an evaluation is to define the problem, identify all possible hypotheses, collect information and evaluate the hypotheses. In the evaluation process, he said it is important to take a contrarian, devils advocate view of each hypothesis even those seen as most likely to be correct check all assumptions and their origins and also consult peers and outside experts, who might notice things you, or your team, are missing. Finally, he recommended choosing the hypothesis that has, the least evidence against it, to keep conclusions tentative while continuing to collect data and forming alternate hypotheses and to consider your organizations goals and customers along its costs in time and personnel. And, if you want your conclusions to have some credibility, show your methodology, he said, noting that too many reports, on everything from the APTs to DDoS attacks and more, issue sensational conclusions, with no methodology shown on how they reached them.
Sign up for CIO Asia eNewsletters.