Leads from the NSA metadata collection program are handed to the FBI, which then has to follow through and investigate them at additional cost. Public reports and disclosures in the media and by government officials suggest that many of the leads supplied by the NSA end up going nowhere. In fact, only 0.2% of the leads typically pan out enough for the FBI to follow through, the authors wrote.
The NSA's intense focus on "connecting the dots" over the past few years has led it to collect and analyze a vast amount of mostly useless information, according to Mueller and Stewart. Bits of data that ordinarily wouldn't have merited a second glance previously, are now collected, filed and analyzed at potentially great cost.
"There is very little evidence that any of this has done much good," Mueller said Friday. "The costs certainly outweigh the benefits, especially if you include things like privacy costs and opportunity costs," he said.
According to Mueller, even if the metadata program was to result in the NSA thwarting just one major terrorist attack every four years, it would still not be cost effective when all costs are accounted for.
Noted security researcher Bruce Schneier said the analysis of the value of the metadata program appears solid. "I think they have great framework for analysis, and — given the information we have — have demonstrated that [the data collection program] doesn't have value," he said
"Of course, it's possible that the NSA has secret information that proves that it does have value. But given that the NSA has been pretty desperate to show that they're actually keeping us secure, it's pretty safe to conclude that if they did have evidence they would have presented it," he said.
Sign up for CIO Asia eNewsletters.