A post on Washington's Blog summarizes numerous critiques of Big Data, with reference to its efficacy in finding signs of terrorism amidst gigantic databases of surveillance information. Everyone from Nassim "Black Swan" Taleb to NSA whistleblower William Binney describe how massive surveillance programs and data-mining lead to systems of unaccountable, automated suspicion:
Because of excess data as compared to real signals, someone looking at history from the vantage point of a library will necessarily find many more spurious relationships than one who sees matters in the making; he will be duped by more epiphenomena. Even experiments can be marred with bias, especially when researchers hide failed attempts or formulate a hypothesis after the results — thus fitting the hypothesis to the experiment (though the bias is smaller there).This is the tragedy of big data: The more variables, the more correlations that can show significance. Falsity also grows faster than information; it is nonlinear (convex) with respect to data (this convexity in fact resembles that of a financial option payoff). Noise is antifragile. Source: N.N. Taleb
The Dirty Little Secret About Mass Surveillance: It Doesn’t Keep Us Safe
(via Reddit)
