Whether you love it or hate it, the technology behind it all is here to stay
Poor Edward Snowden. The former NSA subcontractor has sacrificed his career to expose US government surveillance programs that were revealed years ago. Except for minor details, data-mining operations like “PRISM” were outed in 2006, and have been underway since at least 2003. Newspapers may be dinosaurs, but they beat the Internet to this story by seven years.
Outside Washington, D.C., perhaps the least surprised people were in Silicon Valley, where companies design much of the technology that enables data mining on this vast scale. In 2006, a whistle-blower revealed a secret room in a San Francisco AT&T central office where mysterious equipment had its own intravenous connection to the network. It had existed for three years. Also in 2006, USA Today detailed the data-mining operation we know today as Prism.
That same year, I wrote about a prototype processor with 4,096 cores. It was designed for commercial video processing, but an early adopter was In-Q-Tel, a government-sponsored venture capital fund that promotes research and development for the CIA. Nobody would talk about the CIA’s application, but I can make some good guesses.
Keep in mind that intelligence agencies needn’t collect and store the data they search. Private sector companies already do that. Those companies sell the data to each other and mine it for money. The government mines it for security threats. Both types of miners get lots of help from specialized processors designed for networking.
We absolutely need these processors to scan Internet traffic for malware and to speed network packets to their destinations. That these chips can also be used for mass surveillance is either a bonus or a menace, depending on your viewpoint. But the technology is here to stay, because big data has become a profitable commodity for the private sector and an irresistible resource for cops and spooks.