The UK has been using massive datasets to spy on innocent civilians for years
For over a decade, UK intelligence agencies GCHQ, MI5 and MI6 have been collecting vast amounts of personal information on UK citizens not suspected of committing any crime – including financial data, travel records, passport information, medical documentation and telephone directories.
These so-called bulk personal datasets, as defined by the agencies, consist of "personal data about a wide range of individuals, the majority of whom are not of direct intelligence interest." Additionally, they are "too large to be manually processed" and are stored on easily-accessible and searchable computer systems.
While the idea of these 'datasets' being collected has been long-discussed, especially following the Edward Snowden revelations in 2013, their existence was only officially confirmed by Prime Minister David Cameron in March last year.
Now, a fresh trove of disclosures from campaign group Privacy International has unleashed hundreds of pages of previously confidential documents, policy papers and codes of practice. The group aims to shine a light on the secretive nature of domestic spying and bring 21st century surveillance practices out of the shadows and into the public domain.
The spy papers outline, in vivid detail, how the agencies work together to share the information, how data from innocent UK civilians is shared to foreign governments and even reveals cases of misuse, saying that 'spooks' used the massive database for mundane purposes.
According to the papers, the real worth of the data is in how it is collated. Meanwhile, the documents indicate the process of collection and interception had been signed off by "successive Foreign Secretaries" that dated back to at least 2001, under Section 94 of the Telecommunications Act 1984.
Reuters/Peter Nicholls
In the case of GCHQ, the spy papers reveal any data ingested is merged with intercepted communications data – or phone calls, text messages and social media information – to "identify and develop intelligence targets." While 'metadata' from a single source is fairly innocuous, when mixed with the reams of other personal data, a clear picture of an individual and their actions quickly becomes clear.
One of the more contentious datasets the agencies collect contains financial data – the papers show the spooks were well aware that if its collection was made public the response would "most likely be unfavorable."
"The fact that [MI5] holds bulk financial, albeit anonymized data is assessed to be a high corporate risk since there is no public expectation that the service will hold or have access to this data in bulk," notes one document. "Were it to become widely known that the service held this data, the media response would most likely be unfavorable and probably inaccurate."
And while the agencies refuse to disclose the exact nature of the bulk dataset contents for national security reasons, the disclosures include a number of 'acquisition forms' that GCHQ and MI5 can use to request data from a source.
The criteria included on the form contains a tick box that lists data including: "biometric, financial, medical, racial or ethnic origin, religious, journalistic, political, legal, sexual or criminal activity." While crucial detail is redacted, the papers also reveal bulk personal data can be collected via 'third party' or via interception.
Furthermore, any data scooped up as part of a bulk personal dataset is routinely shared with foreign partners - meaning those signed up to the Five Eye's Agreement which, alongside the UK, consists of Australia, Canada, New Zealand and the United States. As a result they also get access to citizen data to help fight crime.
A powerful tool
Reuters/Peter Nicholls
"The database is a powerful data exploitation tool [...] access to data on the database will involve an interference with privacy," admits one MI6 document from 2011. Additionally the paper warns that agents using the system must not abuse the system as a 'free for all'.
Yet, as the memo reveals, this is not always adhered to. "We've seen a few instances recently of individual users crossing the line with their database use," one unnamed official wrote in the memorandum. This misuse included: looking up addresses in order to send birthday cards, checking passport details to organize personal travel and looking up the details of family members.
It continued: "Please remember that every search has the potential to invade the privacy of individuals, including the privacy of individuals who are not the main subject of your search, so please make sure you always have a business need to conduct that search and that the search is proportionate to the level of intrusion involved."
Following increased scrutiny on privacy in a post-Snowden world, the spy papers note: "In October 2014 a new code of practice was introduced which expressly prohibited 'self-searching'. However, upon investigation, problems remain. One recent legal document stated that between June 2014 and February this year there was "47 instances of non-compliance" within MI5 alone.
For their part, the agencies maintain the data collected is of "significant intelligence and security value." The papers also list a fairly rigorous oversight regime that GCHQ, MI5 and MI6 are expected to adhere to – even if human nature sometimes gets in the way of such regulation.
The Home Office told IBTimes UK that bulk powers have "been essential to the security and intelligence agencies over the last decade and will be increasingly important in the future."
A spokesperson said: "Terrorists and criminals have embraced modern communication networks to plan, coordinate and increasingly to execute their attacks, reducing the ability of conventional and targeted intelligence approaches alone to tackle these challenges. The acquisition and use of bulk provides vital and unique intelligence that the security and intelligence agencies cannot obtain by any other means. The security and intelligence agencies use the same techniques that modern businesses increasingly rely on to analyze data in order to overcome the most significant national security challenges."
Meanwhile, Millie Graham Wood, legal officer at Privacy International, said the disclosure shows the "staggering extent" to which UK intelligence agencies collect data on innocent people – from what petitions you may have signed to your personal correspondence messages to your lawyers.
"This data is integrated into databases that could be used to build detailed profiles about all of us," Wood continued. "The agencies themselves admit that the majority of data collected relates to individuals who are not a threat to national security or suspected of a crime. These documents reveal a lack of openness and transparency with the public about these staggering powers and a failure to subject them to effective parliamentary scrutiny."
Such scrutiny came to light while the Investigatory Powers Bill was passing through the UK political system this year. The bill – which has been branded a Snoopers' Charter by critics – aims to further enhance both the surveillance capabilities and oversight of the current spying regime. Despite objections from political figures, technology firms and activist groups, it is expected to pass into law by the end of 2016.
NOW WATCH: Here’s why airlines ask you to raise the window shades for takeoffs and landings