From Radio to Porn, British Spies Track Web Users’ Online Identities

https://theintercept.com/2015/09/25/gchq-radio-porn-spies-track-web-users-online-identities/

by jakobdabo • 11 years ago

450 230 11 years ago

cryoshon2• 11 years ago

Yep. Why do you think they're tracking porn habits, if not for blackmail material?

There's no legitimate reason why a government needs to know anything about citizens sexual preferences or habits, but such information is perfect for shaming or threatening. I will note that the collection and analysis of this sort of information far surpasses anything envisioned in dystopian fiction or actually practiced by the likes of Stasi/KGB during the worst of times.

EDIT: there were actually a bunch of documents published with this article. I'll check those out then re-comment.

rm_-rf_slash6• 11 years ago

It can be useful for blackmailing prominent figures of adversarial foreign regimes, though to be fair Britain's spies will have to sift through their own Parliament first.

ionised3• 11 years ago

I've never been so ashamed to be British.

These last few years have really started to wear me down. The indiscriminate mass-surveillance, the savaging of our public services by an obviously corrupt government, the blatent hyprocrisy of our foreign policy, the repeated and unfettered fraud of the City of London financial sector that has yet to see anyone prosecuted or any institution meaningfully punished.

I really hate this place.

rm_-rf_slash6• 11 years ago

Given how much British politicians seem determined to make life shitty for anyone making less than six figures, you'd think they'd at least let people spend what idle time they have high on drugs instead of questioning a system that seems intent on shredding the middle class.

branchless4• 11 years ago

Just get out. However this ends it's not a happy ending.

The UK establishment are nuts.

MrZongle22• 11 years ago

The American establishment isn't much better.

voidz• 11 years ago

Get out? To where?!

Zigurd• 11 years ago

Every country in the Five Eyes is doing this, and many if not most who have the money to fund this kind of thing are doing it.

Move to Greece. They can't afford surveillance.

pakled_engineer• 11 years ago

CSEC (Canada) is up to the same shenanigans, approx zero politicians are talking about reigning in the surveillance state in any meaningful way during the election right now so it will keep on rolling. I often talk to people who have no idea who Snowden is, nor have they ever heard of CSEC.

buffoon2• 11 years ago

Perhaps I'm older than you but it's been the same in my mind since the 1980s at least. My father informs me earlier.

At least we have the joys of privatized services now (I'm serious). They are an order of magnitude better than the public railway, gas and electricity infrastructure we had. Controversially perhaps, I've had much better service from the NHS since they moved to a service model.

I'm not ashamed of my identity at all.

crdoconnor4• 11 years ago

They are an order of magnitude better than the public railway

Order of magnitude more expensive, maybe. Better? No.

dingaling• 11 years ago

Plus we ( the UK ) have now jumped onboard with remote drone assassination of UK citizens abroad. Even though there hasn't been a death penalty on the statute books since 1998.

SixSigma• 11 years ago

You should read more history. We have been awful for 100s of years. Just in the 20thC we destabilised the Middle East, exploded bombs in pubs in Eire to drum up discontent, jailed the innocent, protected the powerful, destroyed our own manufacturing in favour of financial services, exported asset stripping to the US.....

The list could go on.

Your shame is proportional to your knowledge, the level of reprehensible action is a constant.

notNow4• 11 years ago

  I've never been so ashamed to be British.

The "illustrious" colonial and expansionist history of the British empire wasn't enough to tip the scales for you before?

noir_lord2• 11 years ago

Why should we be ashamed of something that was done before we where born, should the Germans still be ashamed for world war two, should US citizens still be ashamed for what the US Army did, should the Italians be ashamed for what the Romans did, the French be ashamed for what Napoleon did?.

I accept our history, I'm not ashamed of it, it's history and I wasn't there.

However I am ashamed to be British for the things we currently do.

rorykoehler2• 11 years ago

Britain has a history of far worse things. I find the lack of awareness of Britains brutal past in the UK to be terribly disturbing. Morally it has one of the most repulsive histories of countries on earth yet all we hear from politicians is about British values as if they are some kind of goal to look up to. The superiority complex prevalent in British political discourse has no foundation yet is rolled out time and time again.

logn• 11 years ago

Selectors accept wildcards: https://firstlook.org/wp-uploads/sites/1/2015/09/mutant-brot...

I wonder what this query returns: _@_

or: _._._._

And the justification textbox has a default width that accommodates about 7 words.

At one point in the article it says the domain of a logged website is considered metadata but the path (full URL) is considered content. However, this screenshot shows a logged HTTP GET including its full path: https://firstlook.org/wp-uploads/sites/1/2015/09/cryptome.pn...

"When compared to surveillance rules in place in the U.S., GCHQ notes in one document that the U.K. has 'a light oversight regime.'" ... and UK is one of the US's Five Eyes partners, so the US has a light oversight regime, in effect.

At least the GCHQ shows some honest self-awareness in naming their surveillance apps. Karma Police. For a minute here, it seems like the intelligence agencies have lost themselves. http://genius.com/Radiohead-karma-police-lyrics ... I particularly like the last comment on the page analyzing the song, which begins, "This song is about the projection of guilt onto the Other", and concludes: "Prisoners, soldiers, soldiers' victims; these are all sacrifices, onto which we project our guilt, so that we can be saved."

rm_-rf_slash6• 11 years ago

I have two takeaways from this piece:

1: We should never assume any online activity isn't being tracked by someone.

2: The above doesn't give the government a pass on due process when citizens choose to avoid tracking by encryption or otherwise.

LouisSayers• 11 years ago

Except where they have a warrant and reach their dirty little fingers into certificate authorities. Unless you're doing key exchange yourself I would assume nothing is truly private.

M2Ys4U3• 11 years ago

They don't need warrants. They'll just hack in like they did with Belgacom.

And even if they don't, they won't get a warrant. They'll get a certificate from the relevant politician that'll authorise them to do it.

cynoclast• 11 years ago

"HERE WAS A SIMPLE AIM at the heart of the top-secret program: Record the website browsing habits of “every visible user on the Internet.”

Jesus fucking christ.

clort• 11 years ago

Heh, not trying to be pendantic because I actually thought that first sentence didn't make much sense.. there is a graphic T on the left, so it actually read "There was a simple aim"

junto• 11 years ago

Ladies and gentlemen. This is how you do blackmail, corporate espionage, and sway politicians on a truly industrial scale.

Want to push a bill through the House of Commons to curb GCHQ's powers? Think again Mr. Politician. Someone might just leak your sexual deviances to the newspaper.

rm_-rf_slash6• 11 years ago

I think we're due for a second sexual revolution, one where we once and for all settle our relationship between individual, society, and sex, where the only reason you should be concerned with someone else's consensual behavior is if:

1: You're banging them

2: You want to bang them

3: It's your kid and they're underage.

mkaziz• 11 years ago

I'd correct #3 to *they're underage.

herge• 11 years ago

Tell that to everyone who clicked on a tabloid headline about David Cameron and a pig's head.

Lawtonfogle3• 11 years ago

I'm finding humor in the fact that number 3 has an AND and not an OR in it.

leppr• 11 years ago

How about a political revolution, where legislative power would rest on every citizen in the country thus making the aforementioned tactic impossible (if such system ever came to exist I think we should call it a "Democracy").

petra• 11 years ago

I think this is far different from a sexual revolution. Through history, our notions of acceptable sexual behaviour changed quite dramatically, so the sexual revolution isn't that unique.

But all through history, human beings had some notion of a private space. The catholic confession booth is the only counter example i can think of, and even there it was just between you and you priest which was trained specifically for that role, and you had some control of what you did expose there.

And at least some psychologists ,when asked about this issue, believe that a right for privacy is critical for the psychological well being of people, not something we can give away without consequences.

Asbostos• 11 years ago

Isn't that just a statement of contemporary popular standards? They're not rules set in stone. The definitions of "underage" and "consent" are not nearly as clear cut as you might hope. Lots of people (in couples) have sex as a kind of trade for something they want. Is that consensual? What if it's a trade for something really really important to them? What if it's a trade to avoid being verbally abused by your partner? What if it's for money you need to survive?

Regarding age of consent, what if a mature 15yr old seduces a naive and vulnerable 21yr old and causes lasting emotional harm? How is that "none of my business" but the reverse isn't?

What if a 16 year old from England travels to America and has sex with a 20 year old? Is that OK (UK law) or not (US law)?

What you call a revolution sounds more like the way it already is for many people, and is full of arbitrary and changeable definitions that unfairly discriminate against harmless activity while also allowing harmful activity.

pdkl954• 11 years ago

The pr0n gets you blackmail material, but the social networks give you relationship maps.

Leaning on a politician is an obvious tactic, but it's somewhat heavy-handed. If the politician were to fight back, there could be expensive blowback. Worse, it's a single (or nearly single) point of failure. The bill could pass regardless of the actions of that one politician. A talented blackmailer should only want to directly lean on someone as a last resort.

There is much better tactic: find the people who may organize other people against your interests. If you take out the potential future leaders, journalists, and educators - aka, the people that will organize the writing of that bill - then you prevent the problem before it starts. When the FBI was busting up the hippies, this was known as COINTELPRO. It's laughable to think the program ended with the Church Committee - like "total information awareness", the program was simply split up into various existing programs.

A politician could be idealistic and fight back against blackmail. With social network information, a counter intelligence program can avoid this risk by leaning on the target's friends or family instead.

lotharbot• 11 years ago

It reminds me of the post about using metadata to find Paul Revere: http://kieranhealy.org/blog/archives/2013/06/09/using-metada...

Last-second blackmail isn't really the best way to defeat a political movement. Compromising the central figures in organizing the movement -- not necessarily "taking them out", but potentially just reducing their involvement or creating some alienation -- when the movement is still young is the way to really stop it.

a3n• 11 years ago

And thus Angela Merkel has been spied on since her earliest political days.

caseysoftware• 11 years ago

I was thinking of how to use it to de-cloak anonymous sources for journalists. Many of them are going to use anonymous email addresses, potentially VPNs/tor to further protect that but that's not the weak part.

Before you choose to share information, you're going to find out who will actually use it effectively which implies research. Then once you share the information, you're going to pay attention to what they write with it. In both cases, since you're not doing the actual leak then, you're unlikely to take any countermeasures, let alone effective ones.

For example, odds are Snowden researched Greenwald before he released anything to him. He probably read a ton of his pieces, looked at his wikipedia, and dug into his professional, etc profiles. And then after the leak, he probably continued to check Greenwald's regularly keeping an eye out for the new pieces. Even if we didn't know who Snowden was, a program like this would uncloak* him relatively quickly.

* Or at least narrow down the suspects.

blazespin• 11 years ago

You don't blackmail them, you proactively undermine their credibility by just leaking anything that might discredit them.

j-l-• 11 years ago

It is done because of data that it brings. Not because of deviances of House of Commons members. Moreover, in western countries like UK it is quite difficult to be classified as deviant.

bootload• 11 years ago

"Someone might just leak your sexual deviances to the newspaper."

Within the services, this is colloquially called the 'sluts & nuts' file.

rorykoehler2• 11 years ago

Who's really the weirdo. The guy watching pr0n or the guys watching the guy watching pr0n?

branchless4• 11 years ago

I love the way this article cites loop-holes. These guys do whatever they like. They have no mandate and they do it anyway with the blessing of politicians who then pretend to be concerned.

For me as a Brit the UK establishment is the biggest threat to people in the West. I really hope the UK has a big financial crash and the resulting chaos leads to a full regime change.

The UK really scares me a lot more than Isis.

JupiterMoon6• 11 years ago

The trouble is that any regime change could be worse and the full tools of oppression are already built...

noir_lord2• 11 years ago

Could be worse, could be a lot better.

We are in diminishing returns for how much worse our government could be without going outright "pick up that can" authoritarian.

The list of scandals, corruption, privacy invasions and out right shittiness just goes on and on.

To echo the GP, I worry alot more about what the politicians are up to than I do terrorists.

venomsnake• 11 years ago

But Snowden is the criminal.

So do the brits have all my porn records or only the porn I watched in the last 60 days? The article didn't make it clear.

mavdi• 11 years ago

With all honesty, if you're a programmer or a hacker working for GCHQ (I'm certain there are some of you around here) you should really bury your head in shame. No excuses, you are simply disgusting.

rdtsc• 11 years ago

A fat paycheck and job security can do wonders in solving those pesky cognitive dissonance issues.

pen2l2• 11 years ago

That's true. Truer than people would think. The funny thing is, I've seen the very people who were passionately against these things join these organizations, because they pay well. I don't entirely fault them -- gotta put food on the plate.

blibble3• 11 years ago

there's no fat pay cheques at GCHQ, they're well known for paying 2-3 times less than the private sector: http://www.gizmodo.co.uk/2015/05/gchqs-government-sancitoned...

it really makes you wonder what sort of people their roles appeal to...

cryoshon2• 11 years ago

And yet here in the US where there is similar, the NSA is viewed as a refuge for the "best and brightest" and "most patriotic". People nod approvingly when they learn someone works at the NSA, even in light of the disclosures.

Who has the courage to shame these people to their face for collaborating in our oppression? It's easy to wax poetic and spit fire on the internet, but it's much harder to actually shun someone standing in front of you.

bufordsharkley• 11 years ago

Really? In my circles, anyone working for Palantir (much less the NSA itself) is considered disgusting.

diyorgasms2• 11 years ago

I speak only for myself, but I recently met someone who works for the NSA, and I told him off like I've never told anyone off before. It was my goal to do my best to break him as a person as much as I can with words.

People who work for these agencies deserve no quarter, and they serve no useful place in a free and open society. I don't view myself as a 'mean' person, and I probably wasn't particularly effective when abusing this guy, but I like to think it's the thought that counts.

CardenB• 11 years ago

I don't personally know anyone who respects the NSA like you described.

davesque2• 11 years ago

I think that really depends on your circles. I can't think of anyone I know who trusts the NSA or looks at an employee of the NSA as a role model by default.

nailer2• 11 years ago

It's a pity they have a hiring booth at jQuery UK, which is otherwise a great conference.

3pt141592• 11 years ago

Really? Do you really expect intelligence organizations not to do this? This is what we pay them to do. International relations is anarchy. We shouldn't be blaming GCHQ we should be lauding Snowden while building surveillance hardened systems.

JosephRedfern• 11 years ago

"This is what we pay them to do". Who do you mean by we? Sure, we pay tax, but I don't remember ticking the "I'd like to sign up to pay for GCHQ to monitor people porn habits" box on my tax return.

If the government were building a device to blow up the moon, would saying "that's what we pay them to do" (due to us paying tax) be a valid excuse?

yarrel• 11 years ago

Really? You expect plumbers not to flood tower blocks and drown everyone in them? Plumbing is what they do!!1

mikosty• 11 years ago

Still somebody makes a decision to contribute to these systems. Most of those people think they do the right thing I believe. It is important to show that this kind of surveillance is clearly wrong and painfully shameful to be part of.

dreamfactory24• 11 years ago

Yes, I actually do expect them not to be the kind of morons who could decide it's smart to hoover, aggregate, and store a toxic mountain of data which massively undermines personal and state security.

There's a reason for external oversight and governance - you don't need malice in a large organisation to get a really shitty outcome, just a bunch of cogs in the machine each chasing local optima.

Afforess4• 11 years ago

I completely disagree. GCHQ, NSA, CIA, et al do serve a legitimate purpose. The world is not all roses and sunshine, there are rogue nation-states and rogue actors, and they do want to harm the public. It is the duty of the government to protect its citizens from attackers. These agencies exist to protect the public by identifying threats, and preventing attacks.

I know you are upset about the spying on private citizens for deceptive purposes, but lets not throw out the good with the bad. Yes, the agencies are overzealous and have overstepped their mandate, but that does not mean every single person working for them is an evil person out to get you. Real people work there, and nearly all of them probably have nothing to do with this. Reform, not harassing innocent bystanders, should be the aim here.

Silhouette4• 11 years ago

It's sad that some people are downvoting the parent post. It makes a valid point in a reasonable way.

There certainly are problems with the excessive government surveillance -- it is also the duty of a government in any civilised, democratic nation to protect its citizens from itself, after all.

However, it seems unconstructive to go all gung-ho against people who are probably mostly just trying to do some good from what is inevitably sometimes an awkward position. It's like arguing we'd be better without a police force because sometimes bad people manage to become police officers and sometimes their colleagues can't just flick a switch and shut them off, or arguing that we shouldn't invest in military forces because it gives the government the power to kill people from afar and in any case diplomacy is preferable to violence. Those things may all be true, but that doesn't mean we'd really be better off if the organisations with a degree of corruption or inadequate safeguards were eliminated rather than fixed.

mtgx• 11 years ago

Then why don't they focus on that alone? If you worked for an organization that saved 10 babies a month, but used 90 women for sex trade, would you continue to work there because "you're helping save babies", or would you be disgusted with that organization?

I know my example is a little extreme, but it's kind of what GCHQ and NSA do, too. They use their powers maybe 10% of the time to look for terrorists and whatnot, and then the other 90% to get data on everyone, including people in power that they can blackmail and influence. Don't think this is just a theory. There have been articles in the UK press about MI5 for instance covering up child abuse by politicians because they wanted to blackmail them with that. Enough said?

kjgo8y88• 11 years ago

"but that does not mean every single person working for them is an evil person out to get you."

I agree. Those engineers are just following orders. They bear no personal responsibility at all...

morgante2• 11 years ago

The mafia also does lots of good and helpful things in the community.

Does that means mobsters also shouldn't be reviled?

baconner3• 11 years ago

The question is - is reform even possible in organizations like these that have repeatedly broken the law with impunity? Is oversight possible at all when those who are supposed to have the power of oversight are unable to do so because these organizations are actively and intentionally hiding what they are doing?

When what they've done, effectively, is surveil everyone including the very people who are supposed to be able to exercise oversight and enact reforms how can oversight succeed?

I for one think the corruption is simply too deep to root out while keeping these orgs intact. They need to be torn down and rebuilt with clear missions consistent with our constitution (in the us), strict boundaries, and oversight. Reform is insurmountable when they still hold this wealth of illegally gathered surveillance data and retain their existing command structure.

The data must be destroyed. These orgs must be broken up and rebuilt so they are fit for purpose. Criminal, not overzealous is the word for what they've done. That in no way makes every person working there a criminal and I don't think demonizing the rank and file is productive or appropriate but we cannot keep watering down the severity of the problem for the sake of their feelings either.

pjc502• 11 years ago

The categorisation of "threat" is what we have a problem with. There was a scandal a while ago about the embedded deep cover agents in environmental groups: one had been there so long that he'd married and had a child with one of the people he was spying on. Further back there is the fairly terrible record of Special Branch et al in Northern Ireland.

Then there's the whole business of supplying intelligence to support the policy of invading Iraq, and the mysterious death of Dr. David Kelly. The Iraq invasion made us considerably less safe at the cost of about a million (non-coalition) lives.

Netcob• 11 years ago

What most people forget when they are sitting at home being scared about "rogue nation-states" from across the globe, or worry more about dying by the hand of people with dangerously fluffy beards instead of losing a foot to diabetes, is that complete security simply doesn't exist.

But what would we do if we wanted to provide total security? The naive and most popular approach is to give total control to the government. Not secure yet? More control then. Technology is the most promising way to achieve total control yet, and it's actually growing much faster than any global threat.

Anytime I read an article about some spying technology, I can imagine those developers looking at the existing infrastructure and internet technologies, thinking "we can totally do something to control this and that as well". That will actually be the "office culture" in places like the NSA. I know that's how I would think if I was sure my organization was serving a legitimate purpose. My colleagues would agree, and we'd high-five each other about redirecting yet another stream of information into our system. And whenever anyone has any doubts, group-dynamics or -think will take care of that. And if that's still not enough, maybe you need deeper philosophical help? https://boingboing.net/2015/08/11/the-failed-writer-who-beca...

With all those leaked documents, I've not even once heard about any limits. None. There's no "we'd like to know and control much more, but obviously we'll stop at that". If you gave the CIA the ultimate device that could provide them with all the information about every single human being on the planet and would also allow them to kill any individual instantly, they would immediately start using that. They don't want any oversight, they don't want anyone to control them, they don't want to have to respect anyone's privacy. They think they have a certain mandate or duty, and those things are by nature standing in their way of fulfilling it.

These developers absolutely should hang their heads in shame. Because every last one of them should understand that their organizations won't stop for ethical reasons, or deny themselves certain powers which should not be concentrated in the hands of a few people, no matter how well they think they mean. Being a "real person", not being out to get the average joe (but still spying on him), and generally meaning well - that doesn't make you any less guilty. At best it means you're a naive idiot who trusts authority too much, at worst you don't care and just want to play with your cool toys.

The world definitely isn't all roses and sunshine. Just because you think you're working for your country, making sure those who promise to protect it get all the help they could ever wish for and more, that doesn't mean the things you create will actually be used for "good". All it means is that someone further up now has more power than before, and they will use it the way everyone uses their power - to further their own interests. You simply hope your interests and theirs are the same and along the lines of "sticking it to those terrorists", but if history has shown anything, it's that power corrupts. I have zero doubt that those systems will be used against all of us eventually.

SapphireSun• 11 years ago

Here's the thing though. The reason there is even a debate about this is because the agencies serve a legitimate purpose that allows this to confuse our sense of morality. Without that rider, these methods would be roundly condemned in any other organization. A programmer working on unjustifiable programs should feel gross when they do something immoral.

Let's be clear here. This is the first time in the history of the world that nearly all ordinary communications by ordinary citizens, globally, are aggressively being monitored. The impact hasn't yet been fully felt by the global populace, but the power of these tools to rig the system and suppress dissent is very large and there is currently no effective check on these powers that is subject to public scrutiny.

cm21874• 11 years ago

Let's be honest. Secret services were always useful thugs. Using methods that would land them in jail in any other organisation. These guys make a career of spying, stealing, blackmailing, corrupting, sometimes killing. It's fair to argue that in certain circumstances they are useful and that a State should have the ability to commit these crimes. But let's not make the employees of these agencies anything else than what they are: hired thugs. And what I find worrying is that these crimes that were meant to be committed on an exceptional basis are now commited on a large scale against their own population.

gaius5• 11 years ago

GCHQ is a relic of the Cold War desperate to find a reason to continue existing.

notNow4• 11 years ago

  there are rogue nation-states and rogue actors

Funny you're saying this in the same sentence about those agencies and you're totally missing how ironic that is.

fit2rule8• 11 years ago

I concur. I once knew someone who worked for GCHQ, making hardware devices. Once it was revealed to me the extent to which he was involved in producing these heinous devices, I lost all respect for him - especially when his authoritarian side came out and he justified his continued involvement by implying that the lives of my children were at stake because "secret reasons I can't tell you about".

NO! THIS IS NOT HOW A FREE SOCIETY WORKS!

The corruption of government begins with its secrets. A truly free people keep no secrets.

If you continue to justify this corruption, you are encouraging the downfall of Western society. We did not attain the heights we have through secrecy and class warfare disguised as security theatre.

We must discourage involvement in these organizations to the same degree that we discourage teenagers from going on jihad. It is truly the same degree of bigotry and intolerance which allows such machinations to persist in our society - on the one hand, violent extremism. On the other hand: covert extremism.

My former GCHQ-supporting associate recently left, to start his own hardware company in an unrelated (non-surveillance-state) field. As much as I appreciate his design and skill at producing appealing devices (synthesizers), I strongly boycott his business. We must punish this "secret clearance class" of society with utter disdain, disrespect, shame and discouragement. We must not let future generations rise to assume that this totalitarian/authoritarian control system is the norm - it may be too late, but nevertheless, civil discouragement must be perpetuated in light of the total failure of our supposed democratic institutions to protect us from this covert violence.

MichaelGG3• 11 years ago

Meh, it's fairly inevitable. Systems are only getting bigger and more competent. This kind of stuff requires resources and some solid thinking about algorithms and data structures. No doubt they could publish some amazing papers. But I don't suspect they have revolutionary breakthroughs ala the NSA with crypto.

I've written a small-scale system for VoIP calls, to archive and index all network traffic - it's amazingly useful for debugging. Several years back, on a single quad-core machine with 1 disk, I was able to handle 5TB (several billion messages) of signalling data per day - indexing, archiving, searching. I'm nothing amazing. (I went to SF thinking I could somehow turn this indexing system into amazing profit. First guy I showed says "oh yeah I recognize this, go get <some intro db textbook>". Oops, so much for breakthrough ideas.)

A team of me-equivalents (with some that _have_ gone to school) and a nice budget could end up designing something like these spy systems. They aren't the Manhattan Project requiring new knowledge no one in the world has. I'm no 3-sigma intellect. There are dozens of millions of people more intelligent than me. Complaining about it and shaming engineers will not work against such critical projects.

objclxt• 11 years ago

But I don't suspect they have revolutionary breakthroughs ala the NSA with crypto.

I wouldn't be so sure - GCHQ recruits heavily from Cambridge and Oxford, and I once met a Mathematics PhD who told me in the UK his two main options for employment were academia or the service. They have a lot of crypto talent there and they're not working on the big data systems talked about here.

elchief• 11 years ago

But I don't suspect they have revolutionary breakthroughs ala the NSA with crypto

Besides inventing public key cryptography?

geographomics• 11 years ago

Privacy issues aside - after all, these programmes have scant interest in the average citizen - these look like fascinating projects to work on from a technical point of view, compared to most software development jobs. And it's surely a more ethical pursuit than, say, writing control software for missile targeting systems.

zenocon2• 11 years ago

Wouldn't it be fantastic if there were a coup in the CIA/NSA/GCHQ, etc. of all tech people who just started deleting data on a massive scale surreptitiously. When the dept. heads were like, "WTF happened to the numbers on all the porn viewers...they just kinda shrugged their shoulders and said...'dunno..guess people got bored with porn'"...

Replace porn with '*' and now we're really cooking...

AdmiralAsshat• 11 years ago

I wonder if Radiohead would be disgusted to find that their song title had been appropriated for this kind of alarming surveillance.

madaxe_again3• 11 years ago

They approve...

multinglets• 11 years ago

The solution is for everyone to mess with their signal-to-noise ratio. Make alts. Tell lies. Use VPNs. Encrypt trivial conversations. Use steganography for real secrets. If they like the job security, give them their damn job security by giving them nothing else.

Asparagirl• 11 years ago

Alternately, you can take the attitude that you will practice radical transparency in your online and offline preferences and habits, so there's nothing to potentially blackmail you with because you live your life unashamed.

I mean, personally my initial reaction when I first heard about the early Snowden leaks two years ago was "welp, hope the NSA enjoyed reading all that Tony Stark/Bruce Banner fanfic as much as I did."

JupiterMoon6• 11 years ago

If this the case for you then please walk down the street naked whilst reciting your banking password... I doubt you want to do that. Everyone has something to hide.

IkmoIkmo• 11 years ago

Absolutely disgusting.

crimsoneer2• 11 years ago

Just to clarify, these are all currently metadata, right? Eg, this stuff is, in theory, not linked to any individual people without a warrant/authority?

azakai• 11 years ago

They mention mass tracking of cookies from sources like google, yahoo, reddit, youtube, etc. - those might technically count as "metadata", but they are used to track individuals.

crimsoneer2• 11 years ago

I'd be fascinated as to how that's justified legally. Mass tracking has been accepted by the high courts, but the moment you start tracking individual users, you need some sort of warrant or RIPA authority.

JupiterMoon6• 11 years ago

No. Read the article.

rconti• 11 years ago

These agencies have all the incentive in the world to "encourage" popular websites (bbc.co.uk perhaps) to make their cookies particularly detailed and leaky.

acd• 11 years ago

This is also how you decloak users from VPN, first you profile their online behaviour without vpn, track cookies and graph site visits. Then when people use vpn you still see and track their identity.

heurist• 11 years ago

There's a huge economic incentive in having this data available - predict markets, sell or provide predictions to businesses, understand what your citizens are thinking at any point in time... Facebook, twitter, et. al. are all sitting on gold mines as well. Ignoring the privacy concerns (which are substantial) this would be an amazing set of data to work with. I'm not surprised there are people willing to do this.

An interesting article [1] was published in Nature a few days ago. I haven't read the entire paper yet but the abstract claims, "group-level ability to produce complex innovations is maximized when social information is easy to acquire and when individuals are organized into large and partially connected populations." Imagine the innovations and industries that would be created if all the information governments and corporations are sitting on were freely available.

I hope there will eventually be a broad 'free data' movement. If Google and Facebook are secretly pushing propoganda that privacy is dead (as stated in some conspiracy theory I recently saw on HN), and if these campaigns work, then the public will eventually not mind having their data open and available. But they should mind that it's all locked behind closed doors. Data is power! Do we want these massive corporations or governments to have so much power over the economy? Should they hinder social and technological progress by keeping their data unavailable? I have no answers but I think these problems will grow in time.

[1] http://www.nature.com/articles/ncomms9398.epdf?shared_access...

legel• 11 years ago

You bring up two very interesting and powerful phenomena: "siren servers" and "social physics".

The first term is used by Jaron Lanier to describe computer systems that, like "sirens" of the sea, lure everyone into their beautiful environs, collect all of the data on them, and statistically model all of this data in ways that effectively master the nature of this individual - her beliefs, her abilities, her emotions - until ultimately that individual is made both completely vulnerable and completely powerless. Lanier's book is intentionally provocative and controversial, while rather intelligent and compassionate in its spirit, and I recommend checking it out: http://www.amazon.com/Who-Owns-Future-Jaron-Lanier/dp/145165....

The second term comes from a talk at Google in 2014 called "Social Physics: How Good Ideas Spread" (https://www.youtube.com/watch?v=HMBl0ttu-Ow) by MIT Professor Sandy Pentland. He describes research supporting the recent Nature publication you link to on social and cultural progress, which concludes that our connectivity - free and fast flow of data on networks, people on roads, goods and services across nations, etc. - is our greatest economic and social virtue. His statistical modeling results really strongly show - "like a law" - that the more connected we are, the wealthier we are as a society.

It's really interesting to consider how we could concentrate on using all of the data and models to make deep positive impacts on our society, while respecting our human dignity. To quote Pope Francis's address to the U.N. today: "Integral human development and the full exercise of human dignity cannot be imposed. They must be built up and allowed to unfold for each individual, for every family, in communion with others, and in a right relationship with all those areas in which human social life develops – friends, communities, towns and cities, schools, businesses and unions, provinces, nations, etc."

nota_bene• 11 years ago

I'm done writing stuff online.

Good luck to y'all.

dotBen• 11 years ago

Does the fact that GCHQ have been able to scrape cookies from Hotmail, YouTube, Facebook, Reddit, WordPress, Amazon, CNN, BBC, Channel 4 indicate that https encryption has been broken?

In many cases of the above sites, authenticated requests (ie those with cookies) are made via https and so they would not have otherwise had access to the plaintext version to extract the identifying usernames/etc.

stordoff• 11 years ago

Do all of those sites only send cookies over HTTPS (or did they at the time the GCHQ document was written)? I know, for example, that the Amazon homepage is sent in the clear - is that sufficient to ID the user?

My gut reaction is that HTTPS isn't generally broken (though is probably susceptible to implementation flaws and targeted attacks to e.g. steal a priv. key) or we'd have seen more direct evidence in the leaked documents. That said, given the volume of documents that remain unpublished (or even unleaked), and the time that has passed since they were written, it's difficult to say for sure.

JupiterMoon6• 11 years ago

The trouble with https is that if you control the certificate authorities you can easily beat https. I don't see how the NSA doesn't control at least some certificate authorities and they work closely with GCHQ (even paying the UK money to fund some of these programs).

M2Ys4U3• 11 years ago

How many of those sites actually run over HTTPS? Reddit doesn't by default, neither do WordPress installs, CNN the BBC or Channel 4.

misiti37802• 11 years ago

"Black Hole contains data collected by GCHQ as part of bulk “unselected” surveillance, meaning it is not focused on particular “selected” targets and instead includes troves of data indiscriminately swept up about ordinary people’s online activities. Between August 2007 and March 2009, GCHQ documents say that Black Hole was used to store more than 1.1 trillion “events” — a term the agency uses to refer to metadata records — with about 10 billion new entries added every day. "

I wonder if this "black hole" is backed by a huge HDFS cluster? When I was working in government consulting a few years back I used to alway see job posting at Fort Meade for Hadoop experts, and I know the government is (or was) a customer of Cloudera as of 2009. Incidentally, Amazon created a separate cloud service like EC2 for government data a few years back also I believe.

I assume the NSA must be utilizing Hadoop, HDFS, Impala and/or Facebook Presto - or have a system they built internally but never released that can processing and store data at the same order of magnitude.

If these documents are all true and we assume they are currently collecting more, not less data, then they were in 2012, then they are probably storing more data than Facebook/google on a daily basis

EDIT - i guess the might not be storing as much data as Facebook or google because this sounds like mostly text, no images or video. It still must be a shitload of data though! I also realize that GCHQ is not the same thing as the NSA, but I assume they are doing similar things with similar size data sets

Thoughts?

nhf• 11 years ago

This presentation (GCHQ) specifically notes use of HDFS and Hadoop to process data: https://theintercept.com/document/2015/09/25/gchq-analytic-c...

misiti37802• 11 years ago

thank you!

FooNull• 11 years ago

Absolutely disgusting.

rl3• 11 years ago

Separate programs were built to keep tabs on “suspicious” Google searches and usage of Google Maps.

I always figured virtual geo-fencing of online maps usage was probably a thing.

plg• 11 years ago

So Tor?

Tails?

qiqing• 11 years ago

Can you imagine if the GCHQ had a breach and all the dirt on all everyone they spied on suddenly became public?

facetube3• 11 years ago

You won't have to imagine – if NSA and GCHQ continue on their current course, it'll eventually happen. It's only a matter of time.

davesque2• 11 years ago

Absolutely disgusting.

zenocon2• 11 years ago

It sucks that it has come to this. I am afraid to do certain types of searches. The other day, I wanted to learn more about "Azan" -- the Muslim morning call to prayer that is often broadcast over loudspeakers from a mosque. I find the topic interesting and started poking around, but it landed me on several websites that made me start to feel uncomfortable that I was going to incite some trigger. I'm betting most people in my neighborhood aren't searching for this information, and I don't really want to be flagged. Pre-Snowden, I wouldn't have given it a second thought and thought it ridiculously paranoid. Now, I am very careful how I use the computer when it is connected to the wider network...and I think this is really sad and Orwellian.

akshatpradhan• 11 years ago

This has to be the nail on the coffin. They're spying on our porn. Those are personal details about myself I would never share with anybody and wouldn't want anybody to know. Fine, keep an image of my penis, but knowing details about my porn usage?

36erhefg• 11 years ago

Well, most people share more with Google (search text) that with their closest friends. And most of them are just fine with that. Until of course they are in the spotlight. Since that almost never happens, we have this privacy apathy going on.

raspasov• 11 years ago

Slightly off topic, but recently watched Citizenfour - great documentary about Snowden and why he chose to reveal all of this information. It has real footage of Snowden himself right before the leaks, communicating via PGP with journalists, etc.

Even though I already knew most of the stuff, rehashing it all over again makes it so much more impactful. Overall, highly recommended for anyone who cares about our world now and in the future.

P.S. The government's idea that by somehow ingesting all of this data and effectively spying on everyone collectively will somehow protect us the helpless citizens sounds pretty bonkers (to use an UK phrase) to me. It's all just wasteful money spending as usual.

Stop the scaremongering.

MichaelMoser123• 11 years ago

When comparing GCHQ or NSA, who is the spookiest of them all? Which one of them is sucking in more information and who does more with that info ? my impression is that there is less institutional oversight over spooks in the UK than in the US, is that true ?

ionised3• 11 years ago

Snowden claimed that the UK's surveillance operations were a lot more severe than the US, basically due to a much weaker oversight framework.

The US might have a secret court with secret judges but all it takes for surveillance to be signed off on in the UK is the signature of a minister in the currently governing party.

caf2• 11 years ago

It would be nice if web browsers had an option to "only send cookies on HTTPS".

Also: "MEMORY HOLE"? Someone seriously approved that as a code name?

adeelx• 11 years ago

Already available, secure cookies: https://www.owasp.org/index.php/SecureFlag

AgentME• 11 years ago

That's a setting for webservers to set on cookies.

caf2• 11 years ago

I am aware of that, and it's not what I'm talking about. That has to be set by the site to be effective.

I'm suggesting a global setting in the browser that means it won't send any cookies to any plain HTTP site, regardless of what the site says.

If enough people enabled an option like this, sites would have to move to HTTPS if they wanted to reliably use cookies, which doesn't seem problematic.

zby• 11 years ago

This is just the beginning - the problem we are facing are the digital personal assistants. They are like servants - but their loyalty is first to the corporation that runs the server not to the user. This is where the really rich data will come from. And the data will be useful in all kinds of criminal investigations - there will be more and more legitimate cases and there will be no will to limit it.

https://medium.com/the-wtf-economy/we-ve-got-this-whole-unic... - see how useful the assistants can be - people will use them

cm21874• 11 years ago

The comparison with the Stasi is usually perceived as a Godwin point, but I think it is now a very reasonable comparison. I am amazed and honestly surprised how easily the secret services turned against their own population in the world's oldest democracy. It happens slowly and like the boiling frog we won't realise we are in a totalitarian regime until we're there.

rakoo• 11 years ago

The comparison with the Stasi is usually perceived as a Godwin point

I don't see how, the Stasi was Communist, not Nazi. Perhaps you're thinking about the Gestapo.

te_chris• 11 years ago

Worst part of all this? It's not a feature on any of the major british news outlets. Go look for yourself.

igravious• 11 years ago

Let's make a list of the UK[0] press and see who reports. Correct as of 6pm GMT Sept. 26th. The story is just breaking but only the Daily Mail and Daily Mirror have reported it so far. I'll probably be on a nice list now having made all those searches : ) Advice on how to search The Sun's website welcome.

  - The Sun[1], can't find a search box
  - Daily Mail[2], yes!
  - Daily Mirror[3], yes!
  - Evening Standard[4], no
  - Daily Telegraph[5], no
  - Daily Express[6], no
  - Daily Star[7], no
  - The Times[8], no
  - i / The Independent[9], no
  - Financial Times[10], no
  - Daily Record[11], no
  - The Guardian[12], no

[0] https://en.wikipedia.org/wiki/List_of_newspapers_in_the_Unit...

[1] uh [2] http://www.dailymail.co.uk/home/search.html?sel=site&searchP... [3] http://www.mirror.co.uk/search/simple.do?destinationSectionI... [4] http://www.standard.co.uk/search/site/gchq [5] http://www.telegraph.co.uk/search/?queryText=gchq&sort=recen... [6] http://www.express.co.uk/search/gchq [7] http://www.dailystar.co.uk/search/gchq [8] http://www.thetimes.co.uk/tto/public/sitesearch.do?querystri... [9] http://www.independent.co.uk/search/site/gchq [10] http://search.ft.com/search?queryText=gchq [11] http://www.dailyrecord.co.uk/search/simple.do?destinationSec... [12] http://www.theguardian.com/uk/gchq

robk• 11 years ago

Seems like I should turn On open access for my wifi router to pollute these results by running am open hotspot.

machiaweliczny• 11 years ago

I have one question. Any idea how they can extract cookie data from encrypted connections?

nly• 11 years ago

Well sites like YouPorn, RedTube and PornTube etc, despite being ridiculously popular, haven't deployed any kind of TLS. They don't care about your privacy. It's low hanging fruit... far too tempting for the likes of GCHQ, with their capabilities, to just ignore.

If you don't want people spying on your porn habits then boycott sites like this. And if you're running a porn site, consider going HTTPS only, turning off access_logs (or purging them regularly), and providing a Tor hidden service.

gopowerranger• 11 years ago

People jumped down my throat, here on HN, whenever I brought up this very fact over the last couple of years so I find it interesting to see it reach the top. I guess we have to rotate targets more slowly over time.

jjuhl• 11 years ago

Sickening.

From Radio to Porn, British Spies Track Web Users’ Online Identities

Related Stories