New leak shows NSA harvests To, From, and Bcc lines of e-mail data

http://arstechnica.com/tech-policy/2013/06/new-leak-shows-nsa-harvests-to-from-and-bcc-lines-of-e-mail-data/

by evo_9 • 13 years ago

403 106 13 years ago

arstechnica.com

VladRussian2• 13 years ago

yep, and due to a bug in the perl script, it harvests all the lines to the next From.

pfortuny• 13 years ago

Yes, it is probably a regexp bug, like

    /^ $/

Someone pressed the space key once at the wrong place...

webwanderings4• 13 years ago

How the heck do you track Ad interacting habits through just an IP address? I call BS on that particular paragraph.

schmidp• 13 years ago

They could track if you reply to an ad or, if they also track your IP connections, see if you click on a link in an email containing an ad.

webwanderings4• 13 years ago

The IP information is a common and general location, usually of your nearest Telecom tower. IP address in itself does not lead to your Internet device. So the paragraph is still inaccurate if the Ad agency is tracking through IP. Your IP was always naked and available to anyone you send email to (through the headers).

dreamfactory• 13 years ago

via ad agencies?

webwanderings4• 13 years ago

And what does the Ad agencies got to do with this NSA mess? Or in other words, is there any disclosure in this article of any Ad agency doing NSA's job?

SoftwareMaven2• 13 years ago

It's a stretch, admittedly, but most email marketing interactions have campaign-specific email addresses to follow conversions. They would certainly be able to see what companies you've previously interacted with by the fact that you are receiving emails from company.com.

oceanstone2• 13 years ago

NSA harvests the entire email.

Source: http://www.wired.com/threatlevel/2007/05/mark_klein_docu/

chakalakasp• 13 years ago

If I'm not mistaken, isn't fiber optic splitting done with a glass... Prism?

mpyne5• 13 years ago

Careful, you're going down the Glenn Beck-style "OLIGARHY" route.

leecbaker• 13 years ago

I talked to a company that did this kind of work, and they used tiny MEMS mirrors made of gold plated silicon.

They wouldn't tell me much about their customers, but they specialized in switching and splitting high speed data at the physical layer.

http://www.glimmerglass.com/solutions/cyber-security-and-law...

mortehu2• 13 years ago

It's done by bending the fiber, causing leaks. The signal has a single wavelength, so a prism would serve no purpose.

leoc• 13 years ago

What we've seen of the slides seems to indicate that the PRISM name refers specifically to the "direct collection" from cloud-data firms (using FAA70* directives), as distinct from the "upstream collection" from the network. https://news.ycombinator.com/item?id=5887627

fnordfnordfnord• 13 years ago

It's probably an optic with a coating that reflects some percentage (probably a small percentage) of the light, and a second detector.

http://www.edmundoptics.com/optics/beamsplitters/

tacoman• 13 years ago

We use these extensively:

http://vssmonitoring.com/products/traditional_tap.asp

I'm not sure how they work, but the sales engineer said "mirrors". The fibre units are completely passive.

guelo2• 13 years ago

That's obvious since the email header and body are in the same document. Presumably what's going on is they capture all emails, process them to strip off the body, and then make the headers available to the analysis systems. They can then go back and retrieve the body of any email that interests them. One of the points that critics have been making is that analysts can make the decision to retrieve the content on their own without a court order, so this whole metadata safeguard isn't much of a protection at all.

barredo2• 13 years ago

Is there any numbers on the amount of hard drives NSA have?

joe_the_user• 13 years ago

"How Much Is A Zettabyte?" http://foxnewsinsider.com/2013/06/07/how-much-zettabyte-nsa-...

barredo2• 13 years ago

The center will reportedly be able to store five zettabytes worth of information

I am sure some day in the future there will be MicroSD cards with this storage capacity. But now it is just mindblowing

~5.500.000.000.000 Gigabytes

zerohp• 13 years ago

It's been a while since I worked with email headers and smtp, but I don't think the Bcc header actually exists in transit. The mail user agent and/or the mail submission agent remove it.

They could reconstruct this information from the graph.

oceanstone2• 13 years ago

Considering they have an upstream wiretap on all US internet traffic, it seems possible they could determine the BCC.

XorNot2• 13 years ago

Why bother? Google knows the BCC line - your email provider by definition has to in order to BCC the message, and it very much depends on them to properly scrub it out as well (I know for a while it was pretty trivial to see BCC's from most providers).

jlgreco3• 13 years ago

Apparently not always:

"In the second case, recipients specified in the "To:" and "CC:" lines each are sent a copy of the message with the "BCC:" line removed as above, but the recipients on the "BCC:" line get a separate copy of the message containing a "BCC:" line. (When there are multiple recipient addresses in the "BCC:" field, some implementations actually send a separate copy of the message to each recipient with a "BCC:" containing only the address of that particular recipient.)"

http://en.wikipedia.org/wiki/Blind_carbon_copy#Visibility

Depending on combination and location of the MUA and MSA, it is plausible that the NSA was able to get full BCC lines.

tracker1• 13 years ago

It depends on if you get the envelop. The envelop is a list of recipients for a given server to handle... the headers of the actual email are a convenience for display, not used for relay/forward/delivery.

fleitz• 13 years ago

The BCC field itself is usually removed from the email but if you're monitoring the STMP session you can reconstruct the BCC from the RCPT TO commands in SMTP.

ape4• 13 years ago

Yes if somebody is in the RCPT TO but not the To: or other fields then they are Bcc:

throwaway10001• 13 years ago

So who the F can we trust? All those denials from everyone and now we see this, which I kinda suspected since Verizon was ordered to hand over the same for phone calls.

nathas2• 13 years ago

You can't trust anyone. You can trust protocols and implementations and proven math.

That said, I would bet good money they already have working quantum computers, in which case current crypto may have quite a few problems.

philh• 13 years ago

I would bet good money they already have working quantum computers

How much, at what odds, and under what conditions of settling?

nostrademons• 13 years ago

That's pretty restrictive. I know a large number of scientists and computer programmers who believe the only thing you can trust are protocols and math, and they're usually completely unable to function outside of the narrow domain of their work.

I think a better philosophy is to trust that people will behave according to the incentives and information available to them. So if there is an organization out there, you can bet that it will act to expand the scope of the organizations' actions, because organizations that don't do this eventually get replaced by ones that do. If the organization is tasked with keeping tabs on all of America's adversaries, you can bet that they will see adversaries wherever possible to preserve a purpose for the organization.

bithive123• 13 years ago

It may be true that you can't trust anyone in the abstract but practically speaking we need to be able to trust our government, to a point, with all sorts of powers and abilities.

Consensus mistrust of the government should worry us more than any particular capabilities it has.

diminoten• 13 years ago

Good money huh?

Let's do this. 10btc. Name your terms.

bad_user• 13 years ago

I would bet good money they already have working quantum computers, in which case current crypto may have quite a few problems.

Both statements in that sentence are ridiculous. Do you also wear a tin foil hat while having such thoughts?

First, quantum computing is one of those fields for which you need the brightest minds to solve it. Government jobs may still be attractive for researchers, but if they need to keep such developments a secret, it means they have to limit themselves to the people they can actually hire. This means their talent pool will be more limited than that of a company like Google, or a university like MIT, organizations that can always collaborate with whomever they want in the open, including foreign companies and universities. For building a practical quantum computer, they can have big budgets too, given that companies like Google are interested in machine learning, not to mention the pool of investors that would be dying to be a part of the next revolution. Some of the brightest minds we have worked on quantum computing already, in the open. The idea that a single country's government would be able to do a better job, in secret, is preposterous.

Second, quantum computing doesn't solve P = NP. The difficulty of brute-forcing AES-256 is only reduced to that of AES-128. It is something, but not much and that's only speaking about asymptotic complexity. Going from a feeble experiment in building a quantum computer to building farms of such computers to run distributed algorithms on them - well, I can assure you that farms of commodity hardware with capable GPUs will be used instead for a really long time.

dfc5• 13 years ago

I am channeling grugq here. The answer is clearly stated in Biggie's 3rd Commandment of OPSEC:

  Number 3: Never trust nobody
  Your moms'll set that ass up, properly gassed up
  Hoodied and masked up, shit, for that fast buck
  She be laying in the bushes to light that ass up

The answer from the SIGINT community is

  In God We Trust, All Others We Monitor

The answer from President Reagan:

  Trust but verify

CamperBob2• 13 years ago

My favorite recent variant: "YES WE SCAN!"

alex_c• 13 years ago

So who the F can we trust?

Hint: not the secretive spy agency.

rjbwork• 13 years ago

So just use the CC field, problem solved!

hammerzeit3• 13 years ago

This headline is misleading. It implies the program is still ongoing, where the original article clearly states that the program was shut down 2 years ago.

Moreover, after a lot of cynical complaining about Obama not being meaningfully different than previous administrations, it's worth noting that Obama was the one to shut this down.

I'm not interested in reflexively defending the government or Obama but we still need to pay attention to the facts at hand.

Terretta• 13 years ago

Programs are often shut down when superseded by new or better tools. In any case, both the WaPo and the Guardian article on the same leak says metadata collection is ongoing:

A senior administration official queried by the Washington Post denied that the Obama administration was "using this program" to "collect internet metadata in bulk", but added: "I'm not going to say we're not collecting any internet metadata."

http://www.guardian.co.uk/world/2013/jun/27/nsa-data-mining-...

hammerzeit3• 13 years ago

Don't you think if there was evidence that the government was still harvesting CCs,BCCs, etc en masse, that the article would have made this their central claim? The vast majority of the article is about how this stopped in 2011. One line implies they still collect "internet metadata" but not in bulk.

rhizome• 13 years ago

The information that has come out over the last 12 years has taught us that they will change the name of the program, calling "this program" "shut down," while continuing with what they were already doing under another name, sometimes under different authorities (but always the same conduct).

lemming2• 13 years ago

If you read the article carefully, or even better the original Guardian article: http://www.guardian.co.uk/world/2013/jun/27/nsa-online-metad... you'll find claims that while the original program was shut down something very like it although probably larger in scope is still ongoing - details are murky though.

I instinctively like Obama, but I'm forced to admit that his policies on national security are by any objective means worse than his predecessor. He's just more eloquent when he talks about them.

hammerzeit3• 13 years ago

I read both of the original articles (also http://www.guardian.co.uk/world/2013/jun/27/nsa-data-mining-... ) -- there is still nothing in them that would cause this statement to be reliably true. You may surmise it, but speculation != fact.

Also, I think you must have a phenomenally short memory if you think Obama's policies on national security are stricter than Bush's were.

asveikau• 13 years ago

I instinctively like Obama, but I'm forced to admit that his policies on national security are by any objective means worse than his predecessor.

Maybe the difference is a management style that does a better job following through with things.

Example: Bush spends 8 years going after bin Laden, finds nothing. Obama does it in 2. When I witnessed this it always struck me that probably Bush wasn't trying very hard.

Possible parallel: Bush hears from security apparatus that we need lots of wiretapping, so they do that for a while. Obama gets in there, gets similar advice ... You can guess where that goes.

At any rate I have a hard time believing Bush would say no to this stuff.

mbateman• 13 years ago

What exactly is it supposed to mean that the NSA intercepts only data with one "foreign end"? That it intercepts all data that crosses e.g. a transatlantic cable? Or that it scans the IP header of absolutely everything and grabs anything with a non-US IP as either source or destination? Or something else?

guelo2• 13 years ago

That's where the "minimization" documents from last week come in. They actually collect everything, but then there is some kind of filter at the collection point that is supposed to remove any communication that they are certain is american-to-american. But that filter also has exceptions for things like encrypted messages or pretty much anything else they are interested in. They make the filter as loose as they can while still being able to maintain some deniability that they don't collect domestic communication.

mpyne5• 13 years ago

That really depends on the details of what's capable with their collection equipment.

Transoceanic cables are obvious, but there are also satellite-satellite links that are possible that could terminate right in Iowa for all we know, that the NSA wouldn't be able to retain unless/until it made it to one of the Tier 1 ISPs (they might use geolocated IPs for this).

And as the other comment mentions, the minimization procedures seem arranged to blacklist and discard only that data which is clearly US<->US so you could definitely end up with your data being socked away.

moneyrich22• 13 years ago

Can I ask how? How do you have 75% of the traffic or 75% of the servers (as the article states), how the hell is that logistically possible?

grey-area• 13 years ago

GCHQ in the uk have been doing it for most uk traffic, and it seems the approach is to store full data as long as they can, and store headers for longer. All the content is stored for 3 days, then the headers are kept for 30 days, and shared with other agencies like the NSA, who may well keep it all indefitely if they have enough storage available. They probably do some early filtering to keep it manageable, removing duplicate content, unwanted videos etc. and the headers and metadata are probably not that large. Before reading the GCHQ docs I wouldn't have found this claim credible...

If they are not collecting every communication in the world, you can be sure it is not from lack of ambition to do so. In the words of General Alexander:

“Why can’t we collect all the signals all the time?” the N.S.A. director was quoted as saying. “Sounds like a good summer project for Menwith."

Which is a worryng thought when you realise the implications of this ambition. We used to think that only a god could be omniscient, but that is the current ambition of our intelligence services and politicians.

dfc5• 13 years ago

If you don't want to use that awful doc viewer:

  wget http://s3.documentcloud.org/documents/719116/pages/doc03-p1-large.gif
  ...
  wget http://s3.documentcloud.org/documents/719116/pages/doc03-p52-large.gif

The last time the guardian had a document up and I provided these gifs someone replied with a pdf copy. I am unsure of how to get the pdf from documentcloud. So feel free to post a pdf link and please explain where you get the URL from

mh-• 13 years ago

I took a guess and got it right (dont sue me plz DocumentCloud):

  http://s3.documentcloud.org/documents/719116/doc03.pdf

dfc5• 13 years ago

I regret that I have but one upvote to give you. Many thanks. I swore that last time the pdf had no relation to the gif names.

cliffu• 13 years ago

What you just did is no different than the felony they got weev for.

Edit: well, I guess you didn't falsify any headers, though.

LukeShu• 13 years ago

Someone else posted the link to the PDF, but for downloading sequences:

    wget http://s3.documentcloud.org/documents/719116/pages/doc03-p{1..52}-large.gif

eslaught• 13 years ago

Is email traffic typically encrypted between major providers? E.g. could a network attacker, located between Google and Microsoft, intercept unencrypted traffic between gmail and hotmail addresses?

SoftwareMaven2• 13 years ago

It depends. Bigger providers like Google typically prefer to use TLS, but there are a great many smaller providers who do not. In order to keep everything flowing, there has to be the ability to fall back to unencrypted messages.

pjqwdpjoqw• 13 years ago

It is often encrypted, but since the encryption is negotiated using STARTTLS it can easily be stripped by an active attacker. It works fine against passive attackers.

kintamanimatt• 13 years ago

SMTP relay is almost always unencrypted. The client that connects to the SMTP server may connect via an encrypted connection though, but that's mostly to prevent snooping on the client's local network.

astangl• 13 years ago

They discontinued the program to save just the 3 headers because now they've got other programs that save the entire email message. And phone calls, and text messages and tweets, etc.

jccc• 13 years ago

What if the near-term result of all these revelations is that it just becomes the new normal? Is it necessarily such a bad thing for the snoops in the shadows if they know people will eventually just get used to it? After all nothing really feels different day-to-day so ... meh.

raintrees2• 13 years ago

Data is being collected on all people. All people are guilty of something. When the time comes, a case will be built for the person currently under scrutiny.

http://marginalrevolution.com/marginalrevolution/2013/06/no-...

bobsil12• 13 years ago

If you're a techie, knowing that every bit of data you collect from customers will eventually end up in Utah-- you have a duty to either collect the minimum data possible or encrypt both transmission and storage and demand a warrant for access.

kyzyl2• 13 years ago

Question: Does anyone here understand where exactly systems that incorporate a zero-knowledge architecture fit into the recently illuminated legal framework (re: warrants, etc.)?

i.e. if I implement my service so that I don't have the keys and cannot reasonably obtain them, what does that mean for my users and their data, presuming the data is stored in the US? Juicy example: Lastpass. (I am not affiliated with Lastpass.)

I'm sure this has already been discussed on HN recently, but with the dizzying number of PRISM/Snowden/Leaks/Wiretapping threads flying around it's difficult to keep up.

mike-cardwell2• 13 years ago

Lastpass can access your decrypted password vault if they are compelled to. All they have to do is send you some modified JavaScript which steals your password/key.

They're certainly worried about persistant XSS attacks being used to gain access to peoples vaults. There's nothing stopping them performing one of these attacks themselves, targetted to a specific user.

If you think this is unlikely, look up Hushmail being compelled to send modified java applets to their users to steal their keys. It has been done before.

So yeah, if the US government wants access to a list of all of your accounts, when you logged in to them, what IPs you logged in with and your usernames and passwords, they'd probably be quite pleased to find out you're using Lastpass

chuinard• 13 years ago

I disagree. I have an Amazon EC2 instance and use that as a backend for some apps. I doubt that any of this data transmitted over unsecure HTTP is in Utah.

bobsil12• 13 years ago

If it went through one of the peering points with an NSA optical splitter, and it probably did, then it's probably sitting in an NSA data warehouse. The Utah site specifically is still being built.

jameshart• 13 years ago

I think we would all forgive the intrusion if, as a side effect of this program, NSA was using this data to feed a spam filtering service to which we could all subscribe.

raintrees2• 13 years ago

No, for me it would have to be Daniel Suarez' Daemon. Then I could skip over the forgive part, since many alphabet agencies may be rendered obsolete... And all spammers would have ceased or been desisted.

Good story, if you haven't yet read/heard it.

poster692• 13 years ago

This thread is big meaningless distraction... The main point is: You Are all being illegally spied on The land of the free is a big lie.

Aqueous• 13 years ago

Here's an idea: If you don't want to make an anxious public even more anxious, don't name your NSA surveillance program "EvilOlive." Or really, anything starting with 'evil.'

askimto• 13 years ago

Anyone ever hear the rumor that the reason why Google pulled out of China was because Chinese hackers had tapped into a feed of all email metadata? I heard it included subject. This news made me immediately think of that rumor.

tonyplee• 13 years ago

Will google pull out of US now? :-)

Can't fight big government. China or US, really doens't matter.

New leak shows NSA harvests To, From, and Bcc lines of e-mail data

Related Stories