What ISPs can see

https://www.teamupturn.com/reports/2016/what-isps-can-see

by schoen • 10 years ago

225 81 10 years ago

q1t• 10 years ago

So how do you protect form such things? I mean is there a way to analyze all you outcoming traffic (from a specific machine for example) and route every connection(like dns and similar stuff) though desired endpoint?

amelius• 10 years ago

So how do you protect form such things?

Tor?

ams61103• 10 years ago

Unless you believe Tor is a giant honeypot created by the NSA.

bluedino2• 10 years ago

You can use a VPN or ssh tunnel and send DNS queries through that tunnel or proxy - however, that just adds another layer for anyone who wants the information to go through.

dreamfactory22• 10 years ago

Do VPN providers know any less than an ISP?

ams61103• 10 years ago

I don't use my ISP for DNS. Nor for email. Not sure that really protects anything but at least it sidesteps any log mining they're doing on their own servers.

cJ0th• 10 years ago

One (albeit small) thing you can do is to do much as possible offline. For instance you could download Wikipedia and use it offline only.

The same goes for maps. For example, Open Street Map data can be used offline.

pmoriarty• 10 years ago

One significant point that this analysis misses is that even when traffic is encrypted, it can be recorded and later decrypted by the ISP or those they give/sell/leak the recordings to if the encryption is ever compromised (which seems to happen on a pretty regular basis with SSL these days).

Never let yourself get lulled in to a false sense of security just because the information you wish to keep private has been encrypted.

jmiserez• 10 years ago

This is not necessarily true if the cipher used provides forward secrecy, is it?

From https://en.wikipedia.org/wiki/Forward_secrecy:

As of March 2016, 49.4% of TLS-enabled websites are configured to use cipher suites that provide forward secrecy to modern web browsers.

mattmanser• 10 years ago

Serious question, that's not generally how SSL vulnerabilities work is it?

pilsetnieks• 10 years ago

No, it's usually flaws in implementation that are discovered, thus perhaps allowing to recover the key (for example, see the news about openssl for the past few years,) or maybe recover the information in transit.

Whereas if you have a record of encrypted traffic, you have to find a flaw in the encryption algorithm itself, or wait until someone else does.

ams61103• 10 years ago

Until recently it was technically or financially unfeasible for an ISP to record all traffic and keep it forever. Maybe today it isn't (or maybe it still is -- I have some experience using large, petabyte-scale, reliable data archives and they are from what I've seen still expensive and slow), but does anyone have knowledge if it actually happening? If it is, how is it being financed? Why would a competitor not undercut with better performance/cheaper pricing by not building such a logging infrastructure?

yesitis• 10 years ago

Had to create additional account , but for sure some countries do and understand lot of american companies data if not in rest outside but in transit... There are companies that avoid these but hmm....

bendykstra• 10 years ago

TLS connections that implement forward secrecy are not vulnerable to this type of attack. According to SSL Labs, about half of all sites now support forward secrecy.

https://www.trustworthyinternet.org/ssl-pulse/

gcr• 10 years ago

Only if it's implemented correctly.

sibrahim• 10 years ago

This may be true for now, but if/when scalable quantum computing arrives, the recorded key exchange can be used to recover the session key (much easier than attacking AES itself). If you need confidentiality in the face of quantum adversaries, you'll need post quantum crypto but this is still a fairly young area of research.

traff• 10 years ago

It's possible to decrypt the exact URL you're browsing for a large majority of websites with very high probability, even with forward secrecy. There is an undergraduate project that does this for wikipedia pages (it's easy because of all the unique resources loaded for each page). Search for papers on https traffic analysis, for example:

http://arxiv.org/abs/1403.0297

pippy• 10 years ago

Google could easily take the initiative on the DNS query front and implement DNSCrypt by default on Chrome. It would booster client privacy and also block ISP from selling usage data. So it would be a win-win for Google.

PhantomGremlin• 10 years ago

So it would be a win-win for Google

Yes, but would it be win-win for us?

Google's already demonstrated that "don't be evil" is now just a sad memory. I'm not ready to believe them to "do the right thing". Their entire existence is predicated on increasing and refining their data collection and analysis, and acting on such.

Google's seedy behavior already directly impacts me every day. I, for one, don't welcome this new corporate overlord.

eikenberry• 10 years ago

How would this help if they are using the ISP's DNS servers as most people do by default?

stingraycharles• 10 years ago

This can be promoted by offering the user the choice of several dnssec enabled public dns servers in Chrome. Or someone makes an extension that does this by default, maybe that is easier to promote.

mirimir14• 10 years ago

Nobody paying attention uses their ISP's DNS servers. See https://www.wikileaks.org/wiki/Alternative_DNS

Also, good VPN services run their own DNS servers, which are reachable only through the VPN tunnel.

See https://dnsleaktest.com/ to determine which DNS server(s) you're using.

arca_vorago• 10 years ago

What we really neednis DNSCurve. CRypt is full of problems from what I hear. All hail djb.

achernya• 10 years ago

That is not sufficient -- TLS Server Name Indication (SNI) is still cleartext in the handshake.

kjaer• 10 years ago

Wouldn't HSTS preloading solve that?

therealmarv2• 10 years ago

true, but as I said in another comment here: It will not hide the websites you are visiting. This is also a problem in the article... a secure DNS will not make you invisible it is only slightly harder to track the websites you are visiting. All IPs you are visiting can still be transformed through a reverse DNS and you will get all website addresses.

And Chrome cannot be use dnscrypt by default. It uses UDP ports which are sometimes closed on other networks. So there are technical limitations. Even using another DNS than the network one is often not allowed (you will experience that if you travel often).

Also some people will not like using a Google DNS by default ;)

dmichulke• 10 years ago

IMHO this would still be a huge step because the URL path after the hostname reveals very specific information relative to the host (say, pornhub or webmd)

mirimir14• 10 years ago

This is a decent article. However, it's a bit vague on the vulnerabilities of VPN services. The major risk is probably traffic leakage after uplink interruptions, or changing WiFi APs. Once the VPN connection has failed, default routing must be restored in order for reconnection to occur. There must be firewall rules to prevent other traffic from using the physical NIC during reconnection. That is, you want the VPN to fail closed.

Another risk, which the article may vaguely allude to, is using ISP-associated DNS servers. Even if all traffic uses the VPN tunnel, DNS requests reveal sites being visited, and it's trivial for ISPs to correlate them with traffic.

IPv6 is a huge looming risk. Many VPN services don't route or block IPv6 traffic. As full IPv6 service becomes widespread, there will be major pwnage. However, this is easy to firewall, and good custom VPN clients do so.

Edit: For suggestions about leak-testing, see https://www.ivpn.net/privacy-guides/how-to-perform-a-vpn-lea...

Edit: Changed "URLs" to "sites".

nickspacek• 10 years ago

Not to nitpick but DNS requests only reveal hosts being visited, not URLs. Non-HTTPS requests reveal the URL to the ISP.

mirimir14• 10 years ago

Right, thanks for the clarification.

nickysielicki• 10 years ago

If anyone is curious about securing their DNS requests, see https://dnscrypt.org/

therealmarv2• 10 years ago

I've used that for several months. Regarding security in my eyes this only secures that the DNS request is not hijacked by a third party (like Turkey has done with Google's 8.8.8.8 DNS once). All the IP addresses your computer visits can simply be backcoded through a reverse DNS and so it does not hide which websites you are visiting.

Matt3o12_2• 10 years ago

Don't forget WebRTC. It can leak your real IP address and might even be able to use that route.

Nevertheless, setting your Firewall to only allow VPN traffic is pretty easy. I've done that on Linux and Mac OS X. I don't think it is complicated on Windows either.

mirimir14• 10 years ago

Please share how that's done in OS X.

In Windows Firewall, you label the physical and tap adapters as different domains. Then you allow only connections with the VPN server via the physical adapter. It's not all that different from iptables. I'm not very familiar with pf, but I'm guessing that the approach is similar.

WebRTC is indeed invasive. But WebGL is arguably worse. HTML5 overall is hard on privacy.

newman3143• 10 years ago

I guess this is a good time as any to ask what people on HN use for a VPN provider.

I'll create a Ask HN post if there is interest...

tomclancy• 10 years ago

Someone on Reddit created a thorough breakdown of VPN providers: https://docs.google.com/spreadsheets/d/1FJTvWT5RHFSYuEoFVpAe...

newman3143• 10 years ago

Thanks. I'll check it out.

darpa_escapee• 10 years ago

See the thread, this list isn't thoroughly accurate at all.

Estragon• 10 years ago

iVPN. Mainly based on their excellent privacy tutorials: https://www.ivpn.net/blog/privacy-guides

Steer clear of PIA. https://news.ycombinator.com/item?id=8982095

newman3143• 10 years ago

One issue I have with PIA is the non-unique PSK.

ionised• 10 years ago

Try this article;

https://torrentfreak.com/vpn-anonymous-review-160220/

I personally use Mullvad. They let you pay in cash by post which is always nice.

Speed is decent enough to stream/download but it won't get near maxxing out my 200Mbps connection. I've yet to find a VPN provider that offers amazing bandwidth along with anonymous payment.

snug• 10 years ago

SNI header being sent from the client can probably show even better patterns from the user.

userbinator• 10 years ago

I've always wondered why that's sent in the clear --- the usual justification is that the hostname needs to be known before the right certificate can be used, but that can be gotten around by using a certificate named with the IP address of the server, establishing encryption, and then sending the hostname to get that certificate.

droopybuns• 10 years ago

This is a weird article that lacks important context.

When are we going to get an article on "what google can see" from this team?

ISP snooping on network traffic really only happened after Google started getting into the ISP business.

Monetizing your traffic (beyond transport) only became a thing after Google Fiber fired a cannon ball across the broadside of carriers. Carriers responded by offering ad networks around anonymized, aggregated data about customer behavior. It drives the value of Google Adwards down. It doesn't make carriers rich.

Maybe someday we'll need to worry about big-I innovation from carriers in this domain, but I don't see it happening soon.

I'm all in on tearing it all down. But focusing only on the companies that offer services that huge populations are willing to actually pay for is extremely dishonest.

mirimir14• 10 years ago

ISP snooping on network traffic really only happened after Google started getting into the ISP business.

ISPs have always snooped. Maybe just in limited circumstances, true. But the key point is that they can, not how prevalent it is.

scholia• 10 years ago

There's much more information in the original 125-page paper:

Online Privacy and ISPs: ISP Access to Consumer Data is Limited and Often Less than Access by Others http://www.iisp.gatech.edu/sites/default/files/images/online...

sysret• 10 years ago

The fastest DNS lookups are ones that do not need to traverse the network.

A zone file of public DNS information can be served by a daemon bound to the loopback on the user's device, obviating the need for many (but not all) lookups sent over the network.

These local lookups are also more private than ones sent out over the private LAN or public internet.

Same goes for any type of data. It's not limited to DNS information.

If a user downloads publicly available data dumps from Wikipedia, and then serves them from a local database and httpd, the response time will be faster and the requested URL's more private than accessing the same content over the public internet. Not to mention the small benefits of reliability and reduced dependence on the network.

I know a user who does this and has automated the process of setting it up.

To use the examples in the article, the idea is that a user can periodically download bulk data, e.g., information on medical conditions, in an open format, load it into her database of choice and query to her heart's content, without any ISP or website knowing what she has queried.

Same with daily newpaper content, and even a catalog of toys. "Browsing" through the data remains private.

The alternative is to have this data served from third party computers and have the user send each and every request for each small item of information over an untrustworthy, public network (the Internet).

Despite ample, inexpensive local storage space for users to store data of any kind themselves, let us break up the data into little bits and make users request each and every bit individually. (Not only that, let's make them register for the the ability to make numerous queries.) Then we can record all user requests for every item of data.

Metadata. Sell. Profit.

Matt3o12_2• 10 years ago

It should also be mentioned that VPNs, even if correctly used (e.g. no dns/IPv6/webrtc leaks), this simply shifts to trust to another provider. Now, your ISP is not able to see your traffic but your VPN provider is potentially able to (even if you self host it on aws or digital ocean because they still have full access to the box), and their ISP. If you trust them more, use them, but I you don't, I see little reason to utilize a VPN unless you want to unblock geo-blocked services.

The only advantage is that your VPN provider (or their ISP) might have little reason to spy on your traffic instead of your regular VPN.

mirimir14• 10 years ago

The only advantage is that your VPN provider (or their ISP) might have little reason to spy on your traffic ...

Yes, that's the point. You want to pick VPN providers that can't readily be forced to spy. See https://docs.google.com/spreadsheets/d/1FJTvWT5RHFSYuEoFVpAe...

obfusc82• 10 years ago

Always assume an ISP and any intermediate party can see traffic. I like the way this post outlines in simple terms what traffic would look like to an analyst. In terms of obfuscating oneself from an ISP, I would not recommend any form of centralized traffic in the first place. As this post makes clear, even if you're behind seven proxies there are ways to see a traffic's 'shape' on the wire.

To combat this, you can use compartmented, disposable and anonymous 3G sim cards for specific purposes. (One for dating sites, one for health records, etc). Slap them in the microwave after a browsing session. (You can get these for basically free in places like Thailand or India). Block all HTTP. Use something like

    sudo ufw deny out to any port 80

Always assume your connection is tapped. Always assume there's somebody MITM'ing your traffic. (To prove this, download executables several times over time and diff the hashes. It's clear that MITM happens all the time).

Always use a hardware version of TOR. That way if a box is compromised, the naked IP can't be disclosed. The same goes for VPNs, See WebRTC vulns.

Use public Wifi as much as possible (behind a VPN of course). Use your friends phone for casual surfing. Minimize the reliance on one monolithic connection. Use 4G, or even WiMax if they have it in your area.

Share your connection with your neighbor and split the bill if you are so inclined...

crispyambulance• 10 years ago

"always" [assuming the worst case scenario] is more than a little onerous for the vast, vast majority of internet users. Isn't there a reasonable middle ground?

Also I don't follow how does one "prove" a MITM attack by downloading the same executable serveral times and getting different hashes?

obfusc82• 10 years ago

for the vast, vast majority of internet users.

Well, unless this is baked in, which it is not. It's the old privacy rich vs privacy poor debate. If I buy black curtains, I cast less of a (nude) silhouette than my neighbor for all to see, but the tradeoff is, I have to research black curtains on the internet, where there is no privacy, and so I have no choice but to build my own private Internet.

If the internet was private, no such measures need to be taken and I have perfect autonomy. Autonomy being a luxury since the digital space has effectively perfect memory.

This is why I'm against logs and data retention. It's very un-natural and it's why the human brain habitually flushes memories. Nature needs to renew itself and re-invent itself, and in some sense, forget itself (if you believe in a Gaia mind).

What ISPs can see

Related Stories