CloudFlare, SSL and unhealthy security absolutism

https://www.troyhunt.com/cloudflare-ssl-and-unhealthy-security-absolutism/

by jgrahamc • 10 years ago

141 114 10 years ago

TL;DR (for those who know already how CF works): CF doesn't advertise whether a website origin is reached via HTTPs or not ("Full" SSL mode in CF terminology). The author advises them to expose the information in a response header.

phlo5• 10 years ago

On one hand, Troy is right: absolutism can hurt security, and a partially encrypted connection (e.g. CloudFlare's "flexible" version) is better than a wholly unencrypted one.

On the other hand, opportunistic encryption as described can also be misleading and dangerous. Yes, most attackers will have a harder time achieving MITM between CloudFlare and your EC2 hosts than an unsecured Starbucks Wifi, but the possibility remains. And in the case of the non-"(strict)" versions, there is no way for the user to tell that the data was not encrypted. The security community has spent a lot of (well-placed) effort training users to look for the padlock symbol before trusting a website. CF's opportunistic encryption undermindes that. Files can be served over insecure HTTP to CloudFlare, and even a competent end-user will have no way to tell.

As Troy mentioned, Let's Encrypt and CF's Origin CA are steps in the right direction. Trust On First Use could be another one, allowing for a partial departure from the CA model. In any case: opportunistic encryption is a big improvement over no encryption at all -- but it should be clearly recognizable as such, and must not be confused with identity verification.

saurik• 10 years ago

A CDN is effectively part of the developer's infrastructure. One could also take data from their endpoint and publish it on a public log somewhere with the compete traffic dump. The green lock can only tell you that your connection from your network to some endpoint not on your network is encrypted, not that the connection is secure; trying to force CloudFront to display "past this point things are less secure" will, if anything, train users to believe the lock means something it can't ever mean: that your data is secure in general, as opposed to just being secure on your network. If the developer has another proxy (maybe nginx) behind CloudFront managing connections to an application server over HTTP, should that also send this header? If the web application is using an unencrypted database backend connection, should common web frameworks start returning that header? At what point do we start only giving websites green lock icons when they pass an independent security audit? We need to be downplaying the overall importance of the green lock icon, not reinforcing it: users already think it means way way more than it ever could (specially, "it is safe to give my credit card number to this website", which is almost totally unrelated to what the lock icon is for :/).

phlo5• 10 years ago

A CDN is effectively part of the developer's infrastructure.

Agree. If the CDN is compromised, all bets are off. In my opinion, CloudFlare Flex/Full are a lot more vulnerable to attack (because the resources can be obtained insecurely over the public internet) than others (where TLS is either terminated at the network boundary and encryption is only lacking thereafter, or where resources are pushed through secure connections over the Internet).

as opposed to just being secure on your network.

I'm not sure it is a reasonable thing to ask end-users to understand how many networks are involved in serving their content. I also don't think the situation would benefit from CloudFlare (or any other proxy) announcing which parts of the connection they encrypt. Instead, I think CloudFlare should be more strict in their origin connectivity: only accept encrypted data, and if the origin certificate is not a CA-issued one, Trust on First Use and ask the site admin to verify updated certs. CF is also in an excellent position to adopt a perspectives [1]-like approach, if an MITM is suspected.

[1] https://perspectives-project.org

_pmf_• 10 years ago

Trust On First Use could be another one, allowing for a partial departure from the CA model.

Any step away from the horrible "HTTP is fine, but god help you if using a custom certificate" scaremongering currently implemented by browsers is good to me.

I absolutely cannot understand this: if is's a trusted, non expired certificate, show it as green. If not, there's nothing more insecure about it than using plain HTTP; should we display huge scary warnings for plain HTTP, too?

acdha• 10 years ago

That's been in progress for the last couple years. They've been making certain features like geolocation only available to secure origins and the browser UI is changing:

https://www.chromium.org/Home/chromium-security/marking-http...

regecks• 10 years ago

Opportunistic encryption in HTTPS was meant to address this, but I think there were problems.

tptacek6• 10 years ago

What's a "custom certificate"? A self-signed certificate provides no security: any MITM could simply substitute their own.

nbevans• 10 years ago

I didn't know about their new'ish Origin CA offering. Cloudflare should be pushing this more as it is clearly the solution to their Flexible SSL and non-strict Full criticism.

prdonahue6• 10 years ago

We announced it in May (https://blog.cloudflare.com/cloudflare-ca-encryption-origin/), and built it for some of the exact reasons you specify.

We've made the generation as simple as possible—just two clicks—and also offer an API for users aiming to automate the process: https://api.cloudflare.com/#cloudflare-ca-create-certificate.

At some point during onboarding of new zones/hosts/etc. I'd like to start encouraging users without a (valid) origin certificate to install an Origin CA cert.

Curious to hear your thoughts on how else you'd "push it" more?

ihunter• 10 years ago

Full: Still HTTPS from CloudFlare to the browser but they'll also talk HTTPS to the origin although they won't validate the certificate

This isn't true anymore. I would consistently get SSL invalid errors from them when trying this setup. You have to have a valid SSL cert for CF <-> Your server now it seems.

icebraining3• 10 years ago

Are you sure you didn't select "Full (Strict)" ?

prdonahue6• 10 years ago

If you're getting a 526 error, it's almost definitely because you had Full (Strict) mode enabled. Please reach out to https://support.cloudflare.com if you are still having difficulty and mention in the ticket that I sent you.

DyslexicAtheist2• 10 years ago

> CloudFlare being "insecure" is much more nuanced ...

like different shades of beige, but in the end it's still beige ... [0][1][2][3]:

[0] https://scotthelme.co.uk/tls-conundrum-and-leaving-cloudflar...

[1] https://blog.torproject.org/blog/trouble-cloudflare

[2] https://people.torproject.org/~lunar/20160331-CloudFlare_Fac...

[3] http://cryto.net/~joepie91/blog/2016/07/14/cloudflare-we-hav...

Specifically they:

  - Inject javascript into your code while returning pages.
  - It modifies the code of your pages.
  - Modify the http headers of your pages.

They can block your site for visitors and of course track everyone on it.

Cloudflare is a pest. They hired smart people to do stupid things. What's worrying is that a smart guy like Troy (who I hugely respect and follow on twitter) doesn't see through their BS.

DyslexicAtheist2• 10 years ago

additionally you might want to check what other domains are sitting on the cloudflare IP. last time one of my sites shared the IP with an adult content & gambling site.

scrollaway3• 10 years ago

You're sharing the internet with far worse than gambling sites, and it's just as irrelevant. Did you expect to get your own private IP4? If not, why does it matter who else is on there?

FUD.

dineshp2• 10 years ago

The fact that all traffic is decrypted at their servers makes possible caching optimizations (what most users care about) and inject content,headers. This is the best example of the centralization of the internet.

Does anyone know what Cloudflare's response was regarding them treating Tor traffic suspiciously by putting up never ending captchas?

ryanlol4• 10 years ago

caching optimizations (what most users care about)

Cloudflare does a really shitty job at that though, I've literally never seen them beat a reasonably configured nginx instance running on GCE or even OVH. (That is for a single instance serving both EU and US markets).

It's honestly baffling that anyone who isn't drowning in bandwidth bills would use them, but I guess antiviruses and PC optimizers are a big industry too.

jgrahamc6• 10 years ago

We are working closely with the Tor project. We made a large number of changes to reduce impact and improve the experience.

hyperpape• 10 years ago

Cloudflare and Tor, several months ago: https://news.ycombinator.com/item?id=11404770 https://news.ycombinator.com/item?id=11388560

MichaelGG4• 10 years ago

Trump's website throws a captcha every time I access it from Central America. This is for read only pages. I can only assume CF doesn't care too much and certainly doesn't educate customers very well.

Edit: I'll also admit I love CF as a customer. It makes things fast and easy. But it's concerning. Sorta like every time I use Google search.

92482• 10 years ago

This is actually not just about Tor. Their definition of "suspicious" is quite liberal and naive to be honest.

As an anecdote, I get ~1 captcha per month because I live in Eastern Europe. Add to that Linux, Firefox, NoScript and you're in for a fun ride.

anonbanker• 10 years ago

I get the very real feeling, when using cloudflare-based sites over Tor, that I'm being tracked specifically by the captchas used. It's always the same two or three.

revanx_• 10 years ago

'Inject javascript into your code while returning pages'

I absolutely hate this, you go to a completely benign website but you can't see the content unless you allow the injected CF to run in your browser.

tombrossman• 10 years ago

Here's another interesting link, for what it's worth: http://www.crimeflare.com/cfssl.html (don't let the funny title and amateur look fool you, it's regularly updated).

_asummers2• 10 years ago

Don't forget about eating requests it thinks are SQLi. We had an English sentence in the POST body that we reduced the content of to '"a" or b in c' that cloud-based was returning a 401 on. But '"a" or b' nor 'a or b in c' did. That was a fun time tracking down. Luckily the users that experiencing the issue were in house, so they actually reported the bug.

jgrahamc6• 10 years ago

Did you report that to us?

lmm6• 10 years ago

Not wanting the green padlock in the URL bar when traffic is being sent unencrypted and unauthenticated over the open internet is not unhealthy absolutism. It's a basic necessity if the green padlock is to mean anything at all.

It is of course theoretically possible for any site operator to pass information that was submitted securely, insecurely over the open internet. It probably happens, and we should look for ways to eliminate that possibility. But the fact that we can't eliminate all the possible ways that might happen is not a good reason for CloudFlare to make it very easy for site operators to do.

"Flexible" does prevent a small class of attacks that are possible against plain HTTP. But it does so at the cost of completely invalidating HTTPS URLs, padlock icons and so forth. Making it easier for small site operators to get a bit of security against the most simplistic attacks is not worth making it impossible for users who are serious about security to have good security. Worse, the benefits accrue to CloudFlare and their customers, but the costs are shared by all internet users. It is an antisocial practice and it needs to stop.

andybak• 10 years ago

"Flexible" does prevent a small class of attacks that are possible against plain HTTP.

Or as Troy phrases it: "protect against the 95% (or thereabouts) of transport layer threats that exist between your visitors and your origin"

Would you consider your use here of the phrasing 'small class of attacks' justifiable? If so then you should describe your threat model in a bit more detail.

lmm6• 10 years ago

Sure. My threat model is roughly to divide attackers into three categories:

1) Amateurs 2) Professionals not targeting specific individuals (i.e. spammers, credit card fraudsters etc.) 3) Professionals targeting specific individuals (i.e. corporate espionage, government-like threats)

Let's go with the article and assume that category 3) is virtually impossible to defend against, and ignore them hereafter.

People in category 2 are concerned about volume efficiency - they will perform those attacks that are cheapest per CC#. They don't care about handcrafted attacks against individuals, or even edge networks of a few hundred people. Their main attack on unencrypted HTTP will be passive listening on big backbone connections, which is achievable with their resources, and which "flexible" does not protect against - indeed it makes people much more vulnerable to.

Who are the people using Firesheep and similar tools? Technically-inclined students playing pranks (maybe a decent chunk of "threats" by number, but not really a threat you need to defend against)? I just don't see it being used as part of serious attacks. No carder is going to bother. No spam-information-harvester is going to bother. Maybe people with personal grudges who are at just the right level of technical skill? But how common is that? (This can vary a lot - I can see that it would be an issue for public figures with controversial political positions, victims of stalking etc. - but not for the typical "person in the street")

In other words I think category 2 is the vast majority of (damage-weighted) threats; the threats Flexible protects against are mostly irrelevant, and the threats that Flexible makes worse are precisely the most important class of threats for most "ordinary people".

anon13852• 10 years ago

if the green padlock is to mean anything at all.

The padlock doesn't mean anything at all and never has. It's snake oil from an era when e-commerce wasn't trusted by the general public. It only ever existed to encourage/trick technologically illiterate people into handing over their credit card details to websites when the pervasive public opinion was that entering your CC number into a website was dangerous. It was very successful in achieving that aim.

In 2016 it has no purpose and browsers should remove it.

icebraining3• 10 years ago

It means "the most vulnerable part of the connection - the last-mile - is probably secure". That's still useful information.

nothrabannosir• 10 years ago

All this talk about what CF should do, I'm surprised they didn't mention the most obvious step to take: offering an SSL mode that lets you configure what host name to validate the origin as. This would work perfectly for anyone using GitHub pages, cloud front, hyperdev, etc etc etc, all of which have valid ssl Certs, just not for your origin but for theirs.

Am I missing an obvious reason for this not being an option? It must have crossed their minds at some point..?

(Edit: come to think of it, iirc cloud front does support wild card ssl for your own domain these days?)

lorenzhs4• 10 years ago

This would solve Troy's issue with his own site, too - his hoster serves it with a * .ghost.io certificate. If he could configure CloudFlare to only accept certificates that match troyhunt.ghost.io, that would be a lot safer.

I have the same use case with GitHub pages -- https://www.glowing-bear.org is Full SSL in Cloudflare, but I can't switch it to Full (strict) because the upstream certificate is for * .github.io

prdonahue6• 10 years ago

In the future we plan to allow for more granular origin certificate validation.

For example, if you are CNAME'ing www.example.com to example.anotherprovider.com, you /may/ be fine with us checking that the origin certificate has a SAN matching that destination, *.anotherprovider.com, or a hostname that you specify.

jacques_chester• 10 years ago

As a nitpick, the proposed `x-cloudflare-crypto: full` header should simply be `cloudflare-crypto: full`, per RFC 6648[0].

[0] https://tools.ietf.org/html/rfc6648

w8rbt• 10 years ago

Many technologies break the end to end principle. NAT, load balancers, caches, security inspection/monitoring devices, etc.

This occurs everywhere on most every network and should not come as a surprise, although many technologists and researchers argue against it while others argue the principle is holding back advancements.

https://en.wikipedia.org/wiki/End-to-end_principle

0xmohit• 10 years ago

This is particularly important for those paying for bandwidth as it slashes that cost by 84%

Feel sorry to read such a generic statement.

Full (strict): CloudFlare issues the certificate and they'll intercept your traffic, but then it's all HTTPS to the origin and the cert is validated as well

Ah! Seems to say that it's ok if your certificate issuer were to be a MITM.

What it means is that you can choose how much SSL you want ..

Interesting point. So privacy and data integrity between two parties can vary, the states being: (1) off, (2) flexible, (3) full, (4) full (strict). Amusingly, even when you choose option 4, i.e. full (strict), your certificate provider would act as a man-in-the-middle.

[It makes the entire article seem like a Cloudflare campaign. Thank you.]

tptacek6• 10 years ago

Protection against "passive attacks" is mostly meaningless: if you can observe packets, you can hijack sessions. It's only the certificate validation that Cloudflare apparently performs only in "strict" mode that prevents this attack.

In particular, the "full" TLS they offer that Hunt is taking advantage of (because he can't deploy a working certificate) is grievously insecure: he's essentially running his system on top of the equivalent of the "goto fail" vulnerability in Safari that the Internet was (justifiably) freaking out about two years ago.

Hunt is smart about a whole lot of things, but he's wrong about this. There's nothing "unhealthy" about the absolutism here: you either have end-to-end cryptographic security, or you're vulnerable.

jgrahamc6• 10 years ago

It's easy to use our Strict mode. We'll give you a certificate for the origin for free: https://blog.cloudflare.com/cloudflare-ca-encryption-origin/

Or you can bring your own certificate from someone like Let's Encrypt.

tptacek6• 10 years ago

If it's that easy, why even have "strict" mode? Why not make the branding less misleading, and rename "Full" to "Weak"?

dsl3• 10 years ago

You should really reach out to Andy Ellis. Akamai forces cert pinning when HTTPSing back to customer origins for some very good reasons that he can explain to you.

type0• 10 years ago

There's nothing "unhealthy" about the absolutism here: you either have end-to-end cryptographic security, or you're vulnerable.

I'm not sure why he talks about absolutism, where this usage basically invalidates the whole point of TLS. This seems like an after-rationalization on why he chose to use CloudFlare, there is much more reasons not to use them. In the end of course it depend on what type of site you use it for.

benmmurphy• 10 years ago

it kind of sucks that cloudflare can't check the servername for the certificate he wants to use. i haven't used cloudflare but I assume there are three ways it might be checking the cert at the backend:

1) using the real hostname of the website

2) using the hostname of the backend website - assuming you supply a url with a hostname to cloudflare. however, this hostname in the backend url is used in the Host header which ends up causing lots of problems.

3) using the hostname of the backend website - but the hostname of the real site is used in the Host header.

4) explicitly let you choose the servername you want to verify

imo. it would be nice if cloudflare gave you the option between 1) and 3) or 4).

EDIT: oh. just noticed the cloudflare reply with the link to: https://blog.cloudflare.com/cloudflare-ca-encryption-origin/

drdaeman• 10 years ago

It shouldn't rely on server name, but on the actual key or signature. Actually, I believe many of hosts running behind the CloudFlare wouldn't want to expose any meaningful server names, so the actual server location would be harder to spot.

I.e. you generate your keypair and produce a self-signed certificate, you should be able to give your public key and the actual key would be used for authentication. The subject (and SANs) of the certificate are ignored so it could be like "/CN=localhost" - only the key material is what matters here. That would be secure - in a sense third-party attacker won't be able to sniff what's between you and CloudFlare.

(Or you could generate your own (self-signed) CA, generate a certificate signed by this CA of yours, and give CloudFlare your CA's public key. This makes sense if you have multiple hosts and you don't want to share the same key.)

YBMckUGrk4Pk2• 10 years ago

While in most cases - Cloudflare is better then no Cloudlfare as discussed in the article. It seem however it's nontrivial for a user to determine whether or not a site is hosted on Cloudflare, because there is certain threats that it can't protect against. If people had a browser plugin to alert if the vendor is capable of doing such MITM, then it would be up to the user if they want to recieve content where it's capable of not being sent end-to-end. Sadly it's a bit hard to make that judgement call, since all CDN's do very similar things.

The only way to make this "less bad" would be if there was a way to determine if the backend infrastructure was in fact loaded end-to-end via TLS. Perhaps some non-cachable content? Not sure what the right answer is here.

_--2• 10 years ago

"it's nontrivial for a user to determine whether or not a site is hosted on Cloudflare"

It's easy to detect when Cloudflare is being used. There are multiple ways.

anexprogrammer2• 10 years ago

Just use the net via a VPN when out and about. You'll rapidly find out who's running CF, and how incredibly annoying their constant captchas become. (Approx 25% of the time CF decide my VPN provider (PIA) needs captchas. This lasts a couple of days, then rest of week clear. Repeat)

They're positively surfer hostile.

_--2• 10 years ago

Their business model has always been MITM, sniffing and modifying content. They do not just blindly pass traffic. Cloudflare and end-to-end encryption are incompatible. They need access to the traffic. What is troubling is how easily they can fool customers or end users into believing that they are offering satisfactory encryption when in truth they are not.

Stepping on to soapbox... TLS needs to be replaced with something simpler, with fewer "buttons, dials and knobs". TLS pontificators need to separate authentication from encryption. They are two different problems, and it's possible we may only have a satisfactory solution for one of them. If that's the case, and it's the former problem that remains unsolved, then it could be worthwhile to reconsider the relative value of encrypted "channels" versus per packet encryption.

aidenn02• 10 years ago

The question is what fraction of attackers that can't force cloudflare to pass them their data is capable of attacking server <-> cloudflare versus cloudflare <-> client.

Hunt seems to posit that the above ratio is small, I take it you disagree?

On a side note, I don't get why cloudflare doesn't let you pin a public key through their interface; that would remove the need for CA signing. Considering you have exactly one client (cloudflare) connecting to your server, it's not like key distribution is hard. Of course this is moot now that cloudflare offers you signed certificates for free, but would have been a simple solution.

tptacek6• 10 years ago

"Small"? What does that mean? Cloudflare doesn't have special private backhaul networks to origin servers; they use the public Internet like everyone else.

brians• 10 years ago

He even has a working certificate---just for a different name.

It sounds like Hunt expects to not see much interception and manipulation at the "backbone." My own experience suggests that such is common at national borders, and becoming common at more and more ISP borders.

chx• 10 years ago

TPB has been an enormously popular site for the last decade and a bit that has served primarily to distributed copyrighted material [...] is exactly the sort of site that piques the interest of governments.

Well this deserves at least a comment -- I know this is a rhetorical comment: why on earth would a government need to be involved with copyright infringement to the level where now it's "exactly the sort of site". Are these websites used to plot mass murder / rape, treason and the like? Nope. Then why on earth would a nation state get involved with it?

arca_vorago• 10 years ago

Honestly Im curious how mich cloudflare is needed for most sites. What amount of views per day would make it worth it?

I know also that malware is now abusing these cdns, so I have been forced to ban entire cdn ranges. I wonder if cloudflare has made any progress implimenting DNSCurve?

phonon• 10 years ago

Yes, except for missing the critical aspect that you can't restrict your site from delivering SHA-1 signed Cloudflare certs over TLS 1.0 on their Pro plan...in order to restrict to TLS 1.2 and SHA256 you have to pay $200/m+.

patcheudor• 10 years ago

"Even with the presence of encryption to the origin, an MitM can still observe the HTTPS CONNECTs the browser makes at the proxy level. It won't show what's in the request, but it will show who they're talking to so there's your metadata. In case you think that doesn't matter, remember that governments kill people based on metadata and in many ways it's more valuable than the message contents itself."

While Apple is mostly patched at this point, numerous flaws were recently discovered with how the CONNECT method was implemented, allowing an attacker to fully get into the middle of HTTPS communications in at least iOS and OS X:

http://falseconnect.com

Slightly less concerning, but still very bad, researchers also discovered methods for reading HTTPS URLs and cookies through WPAD vectors:

https://auth0.com/blog/heads-up-https-is-not-enough-when-usi...

deftnerd• 10 years ago

Cloudflare gets a lot of hate for the core concept of their business model, but I still just don't see it as nefarious or even weak in terms of security if you set it up correctly.

I always configure my Nginx instances to only respond to Cloudflare's IP ranges.

I tell Cloudflare to sign all communications with my origin server and I configure Nginx to validate those signatures.

I encrypt the communications between my origin and their system.

How can this be man-in-the-middle attacked even by a state actor? I'm ensuring that not only is the data encrypted between Cloudflare and my origin server, but that I'm verifying that nobody is able to impersonate Cloudflare.

jgrahamc6• 10 years ago

Nice to see you using our authenicated origin pulls feature: https://blog.cloudflare.com/protecting-the-origin-with-tls-a...

I think we get a lot of criticism because we explain clearly what we are doing and people pick over it. There are always going to be criticisms but I'd rather we be transparent and take some heat than say little and clients not fully understanding how we operate.

lmm6• 10 years ago

I still just don't see it as nefarious or even weak in terms of security if you set it up correctly.

Sure. With the correct settings it's fine. The problem is their "Flexible SSL" setting, which should be labelled something like "My site is not secure, but I want to lie to my users and pretend that it is." Yes, clients still have to choose the bad option, but CloudFlare should not be offering it to them.

zAy0LfpBZLC8mAC• 10 years ago

That's 69% of my requests that didn't need to hit my website. That's also 57GB in that same period [...] This is particularly important for those paying for bandwidth as it slashes that cost by 84%:

Oh, wow, so, you paid only 2 cents instead of 10 cents for bandwidth? What a huge advantage!

I mean, seriously? Where do you buy your bandwidth that it's a relevant cost factor below 10 TB per month?

modoc• 10 years ago

Firstly not sure you being sarcastic brings much to the table. And many people use enough BW that it's a real cost. If you don't that's fine. But don't assume that it's a non-issue.

noahmbarr• 10 years ago

CloudFlare is the very definition of MitM

alberth• 10 years ago

Off topic: I know there's a lot of HN love for CloudFlare, and I might eve get down voted just for asking ... but how do HN readers gain comfort in using them given that they are essentially a MitM (man-in-the-middle) and can do all kinds of unscrupulous things if they want?

Edit, typo

aianus• 10 years ago

When the alternative is waking up at 3am every night to fight yet another DDoS...

Edit: if there's an alternative to a Cloudflare-type MITM for fighting DDoS in layers two through five I'd love to hear about it

arpa• 10 years ago

Lovely, so when the state of Kazakhstan does a TLS mitm, there's much outrage, when CloudFlare does it, it's suddenly okay. It feels like i've forgotten to take my crazy pills today.

CloudFlare, SSL and unhealthy security absolutism

Related Stories