Cloudflare automatically fixes Polyfill.io for free sites

https://blog.cloudflare.com/automatically-replacing-polyfill-io-links-with-cloudflares-mirror-for-a-safer-internet

by kylehotchkiss • 2 years ago

152 80 2 years ago

blog.cloudflare.com

skilled• 2 years ago

We have taken the exceptional step of using our ability to modify HTML on the fly to replace references to the polyfill.io CDN in our customers’ websites with links to our own, safe, mirror created back in February.

I get that they want to do a good thing, but is this something you agree to when you sign up as a Cloudflare customer? If so, that’s kind of crazy.

edit: I am talking about both free and paid users. A toggle does not discount my question whatsoever, especially if this is on by default for free users. I am asking specifically about terms of service.

JohnMakin2• 2 years ago

I get that they want to do a good thing, but is this something you agree to when you sign up as a Cloudflare customer? If so, that’s kind of crazy.

The article says that if you are a paying customer this is off by default.

Mathnerd314• 2 years ago

True, but they say "we decided". That implies they could also have decided to turn it on immediately. And they did for the log4j and Shellshock issues.

datadrivenangel• 2 years ago

A manual toggle is required for paid customers. Only free customers were automatically turned on.

It does show that you have to trust cloudflare, which seems like a safe bet given their desire to keep the internet secure.

lolinder5• 2 years ago

It does show that you have to trust cloudflare, which seems like a safe bet given their desire to keep the internet secure.

This is assuming that Cloudflare's interests will now and always align with your own interests, and that their desire to "keep the internet secure" will never lead to them actually screwing over you and your customers.

It's a lot like many classic AI Sci-Fi stories. If their mission ever comes into conflict with your real interests, a mission statement that sounds perfect in theory can suddenly become very very dangerous.

scblock• 2 years ago

I don't trust cloudflare at all. Why would I trust someone trying to monopolize all internet traffic?

fblp• 2 years ago

People are often using cloud flare to improve the security of their site. I think in this case if they failed to act when they could that would be worse.

hanniabu• 2 years ago

Users of these sites they fixed the polyfill for didn't agree to getting infected with malware.

xnorswap2• 2 years ago

Today cloudflare are re-writing payloads to remove malware.

Some grim future who knows if cloudflare will be the ones under new owners re-writing payloads to serve adverts (or worse).

edit: I think my comment is being misunderstood, I'm not saying this will happen, there's just a neat symmetry between exploit and mitigation.

janice1999• 2 years ago

They're a US based man-in-the-middle for large parts of the internet at the mercy of the America's secretive FISA Court. There is no way they don't have a Room 641A.

https://en.wikipedia.org/wiki/Room_641A

xnorswap2• 2 years ago

They're US based now, but there's no guarantee it'll stay that way, if a buyer were to come along with enough money. I guess if your theory is correct then pressure would be applied to make sure any foreign buyout didn't happen.

In any case it's extremely unlikely to happen any time soon, but who knows what could happen 50 years down the line, especially if they were to lose market share or the internet fades from relevance.

I'm not accusing cloudflare of anything malicious, I want to be clear. But this polyfill wasn't originally malicious either, it was just eventually bought by a malicious actor.

My original comment was just commentary on the symmetry of the exploit and the mitigation, they're essentially the same vector.

0x1ch• 2 years ago

In other words, Cloudflare is man in the middling their customers and hijacking delivery to customer CDNs? This time it was beneficial...

jen729w• 2 years ago

…and next time when it’s harmful they lose their entire business.

Doesn’t seem to make a lot of sense, does it?

IggleSniggle• 2 years ago

Only if the harm becomes known.

realusername• 2 years ago

What doesn't make sense for a business might for a government agency. Cloudflare way too juicy of a target to be left alone.

asddubs• 2 years ago

cloudflares business model is literally to mitm their customers. they incidentally also get access to unencrypted versions of all https traffic to cloudfare enabled sites

curiousfab• 2 years ago

Setting a dangerous precedent, especially doing this by default (no opt-in) needed.

But then again, if the people who carelessly include 3rd party dependencies (i.e. playing with fire) are those who use CF... they probably won't object to it :-)

hipadev23• 2 years ago

Your implied correlation between sloppy devs and customers of cloudflare is a bit rich.

politelemon• 2 years ago

I totally get the intention behind this and the final outcome is definitely a safer internet. It's also somewhat justified considering the author has mentioned they never controlled its domain, yet the library has been distributed through that domain, correct? This is a reflection of the extremely poor security practices in the web development world.

At the same time, there is a wrongness in Cloudflare being able to overwrite content and changing the 'truth'. I'm like that larry david gif, I just don't know how to process this.

One more thing to note, if you go to the polyfill repo, they've also mentioned they're using Cloudflare to distribute the library.

https://github.com/polyfillpolyfill/polyfill-service/commit/...

madeofpalk3• 2 years ago

At the same time, there is a wrongness in Cloudflare being able to overwrite content and changing the 'truth'.

This is one of the many features of Clouldflare though, that you can enable additional features that modify your website in various ways - whether it's image resizing or analytics or security scanning.

If you don't trust Cloudflare to deliver your website, then you shouldn't use Cloudflare.

dclowd99012• 2 years ago

I think GP’s point was more about the truth for the consumer of the site. As a user, I trust a site to deliver the content it intends to deliver. Of course this is just a trust by proxy with Cloudflare because you’re trusting the site’s judgment on who to trust. As a user of the site, it makes me just a little bit more uncomfortable knowing there’s another layer of indirect control over the content coming in.

Of course this is all just philosophical anyway. We’ve not even touched on external module usage.

ec109685• 2 years ago

"Contrary to what is stated on the polyfill.io website, Cloudflare has never recommended the polyfill.io service or authorized their use of Cloudflare’s name on their website. We have asked them to remove the false statement and they have, so far, ignored our requests. This is yet another warning sign that they cannot be trusted."

austhrow743• 2 years ago

I believe that’s what pushed them over the edge on booting stormfront.

Probably a good idea to not claim affiliation with cloudflare without permission.

bastawhiz4• 2 years ago

changing the 'truth'.

This is sort of exactly what cloudflare does, though. They host your content on their servers. They're serving a mirror of the original files from polyfill.io and browsers are getting the same original bytes. If it's a fact that pulling content from polyfill.io is a security issue, why wouldn't you want them to do this? The "truth" as you describe is a third party that you don't trust (who acquired a service you use from the original provider). The status quo is inherently bad. "We served your page for you, but without the injected code and with full compatibility" feels like an awfully nice thing for Cloudflare to do that they simply didn't need to do.

tlhunter• 2 years ago

uBlock origin also blocks polyfill.io. Is that also wrong?

dzikimarian• 2 years ago

Client is able to see and control what uBlock does. Each one separately, as they see fit. Quite a difference, isn't it?

mdasen• 2 years ago

For me, this is why it makes sense: customers on Cloudflare signup (at least in part) for Cloudflare to protect them against attacks.

Cloudflare is all about changing the "truth". When your site is behind Cloudflare, they block many requests to your site. The lock in the browser can be lies. Cloudflare is decrypting that content - and if the site owner hasn't setup TLS between Cloudflare and the backend, it's being re-transmitted over the internet unencrypted. On paid plans, Cloudflare will compress images and swap in their version. Cloudflare will compress an uncompressed response before returning it. Cloudflare will take the HTML returned by your backend and obfuscate any email addresses in that HTML before sending it along to the browser.

Your server returns a page with evil-polyfill/bad.js and Cloudflare inspects the HTML and rewrites it to say good-polyfill/good.js

You might not want this behavior and you can shut it off in Cloudflare, but it seems like a reasonable default given that customers have signed up for a product meant to protect them against attacks. Cloudflare has never been about passing back the raw HTML it receives from the backend or passing along the raw requests it receives from browsers.

ozim2• 2 years ago

My question is when does cloudflare starts to inject ads into my content?

Is it “shots fired” situation?

Because you know bullies and other bad people start testing ground to see what they can get away with. That is why you slap them hard and quick on the first attempt right away so they see that they cannot fool around.

So I am asking can it be we have to say - well yeah technically they are right - but stop right there doing that and never do it again!

They could make terms that they don’t serve vulnerable things from their cache and it is up for the customer to update or fix it - but they shouldn’t overwrite stuff, period.

sva_• 2 years ago

Interesting comment detailing when the polyfill.io domain actually delivers the malicious code: https://github.com/polyfillpolyfill/polyfill-service/issues/...

jimbobthrowawy• 2 years ago

Odd that github archived the repo claiming it contains malware. Was any of the malicious code even in the repo? I would guess it's only on the server they're serving the polyfill from, since it has very specific conditions for triggering.

jraph• 2 years ago

Here's how I process it:

Cloudflare shouldn't have such a control over a part so big of the internet.

But since they do, they are right to use this control against malware.

aaomidi• 2 years ago

I’d go as far as saying at this point they’re expected to.

Sephr4• 2 years ago

The owner of the domain was Jake Champion. He transferred ownership to Funnull.

skybrian• 2 years ago

Cloudflare is rewriting the link on websites hosted on Cloudflare. I think it can be argued that those websites have in some sense opted in to some limited rewrites by Cloudflare? (Presumably they wouldn't be doing it unless their terms allowed it.)

sneak• 2 years ago

Nobody tell him about the “Google Safe Browsing” censorship bloom filter list used by Firefox, Chrome, AND Safari.

Relevant today as ever given that the Supreme Court ruled that the 1A challenge to the executive branch asking big tech to censor doesn’t have standing to be ruled on.

Google has a big red button that can shut down any webpage on the internet for 99.9%+ of the web browsing public. There’s no bypass button in the UI like a TLS failure.

paxys2• 2 years ago

Cloudflare can't just do this at will. This only applies to websites that have explicitly trusted Cloudflare with their TLS cert in order to protect themselves from cases exactly like this one.

rasz• 2 years ago

author has mentioned they never controlled its domain

Didnt his friend from work own and sell it?

jgalt212• 2 years ago

I understand all the negativity, but to me CF has provided a least bad immediate solution.

ubanholzer• 2 years ago

CloudFlare is "fixing" all websites using their proxy automatically, meanwhile polyfill.io is using CloudFlare as a CDN. That's funny.

ChrisArchitect• 2 years ago

More discussion on the attack: https://news.ycombinator.com/item?id=40791829

stavros2• 2 years ago

At this point, shouldn't all includes use SRI?

jsheard• 2 years ago

Not in this case, for better or worse. The point of polyfill.io is they tailor the delivered Javascript based on the deficiencies of the user agent making the request, so it's not compatible with SRI.

stavros2• 2 years ago

Ah, wow. I didn't realize this was a thing, very convenient target for a supply chain attack, then.

bastawhiz4• 2 years ago

We have not outright blocked the domain through any of the mechanisms we have because we are concerned it could cause widespread web outages given how broadly polyfill.io is used with some estimates indicating usage on nearly 4% of all websites.

Frankly I find this to be a weak argument. "We'll continue serving malware because broken websites are worse than infected ones" is not a good argument. I'm a cloudflare customer and I think this is a very bad look, even if it does mean websites keep "working."

slowmovintarget• 2 years ago

uBlock origin blocks Polyfill.io https://news.ycombinator.com/item?id=40802393

Polyfill supply chain attack: https://news.ycombinator.com/item?id=40791829

Sephr4• 2 years ago

I'm curious as to how Fastly can defend their choice to keep Jake Champion in their employ after the damage he caused by transferring polyfill.io to Funnull[1], triggering this entire incident.

The warning signs should have been obvious. There's no profit to be had in this kind of free CDN.

1. https://x.com/JFSIII/status/1761385341951361182

pciexpgpu• 2 years ago

Why assume bad faith and bring pitch forks out? The actual scammer needs to be out - not some person who maintained it for free.

Nobody paid this OSS person - only when there is a problem do we ’accuse’ OSS maintainers not when they were actually doing their job for free.

Zafira• 2 years ago

I think this touches on an interesting question. What obligation do free or open source project maintainers have?

Even if a maintainer slaps on a, “I do what I want with this project. I am not responsible for any damages. There is no support” disclaimer, I am not sure that necessarily removes some social responsibilities.

SahAssar2• 2 years ago

If you own the domain for a service used for free by many many websites and have encouraged people to use that service for free I think there is some responsibility to not transfer that to a bad actor.

If he simply made the DNS not resolve to a server anymore I'd be fine with it (and if giving people a warning a few months in advance would be great) but this is not inaction, this is selling trust. It's reasonable to have less than zero trust in anyone that would willingly sell my trust to an unknown third party for profit in this manner.

I think that people should have never used this service or used it with subresource integrity (which by design is not possible in this case), but that's not how it was pitched so now the owner has some responsibility if they want to maintain dignity and trust.

Sephr4• 2 years ago

I never claimed bad faith. Lack of professional competency can also cause poor decisions. As it stands, Jake hasn't even admitted to his mistake yet.

This isn't just about an open source project. It's about an online service. The owner could have simply shut it down instead of allowing it to be acquired.

lolinder5• 2 years ago

Comments like this are part of why I'll never run a free service or open source project that people might grow to depend on. I'm not interested in taking on the risk that a mistake I make in my volunteer time causes an internet mob to call for the termination of my employment.

Sephr4• 2 years ago

I imagine that your example scenario would involve talking to your users to keep their trust and owning up to mistakes if you were running a free service whose security ended up being affected by those mistakes.

sakisv• 2 years ago

While I appreciate their intention and their transparency on what they're doing, I'll echo other comments here in that I, too, feel a bit uneasy that a 3rd party is rewriting my content.

The good news is that a (strict) CSP can help with that - and they do mention it in their blog post that they don't rewrite anything if there's a CSP header.

It's also worth noting that a (strict) CSP also prevents them from injecting their analytics JS from being injected to your site.

omoikane• 2 years ago

This post includes a link to an earlier blog post on "reduce your supply chain risk" in 2024-02-29:

https://blog.cloudflare.com/polyfill-io-now-available-on-cdn...

In the earlier thread, there was a link to triblondon's post on 2024-02-25 where he urges users to remove that dependency:

https://x.com/triblondon/status/1761852117579427975

Unfortunately, neither of these early warnings seem to have gotten much attention. I wonder if there is some news site or service that would make suspicious ownership transfers of this sort more noticeable? Or maybe this type of supply chain changes actually happen all the time and we just got used to them?

ilaksh• 2 years ago

One obvious issue to me is that we have over-centralized the internet on Cloudflare.

That's great they are fixing it on free sites though.

dclowd99012• 2 years ago

Yeah, I agree, but also, what is the way out of it? By virtue of their value proposition, the only natural progression for a CDN is maximum scale, isn’t it?

danslo• 2 years ago

It doesn't matter anymore, Namecheap has taken down polyfill[.]io: https://x.com/malwrhunterteam/status/1806074377383121148

eastdakota• 2 years ago

It matters still in so far as if you turn this feature on your site will continue to work as expected even with Polyfill[.]io offline.

tristanperry• 2 years ago

Hi Matthew, Cloudflare just charged my card $235 with no prior warning (for a service I intended to cancel when I received the renewal reminder). I did not receive any renewal reminders, no advance notice of payment, no invoice. Nothing.

Is this normal operating behaviour for Cloudflare? This seems like very very bad anti-consumer behaviour?

X-Istence• 2 years ago

Looks like it is down at the .io registry level, I am not getting any NS records for polyfill.io

Cloudflare automatically fixes Polyfill.io for free sites

Related Stories