A closer look at a BGP anomaly in Venezuela

https://blog.cloudflare.com/bgp-route-leak-venezuela/

by ChrisArchitect • 7 months ago

400 191 7 months ago

blog.cloudflare.com

ChrisArchitect• 7 months ago

Related initial aftermath:

There were BGP anomalies during the Venezuela blackout

https://news.ycombinator.com/item?id=46504963

DinakarS• 7 months ago

The depth and coverage that cloudflare has is crazy

sschueller• 7 months ago

Yes and that is a very bad thing for the rest of the world. Time for non-us companies especially ones not doing business in the US to migrate away.

Fokamul• 7 months ago

I wouldn't touch anything US-based with 10m long pole.

At this this, US is basically enemy to EU. Good for us, we will be less dependent on US global oil police.

I hope EU companies will stop manufacturing US airplanes and other things.

supriyo-biswas• 7 months ago

It is perfectly normal for an anycast network operator to have multiple sites from which they make BGP announcements (which is how anycast works in the first place), which gives them multiple vantage points for this sort of analysis.

Other CDN companies can do it too, it's just that they don't work on signalling their engineering focused organization.

bandrami• 7 months ago

This was a while ago but I think it was Akamai that pioneered that. I remember how impressive it was in the early oughts though we take it for granted now.

samsari• 7 months ago

They have a lot of resources, Cloudflare Is Awesome

bayindirh4• 7 months ago

Cloudflare Is Awesome

Until their systems block you for no reason. I recently had a similar issue on a work related site. Fortunately, I was able to reach to the administrator (which is on another country) and had the knowledge to write a report which was useful enough for the said administrator.

And this is for a system which has the same static IP which is not shared with anything for 10ish years.

bauruine• 7 months ago

You don't have to be cloudflare for this kind of analysis you can do it yourself without even needing an ASN using RIPE RIS.

https://www.ripe.net/analyse/internet-measurements/routing-i...

Fiveplus2• 7 months ago

I'm half sleepy but I liked the post. The analysis regarding path prepending really drives the accident theory home. If a state actor were trying to intercept traffic (MITM), the last thing they would do is pad the AS path multiple times because that tells the global routing table, "Don't come this way, I am the long scenic route" lol

This could be a classic fat finger config error, most likely a route map intended to manipulate traffic engineering for their own upstream links that inadvertently leaked widely because of a missing deny-all clause. Neverthless, a good reminder that BGP is still fundamentally a trust based system where a single typo in a config file can cascade globally. Never attribute to malice that which is adequately explained by a missing export filter.

geocar5• 7 months ago

If a state actor were trying to intercept traffic (MITM), the last thing they would do is pad the AS path

That's presumptuous: A state actor would (and could trivially) pad the wrong directions to flow traffic down to pops that are not making new announcements (and thus not-implicated by cloudflare and other "journalistic" efforts).

There's also a lot between fat-fingers and deep-state: I know of some non-state actors who do this sort of thing just to fuck with ad impressions. I also doubt much usable intelligence can be gained from mere route-manipulation thing, but I do know that if it is a fat-finger, every techdude in the area was busy at that time trying to figure it out, and wasn't doing their best work twelve hours later...

most likely a route map intended to manipulate traffic engineering for their own upstream links

...that being said, this does seem plausible: Most smaller multihomed sites I've seen (and a few big ones!) have some kind of adhoc health monitoring/rebalance function that snmp or something and does autoexpect/curl or something-else to the router to run some (probably broken) script, because even if your uplinks are symmetrical, the rest of the Internet isn't, so route-stuffing remains the best way to manipulate ingress traffic.

Never attribute to malice that which is adequately explained by a missing export filter.

As soon as I peer with two big sites that don't peer directly with each-other, they both gotta let me forward announcements unfiltered across them. Once I have a third, I have a legitimate need to manipulate my own ingress.

The problems with the BGP are legion, and not just one thing that prevents BGP and security from sharing time in a sentence.

mlyle7• 7 months ago

A state actor would (and could trivially) pad the wrong directions

This isn't how BGP works. An AS-PATH isn't the path the traffic will follow; it's the path that this overall announcement has allegedly tranversed and is (one of many attributes) used to judge the quality of route. The next hop tells our peer where they should send the data if they like this route.

Putting more things in the AS path makes the route less attractive. Leaking a new route isn't going to magically make some other route become more preferred.

lucideer• 7 months ago

The post mentions a number of times that leaks happen "all the time", but the only comparative data shown related to this is for historical leaks from AS8048.

Does anyone have data on what the general frequency of these leaks is likely to be across the network?

VBprogrammer• 7 months ago

BGP is outside of my skillset, and I'm sure the analysis is fair and accurate. However, had billion dollar US based company Cloudflare detected widespread manipulation of routing tables by the US secret services, I certainly wouldn't trust them to publish it.

sgjohnson2• 6 months ago

I’m pretty confident that the US SIGINT agencies wouldn’t manipulate BGP to redirect traffic somewhere, as such a hijack will ALWAYS leave traces that would be observable by anyone impacted, downstream or upstream.

US SIGINT agencies? They’d just pwn the routers they are interested in. And almost certainly they’ve already done it. Like 10+ years ago.

BGP hijacks are really low-tech and trivial to detect. And competent intelligence agencies don’t do either, unless it comes with enough plausible deniability that it would even be insane to suggest foul play.

I operate a small BGP hobbynet under 2 different AS numbers, and even I keep logs about path changes. Not for any practical purpose, just sheer curiosity.

BGP is a globally distributed and decentralized system. The messages (announcements) propogate virtually across the entire internet. If someone hijacked a route to a prefix that I’ve received, and the path I’ve received is the hijacked one, I’d get that information.

So yes, if that happened, I’d totally expect CloudFlare to publish it, unless they got a NSL. Which they most probably wouldn’t get, as NOTHING about the event would be secret—-it would be out in the open for everyone to see the instant it would happen. There are also tools like https://bgp.tools which operate public route collectors, with the data being publicly available. RIPE has one too.

lowpro• 7 months ago

I’ve seen leaks impact my company directly 4 or 5 times in 4 years, so I would think often enough since we own a /9~ and don’t change our routes too often.

patmorgan237• 7 months ago

MANERS has some reporting here

https://observatory.manrs.org/#/overview

And Cloud flare has some publicly available reporting in radar

https://radar.cloudflare.com/routing

moktonar• 7 months ago

The real question tho is: how would you become the mitm reserving for yourself the benefit of the doubt?

Bengalilol2• 7 months ago

That’s a very new feeling for me. I read the entire post (with no prior knowledge of BGP at all) and I got chills from thinking how deeply intertwined US companies and the US government are.

I know this has always been the case, of course, but now I have lost trust. Whatever the reasons of this "leak" were, I am not accepting any information written in this message (search for the link to another coverage of the incident in the comments).

It is quite weird and quite logical at the same time: this is the end of an era.

embedding-shape4• 7 months ago

It's crazy that it seems like we're just going in loops every decade or so. New people enter tech, mostly focus on their own stuff, after a while, it becomes very clear how "deeply intertwined US companies and the US government are", and these people now lose their trust. Eventually, things been going well for some years, so new people enter the industry, with the same naive outlook, thinking "This couldn't be true of the government we have today" yet eventually, even they realize what's going on. Rinse and repeat every last 3 decades, and that's just what I remember, I'm sure others remember even further.

Bengalilol2• 7 months ago

I am 50 yo and did live through multiple intertwinings. This time though, it is really the end of an era. Trust has been lost.

More positively, what's your opinion on this closer look post from Cloudflare?

potato3732842• 7 months ago

The magic of the system is that the ratio of new entrants who don't aren't yet jaded enough to not be useful idiots vs the rate at which people become jaded vs the rate at which those jaded people leave makes it self sustaining.

schainks• 7 months ago

Companies in country X are often intertwined with their governments? I'm not sure this is really news.

an0malous• 7 months ago

You changed it from “deeply intertwined” to “often intertwined” to make your strawman argument

bayindirh4• 7 months ago

I remember the face of one guy after we chatted about lawful interception over a couple of drinks. He was visibly shaken like he has seen the hell through the door just opened before him.

These kinds of infrastructure is present everywhere, for a very long time. Just because not everyone is talking about the matter doesn't make it non-existent.

For example, in 2003, I saw how Japan monitored their network traffic in real time. It was eye opening for me, too. Technologies like DPI which required beefy servers are now trivial to implement with the right hardware.

This is all I can say.

kachapopopow5• 7 months ago

can confirm this is true - a single rack of servers can now handle terabits of traffic.. in real time with near zero added latency, anti-ddos companies do this as a service.

whirlwin• 7 months ago

If you look closely, you can see the color of the orange Cloudflare logo being slightly adjust to match a particular individual's facial color tone.

dizzant• 7 months ago

This is... hard to follow. You seem to be implying that Cloudflare is covering for USG's failed military op-sec surrounding a malicious BGP leak, and judging that this is such a bad action (on the part of Cloudflare) to undermine your trust, not only in Cloudflare, but in all companies and the US government entirely. I don't think the situation is so dire.

Cloudflare's post boils down to Hanlon's razor: a plausible benign interpretation of the facts is available, so we should give some scrutiny to accusations of malice.

Are there specific relevant facts being omitted in the article, or other factors that diminish Cloudflare's credibility? They're clearly a qualified expert in this space.

Let's assume for the sake of argument that the BGP leaks (all of them from the month of December, in fact) were the result of secret US military intelligence operations. The fact that militaries generally use cyber vulnerabilities to achieve their objectives is not news, and the US military is no exception. Keeping specific exploits secret preserves a valuable advantage over competitor states.

One could argue that Cloudflare's post helps to preserve USG's secrecy. We can't know publicly whether USG solicited the article. But even if we assume so (again assuming malice): Is Cloudflare wrong to oblige? I don't think so, but reasonable people could disagree.

Merely pointing out Hanlon's razor doesn't fundamentally change the facts of the situation. In Cloudflare's expert opinion, the facts don't necessarily implicate USG in the BGP leaks without an assumption of malice. Assuming Cloudflare is malicious without justification is just deeper belief in the conspiracy that they're arguing against.

If Cloudflare is distorting the facts, we should believe (rightly) that they're malicious. But I don't see any evidence of it.

EDIT: Clarity tweaks.

patmorgan237• 7 months ago

Respectfully your comment sounds like paranoid thinking.

The section of the article pointing out the AS prepending makes it really clear the route leak is a nothing Burger.

It's incredibly unlikely this leak change how any traffic was flowing, and is more indicative of a network operator with an understaffed/underskilled team. Furry evidence is that a similar leak has been appearing on and off for several weeks.

That's not to say the US government can't, doesn't or didn't use the Internet to spy, it's just that this isn't evidence of it.

Relevant section below: > Many of the leaked routes were also heavily prepended with AS8048, meaning it would have been potentially less attractive for routing when received by other networks. Prepending is the padding of an AS more than one time in an outbound advertisement by a customer or peer, to attempt to switch traffic away from a particular circuit to another. For example, many of the paths during the leak by AS8048 looked like this: “52320,8048,8048,8048,8048,8048,8048,8048,8048,8048,23520,1299,269832,21980”.

You can see that AS8048 has sent their AS multiple times in an advertisement to AS52320, because by means of BGP loop prevention the path would never actually travel in and out of AS8048 multiple times in a row. A non-prepended path would look like this: “52320,8048,23520,1299,269832,21980”.
If AS8048 was intentionally trying to become a man-in-the-middle (MITM) for traffic, why would they make the BGP advertisement less attractive instead of more attractive? Also, why leak prefixes to try and MITM traffic when you’re already a provider for the downstream AS anyway? That wouldn’t make much sense.

azalemeth2• 7 months ago

It might be worth linking this document from the Snowden leaks: https://christopher-parsons.com/wp-content/uploads/2023/01/n...

"NSA Network Shaping 101". Big descriptions of ASINs, and layer 3 shaping. Written in 2007.

immibis• 7 months ago

Funny to see even the NSA makes the mistake of calling a network an ASN (maybe because it's their name backwards), which is like saying I deposited money in my IBAN, or my neighbour lives in the string "123 Main Street", or Hacker News is an interesting DNS name full of great content.

b00ty4breakfast• 7 months ago

Hi, it's me, The Language Fairy

https://en.wikipedia.org/wiki/Metonymy

formerly_proven• 7 months ago

I don't see the relation to BGP anomalies, since this "layer 3 shaping" is basically just "if you send traffic to the IP of an AS router, it probably goes over the link of that IP". None of this would help NSA "shape" arbitrary traffic onto links they are able to tap. (I'm really not sure what exactly the point of this is, the slides talk about exfil a lot, it would seem to me like some random device sending traffic to a router is more suspicious, because normal traffic never targets routers, than hitting an actual server somewhere but idk)

azalemeth2• 7 months ago

In en-us education "101" is often used to refer to an introductory course in a particular topic. My inference from the fact that this _educational_ slide is called "101" is that this is a basic example of core knowledge that people in this area of work are expected to have. It therefore stands to reason that there exists a "102" or "103" course that expands upon it, as well as material going far beyond "the syllabus".

The NSA and thirteen eyes generally have detailed traffic logging capability at core internet exchanges around the world. It is reasonable to think that a good way of exfiltrating data would be by having something like an ICMP or maybe even TTL based covert channel, such that there is no chance that the sent data is ever received by the recipient. I am just speculating – but that's why I thought this was interesting.

potato-peeler3• 7 months ago

Slightly off topic, but if I want to understand the concepts discussed in this article, what all topics should I learn? Is this a good starting place or enough to understand everything in this article - https://beej.us/guide/bgnet/

f1shy6• 7 months ago

Because of the formulation of the question, I assume (please don't be irritated, if wrong) that you have very little knowledge of networking. In that case, it won't harm the Beej guide, but probably not the best point to start. The article discuses BGP protocol, which is totally absent in the linked guide. You may write literally millions of networking applications without need to know anything about BGP. Only if you are working in the backbone of the internet, you will encounter BGP, not even in big private networks. It will be a long way to really start from 0 up to BGP.

123pie123• 7 months ago

I was trying to think of an analogy

Kind of like wanting to learn how a car engine works and asking about fleet management in trucks/ lorries.

BGP is one of things I've learnt then forget the next day (multiple times)

hahahahhaah• 7 months ago

What about this? https://www.cloudflare.com/en-au/learning/security/glossary/...

The goal for me is conceptual understanding, not to build an ISP. And network knowledge is light... just enough to be a cloud monkey

potato-peeler3• 7 months ago

I have to start from somewhere. I don’t mind the grind. I am fascinated by networking. Will begin with beej guide, and go from there.

mbix77• 7 months ago

Scary that so much of the basic internet infrastructure is being managed by US companies. Maybe now the rest of the world will change and become more independent. We should have learnt our lesson long ago though.

jaza• 7 months ago

Considering that the internet was invented and built from scratch by the US military, US universities, and US companies, why are you surprised? And who do you suggest could or should manage much of the internet backbone, if not them?

sgjohnson2• 7 months ago

What do you mean? The internet is virtually entirely decentralized. There is no one central BGP router.

kevin_thibedeau• 7 months ago

The rest of the world exports its talent to the US because they don't pay enough. There's no reason why the EU couldn't have made an Akamai or Cloudflare clone decades ago save for the money.

_heimdall• 7 months ago

The comments here surprise me a bit. The common thread so far seems to be a general fear of US based companies, but how is that relates to the article?

Cloudflare's post is pretty boring here in that regard. They dig into how BGP works and propose that similar leaks seem common for the Venezuelan ISP in question.

Sure they could be wrong or even actively hiding the truth of what happened here, but the article mentions nothing of Cloudflare being involved in the action and they're describing a networking standard by pointing to publicly available BGP log data.

What am I missing here that everyone else seemed to zero in on?

ffsm8• 7 months ago

Probably because most people only read headlines (and maybe 3 paragraphs) combined with the fact that the US has a long history of doing what people are condemning them for, even if this particular instance probably wasn't a case of such behavior. Especially considering how the general sentiment towards the US has gotten bitter with constant threads of invasion of Denmark and Canada by their government.

Or it's just Russian and China socket accounts? Who knows...

xocnad• 7 months ago

I share your view - how does this article imply US companies and/or government involvement? If there were such involvement what aspect of BGP gives the US entities more ability to carry this out vs other nefarious actors? I ask this sincerely knowing almost nothing about BGP and wanting to learn...

jeroenhd2• 7 months ago

You may have missed https://news.ycombinator.com/item?id=46504963 a few days ago where this same anomaly was discussed and American government involvement was directly implied by the article.

jeroenhd2• 7 months ago

There was another post a few days ago that suggested a connection between the American invasion of Venezuela and the BGP anomaly: https://loworbitsecurity.com/radar/radar16/

Combine that with the news of Trump publicly admitting that the US is willing to take military action to bring other countries in line, even against their own allies: https://edition.cnn.com/2026/01/06/politics/us-options-green...

Personally, I don't think the Americans would bother hide their attack and make it look like an accident under the current regime. Trump would announce the CIA/NSA/FBI/whatever did the Greatest Attack, and Amazing Attack, to Completely Control and break the Weak Government of Venezuela to Rescue Their Oil. I'll believe the "it was just a misconfiguration" explanation for now.

I think it only makes sense that people start fearing the influence of American companies given the current developments. When America is in the news, it's either threatening someone, pulling out of cooperative efforts, or delivering on a previous threat. That's bound to derail discussions whenever American companies are involved and it'll only get worse with the way things are developing.

k12sosse• 7 months ago

That's what I find interesting about the billionaire elite standing behind el presidente, like, sooner or later he'll be gone and you guys -and your companies- won't. There's been no more compelling argument to actually overtax the rich to give to the masses than the last 13 months.

Eat the rich. History won't forget.

appreciatorBus• 7 months ago

I think it’s just bog standard, “USA bad, not USA good” thinking.

zug_zug• 7 months ago

I don't think this article provides any evidence of anything to be scared of.

That said, based on what we know already, there is no reason to take everything is this article at face value necessarily.

Firstly, if anybody isn't aware of the history of Stuxnet, it's worth reading, because otherwise you'd underestimate the government's ability to use 0-days by an order of magnitude (we're talking full custom-written multi-month hacking projects with root-kits and custom fake drivers delivered successfully to an airgapped system, source wikipedia). Also worth learning about Dual EC DRBG debacle.

Secondly am immediate friend of mine worked at a FANG company that routinely sent a firehose of all sorts of things matching all sorts of filters directly to governments. In fact many ISPS have back-doors built in and that's not really disputed (wikipedia: room641A).

So the question to ask yourself is -- if this was a deliberate interaction that cloudfare was required to participate in via a warrant, would they legally even be allowed to publish a blog post that contradicted this?

So I think that is probably the default attitude of skepticism you are seeing, which in my opinion is a good default. Plus the primary claim of this article "Look it wasn't 1 routing issue, it's been happening for even longer! Therefore nothing to look at here!" seems really weak.

nucleardog• 7 months ago

So the question to ask yourself is -- if this was a deliberate interaction that cloudfare was required to participate in via a warrant, would they legally even be allowed to publish a blog post that contradicted this?

So you're proposing they could be in a situation where they can either:

1. Publish an untruthful blog post, relying on public data available from multiple parties, trying to somehow explain it all while avoiding talking about their involvement in a way that would get them in PR, legal or political hot water; or

2. Publish nothing.

And they chose #1?

The only way #1 makes any sense at all is if some greater consequence to not publishing was put in place. But that would be more something like "the US gov essentially forced Cloudflare to write this" than "Cloudflare was part of this".

Unless they were part of this, _and_ the government forced them to write a post saying they're _not_ part of it and...

For my money: this is something in the news making it a good marketing opportunity which is ultimately what the blog is--trying to market Cloudflare and the brand to technical crowds.

zozbot234• 7 months ago

"Look it wasn't 1 routing issue, it's been happening for even longer! Therefore nothing to look at here!" seems really weak.

It's actually really strong since it implies that there's no real time-based correlation with the recent action in Caracas. Especially as the purported correlation was rather weak to begin with.

halJordan• 7 months ago

Ah yes, and we're back into "but my buddy told me " if you have to say that then your story just isn't worth saying or hearing and you should reconsider how impervious you are to conspiratorial thinking

HeyLaughingBoy2• 7 months ago

It's even older than Stuxnet, but either Dish Network (Echostar) or DirectTV did something similar in the early 2000's/late 90's.

They were having a lot of trouble with pirate receivers, so they added small chunks of code to normal device updates and this went on over a period of weeks/months. On the final update, it stitched all those bits of code together and every receiver that wasn't a legitimate one displayed the message "GAME OVER" on the screen and stopped working.

Obvs it was a long time ago so forgive me if I get some details wrong.

Aloisius• 7 months ago

I looked at this a couple days ago and my thoughts were basically the same as Cloudflare's. It looks like a misconfiguration - one that's easy to make and isn't terribly uncommon. I can't rule out it wasn't an attack, but absent some other evidence, I don't see any reason to believe it was one.

That said, looking at their Cloudflare radar page now for AS8048, I don't recall there being any other BGP route leaks listed there for December from AS8048 and I definitely don't recall there being any BGP origin hijacks listed. The latter is something rather different from a route leak - that looks like someone blackholing some of CANTV's IPs.

I don't think I somehow just missed that since I definitely looked at CANTV's historical behavior to see if anything they did was unusual and that would have been one of the first things I checked, but perhaps they updated radar with data from other collectors or re-ran anomaly detection on historical data.

caycep• 7 months ago

I mean, it's the context around the article...based on recent events...

keybored• 7 months ago

As news unfolds surrounding the U.S. capture and arrest of Venezuelan leader Nicolás Maduro, ... It is also noteworthy that these leak events begin over twelve hours prior to the U.S. military strikes in Venezuela. ...

This is how I imagine Russian companies in Russia write about the Russian war on Ukraine.

hk1337• 7 months ago

I probably glossed over it in all the posts but…

What is a BGP?

literalAardvark2• 7 months ago

It's the protocol used by carriers to route traffic globally. (Make automated decisions about which core router should receive the traffic coming out of your AS ("network", kinda) )

It's entirely detached from anything else so you're pretty unlikely to have heard of it. In that way it's similar to SS7.

patmorgan237• 7 months ago

The Internet is a network of Networks, BGP (Boarder Gateway Protocol) is how routers tell other routers what networks they are connected to. This allows you to connect to any device on the Internet, even if you have to go through 5 different networks to get there.

__MatrixMan__• 7 months ago

I once was half way through a road trip when google maps routed me off the highway, through a walmart parking lot, and onto another highway.

I assumed it was a badly performing algorithm. But if it had instead routed me through a McDonalds drive through, I'd have assumed it was foul play.

I think the article makes a decent case that this was the former and not the latter, though it would be interesting to see route leaks visualized on a map over time. Too many odd coincidences could sway me the other way.

esseph9• 7 months ago

The only reason BGP route leaks aren't more common is the filtering of other ISPs. It's pretty easy to make a mistake you don't intend to.

patmorgan237• 7 months ago

Truth. There's been a lot of work over the past 10-15 years to strap on best practices and validation to make these kinds of incidents less common or impossible. The article even talks about several upcoming changes/standards at the end.

8organicbits4• 7 months ago

This is a good opportunity to assess what parts of your own online activity could be impacted by an attacker in the middle (assisted by a BGP leak or otherwise) and, if you're a service provider, how you can protect your customers.

At first pass you probably use HTTPS/TLS for the web, and you know that you shouldn't click through invalid certificate warnings. So the web, tentatively, looks pretty safe.

Email jumps out as vulnerable to eavesdropping, as we largely use opportunistic encryption when transferring messages between mail servers and an on-network-path attacker can use STARTTLS stripping or similar techniques. Most mail servers happily send using cleartext or without validating the TLS certificate. Check that you and your counter-parties are using DNSSEC+DANE, or MTA-STS to ensure that authenticated encryption is always used. Adoption is still quite low, but it's a great time to get started. Watch out for transactional email, like password reset messages, which virtually never validate encryption in transit (https://alexsci.com/blog/is-email-confidential-in-transit-ye... ; instead use multi-factor encryption).

TLS certificates themselves are at risk, unfortunately. An attacker who controls the network in-and-out of your DNS servers can issue domain-verified certificates for your domain; even removing protections like CAA records. DNSSEC is the classic solution here, although using a geographically distributed DNS provider should also work (see multi-perspective validation). Certificate transparency log monitoring should detect any attacker-issued certificates (a review of certificates issued for .ve domains would be interesting).

Ideally, we should build an internet where we don't need to trust the network layer. A BGP route leak would be a performance/availability concern only. We're not there yet, but now is a great time to take the next step in that direction.

tptacek4• 6 months ago

Attackers hijacking domains to get certificates issued are generally hijacking registrar accounts, which DNSSEC doesn't help with, which is probably one of the many reasons DNSSEC is so rarely deployed.

8organicbits4• 6 months ago

We know, you've told us many times. But that's not the context of the thread.

David_Osipov• 6 months ago

There is a nuance here for Cloudflare users that makes this problematic.

While analyzing the Jabber.ru incident (which used this exact BGP->TLS vector), I discovered that Cloudflare's "Universal SSL" actively injects permissive CAA records that override user-defined restrictions.

If a user sets a strict accounturi CAA record (RFC 8657) to lock issuance to their specific account—specifically to prevent BGP hijackers from getting a cert—Cloudflare's system automatically appends a wildcard record alongside it to keep their automation working. Because CAs accept any valid record, this effectively nullifies the protection.

It creates a situation where you think you have mitigated the BGP risk via DNS, but the vendor has silently reopened the door.

I wrote up the technical analysis of this override behavior here: https://david-osipov.vision/en/blog/cybersecurity/cloudflare...

tedggh• 7 months ago

People are so wanting to believe there was an advanced cyber attack to Venezuela’s grid and ISPs that they forget this is a country that hasn’t updated its infrastructure in more than two decades while also not providing any significant maintenance. Most of the “new” technology deployed at the state and federal level comes from corrupt foreign and domestic “suitcase” companies that charged a lot of money to deliver poorly designed systems often even lacking the as-sold equipment. So Venezuela isn’t precisely the most formidable adversary when it comes to cybersecurity.

esseph9• 7 months ago

1. There was a cyber attack on the Venezuelan power grid. This disrupted comms coming into the attack and made it much harder to coordinate a response.

2. It was not in any way related to this BGP, of which, as someone in networking, looks like a simple and fairly common mistake. It wouldn't really buy them anything anyway, the breach happened 6+ months before.

patmorgan237• 7 months ago

Yeah, the US government has advanced cyberwar fare capabilities, but this BGP anomaly is not a result of, or evidence of that.

rafaelm• 7 months ago

This is CANTV they are talking about. This is the company I requested a new phone line from and it took 9.5 years to get it installed.

After waiting for 3 years, I gave up ended up paying one of their technicians I randomly found working in the street. He gave me a phone line that apparently used to belong to a taxi company, judging by all the wrong number calls I got. All that just to get 4mbps DSL service in 2019.

Last year, out of nowhere, I finally got a call from the company saying they were ready to install it.

Thankfully, a bunch of companies appeared out of nowhere (a lot of them with links to people in the govt, surprise) in 2020 and we got fiber.

Oh and a couple of years ago, my parents "lost" their phone line and have been without POTS ever since. Maybe it's karma for me paying for a phone line all those years ago...

big-and-small• 7 months ago

People also wanting to believe there was even a need for sophisticated cybersecurity attack in the first place. In a country where average household income is around $230 per month. In much wealthier country like Russia you can literally buy dump of all possible leaked data on any person for $1 and for $100 you can get all information government have about a person including camera and mobile phone tracking, etc.

And Venezuela is very very corrupt country. No cyberattack needed when you can pay $10,000 - $100,000 for a dude to pull the lever or to forget to pull the lever and literally 99.99% of people in a country do it.

Though these theories are easy to explain because people in mostly US community like HN have no understanding of what total corruption look like in a shit hole countries.

2OEH8eoCRo0• 7 months ago

There have been BGP shenanigans before.

https://arstechnica.com/information-technology/2018/11/major...

Google goes down after major BGP mishap routes traffic through China

anticristi• 7 months ago

Whether the claims are true or not, this was a very entertaining BGP refresher. It made me wonder: 15+ years ago, I was network engineer and we used quite a bit of "BGP community magic" to get the routing outcomes we wanted.

If BGP only really needed to represent three types of peers (provider, customer, actual peer), wouldn't BGP configuration and perhaps even BGP be massively simplified?

sophacles• 7 months ago

It would be massively simplified.

Simple isn't always good.

By analogy: i could massively simplify google maps direction algorithm by getting rid of all that annoying and unnecessary traffic information, annoyingly complex labels about speed limits and lane count, and all the data points about stop signs, traffic lights, and so on. Its just a path-finding algorithm after all and all that extra info just makes for more computation and complexity. Who cares if it mean all the traffic for a major metro goes across a 1-lane bridge and leaves all the other roads empty.... its the shortest path, what could go wrong?

hoten• 7 months ago

8 min read

hah.

googlehater• 7 months ago

As someone who knows nothing about networking, this felt really easy to follow. Thanks for sharing!

VanTheBrand• 6 months ago

This article existing at all is a bit suspicious IMO

qalmakka• 6 months ago

Telecom Italia Sparkle

A corollary to the Hanlon's razor: prefer assuming incompetence over malice if there's Telecom Italia involved in any shape or form

A closer look at a BGP anomaly in Venezuela

Related Stories