The internet is held together with spit and baling wire

https://krebsonsecurity.com/2021/11/the-internet-is-held-together-with-spit-baling-wire/

by picture • 5 years ago

374 162 5 years ago

O'Brien: I'm afraid to touch anything. It's all cross-circuited and patched together - I can't make head nor tails of it. Bashir: Sounds like one of your repair jobs.

Seriously though, it seems like every form of infrastructure we rely on is held together in such a fragile manner. I hate hate to think of the chaos should there be a major Internet and physical infra failure in close proximity, time-wise.

hypertele-Xii• 5 years ago

We have a pretty good track record keeping it running though; The Internet has never gone down!

I'd be far more concerned with agricultural logistics, though we've never starved to extinction, either.

Perhaps we've reached a point of positive no return: We can no longer cease to exist!

adventured• 5 years ago

In this context it's a very important point that there isn't one Internet and there isn't one "we." Unless you strictly mean the human race has yet to go extinct due to starvation.

A lot of starvation events have of course occurred across many different nations, peoples, civilizations. Europe, as one example, was numerous times ravaged by extreme starvation events that collectively killed millions of people across the 19th and 20th centuries. I think your agriculture concerns are well placed.

The various Internets have gone down routinely for all sorts of reasons.

cgh2• 5 years ago

There’s a strong probabilistic argument that says otherwise: https://en.wikipedia.org/wiki/Doomsday_argument

Note that the argument is not dependent on any sort of cause, such as climate change or whatever. It is entirely probabilistic.

zwkrt• 5 years ago

Unless you exist in a state of wild over abundance and are very conscientious, most working infrastructure is always at a state of almost-disrepair. If I operate a machine shop and I have 4 partially-stripped screwdrivers, I might get a new one, but inevitably I will use it until it is as stripped as all the rest, and I don’t dare throw away a tool that is currently even in partially working condition since I might need it later if another driver breaks. This is true of every single thing in my shop, and the end result is a system that is robust to absolute failure but prone to constantly needing to be patched up.

As far as I can tell this is a truism across all fields: farming, skilled trades, build systems, transportation infrastructure, housing…

ragingrobot2• 5 years ago

Imagine you use that last screwdriver, and a job needs to get done, and your car won't start so you can't get a new one at a hardware store outside of walking or cycling distance. And now you're unable to repair a vital system.

As I was writing that, I was just thinking of the recent Surfside collapse, what would would have happened if the regions data networks had gone down simultaneously (by chance). A major event two decades ago, cellular networks were overwhelmed and calls could not be made. I dare say we're more reliant on those networks today, as well as the Internet.

Not that I would expect it to happen, but it was just a thought.

ip262• 5 years ago

Are your mills in a similar state? I’m no machinist, but I would think it hard to stay in business when the means of production are constantly down for unplanned service.

Screwdrivers are more of a consumable than a capital good.

joconde• 5 years ago

Software too, unless every small piece of it has a dedicated full-time maintainer. If there's an infinite stream of tasks incoming, why go improve something that fits the current use well?

9999000009993• 5 years ago

You could also break something with your dull tools.

This is why you can't just drive a car until it breaks, you get it checked out.

Then again, cars are a private good. When it's your property vs our property you have more of an incentive to take care of it.

AceyMan• 5 years ago

A corollary to this might be "a new part on the shelf is not truly 'better' than a 90% worn out one still in service."

Having ready spares is great but I'm sure the HN crowd knows how much crib-death there is on new, replacement bits of all kinds.

Until it's actually been run-in for a while how do you really know it's going to work when you unbox it to replace a truly dead piece?

The upshot of this is' RAID-6' type designs are the most reliable in the real world since when one fails at least you are leaning on other parts that have been run-in and are past the leading edge of the bathtub curve.

HPsquared• 5 years ago

Held together with spit and baling wire as it is, the fact that it mostly works proves that the overall architecture is robust.

IncRnd• 5 years ago

proves that the overall architecture is robust.

Not really. What it shows is the stark difference between two ideologies. The first camp contains people who believe in Postel's Law, "be conservative in what you do, be liberal in what you accept from others". The second camp has people who recognize that the current world is not a cooperative network of researchers: "all input is untrusted".

Krebs is absolutely in the second camp.

shadowgovt3• 5 years ago

But the two philosophies aren't really in contention. Proper adherence to Postel's law also includes accepting malicious traffic (and then doing something reasonable with it, like black-holing it).

The "liberal in what you accept" part is mostly honest acceptance of the reality of the network: you cannot control the information sent to your service, only how you respond to it.

inopinatus3• 5 years ago

These are not ideologies, but compatible principles from separate concerns. One relates to protocol errors, the other to information validation & verification.

azeirah• 5 years ago

If Alan Kay is to be believed (which I hope he is!) then the internet was originally inspired by multicellular lifeforms.

I'd say the internet has some sort of a "biological" architecture. Robust in the sense that organisms are robust; extremely messy, sensical from a high-level view, chaotic from a low-level view.

lioeters• 5 years ago

That's an interesting point about how the Internet has a kind of "biological architecture", whose chaotic and self-organizing nature makes it robust and resilient.

It's the same quality that's claimed by proponents of "decentralized autonomous organizations" (DAO), though for that I'm not yet convinced of its practicality.

Someone on this thread mentioned the book, Antifragile: Things That Gain from Disorder.

Just as human bones get stronger when subjected to stress and tension, and rumors or riots intensify when someone tries to repress them, many things in life benefit from stress, disorder, volatility, and turmoil. What Taleb has identified and calls “antifragile” is that category of things that not only gain from chaos but need it in order to survive and flourish.

This reminds me of the concept of "emergence" in living systems.

the arising of novel and coherent structures, patterns and properties during the process of self-organization in complex systems

https://en.wikipedia.org/wiki/Emergence

h2odragon4• 5 years ago

It's Anti-Fragile. If it breaks all the time, everybody is highly experienced at patching together new workarounds, mechanisms for fail over are in place and regularly tested, and there's whole classes of corner case bugs that get flushed out to be stomped (or nurtured as cherished pets) instead of breeding in the dark and jumping out at you all at once.

How can the "Internet routes around failure" be trusted without testing? Everything needs regular exercise or it atrophies.

fredophile• 5 years ago

"The internet routes around failure" hasn't been true for a long time. It refers to the original topography which has been replaced with a hub and spoke model. Remove a few hubs and you have disabled a large portion of the internet.

paxys• 5 years ago

What original topology are you talking about? There was never a time when your end consumer device could be used as even a semi-reliable web server.

ip262• 5 years ago

If by “remove a few hubs” you mean level a few major colo’s then ok, but as far as I can tell there is not any single strand of glass or single switch or single server that can take everything down with it.

toast03• 5 years ago

Back in the 90s when I got on, so much of traffic was exchanged at MAE-West or MAE-East, and a backhoe in Iowa could make nearly all the cross-US traffic go through Europe and Asia instead.

These days, there are lively public internet exchanges up and down both coasts, in texas and chicago and elsewhere. A well placed backhoe can still make a big mess, many 'redundant fibers' are in the same strand, and last mile is fragile, but if my ISP network is up to the local internet exchange, there are many distinct routes to the other coast and a fiber cut is unlikely to route my traffic around the world.

peoplefromibiza3• 5 years ago

back in the 90s all the Internet traffic in Italy was routed around two big hubs, hosted in two public universities (it was mainly one, in Rome).

Internet is much more reliable now.

https://en.m.wikipedia.org/wiki/GARR

I studied CS in Rome at "La Sapienza" and I was merely 10 steps away from INFN (national institute for nuclear physics) where GARR phisically resided.

I spent more time there than in my class.

That's how I got involved in this new "internet thing".

Iv• 5 years ago

Disabled? Throttled down the speed by half? Yes. But to disconnect whole regions, you have to do conscious sabotage and even Mubarak did not manage to switch off Egypt when he wanted to and gave orders to.

throw0101a4• 5 years ago

Remove a few hubs and you have disabled a large portion of the internet.

"A few hubs"? Just Hurricane Electric has presence in many, many IXPs, and they're not even the largest transit provider:

* https://bgp.he.net/AS6939#_ix

AT&T, the largest transit provider, has a ridiculous number of peers:

* https://bgp.he.net/AS7018#_peers

And there are several different major global providers:

* https://en.wikipedia.org/wiki/Tier_1_network

A lot of the Big Tech companies are also building their own private fibre networks so they don't even have to worry about sharing infrastructure with the telcos:

* https://www.submarinecablemap.com

Ekaros2• 5 years ago

And hub here could refer to entity like Level3... One day, no more money. Just turn of the power on everything and start liquidating... Or someone isn't there anymore to fix stuff when it breaks or changes are needed...

tialaramex• 5 years ago

Bob Metcalfe repeatedly predicted a "gigalapse" which he defined as a loss of one billion hours of Internet connectivity. Bob would say look at how fragile this all is, and his point wasn't (although at the time it would too often be portrayed this way) "the Internet will fail" because Bob isn't actually an idiot, but it also shows a substantial lack of foresight anyway. Of course ~everybody will have the Network, the Internet is the Network, and so then normal sleep is an order of magnitude worse than a "gigalapse" yet happens every day and we don't even care.

The "gigalapse" seemed like a big deal because proportionally a billion user hours in 1995 is everything doesn't work for days, or most things don't work for several weeks... but today of course it could also be everybody in the world is inconvenienced for about long enough to make breakfast. Oh no.

xwolfi3• 5 years ago

It s almost alive, it'll wake up one day ask start asking questions :D

Stampo00• 5 years ago

This has big "can you call it broken if it never actually worked in the first place?" energy.

EDIT: I have a soft spot for the person who is so over it that they sit around pontificating the meaning of the word broken instead of actually getting up and helping in the midst of an emergency. Because half the time that's me. When prod is on fire every day, eventually that becomes the norm.

h2odragon4• 5 years ago

It's much worse than that: it becomes philosophy.

Beyond some level of complexity a system isn't one, anymore, it is an emergent phenomenon that has many of the aspects we ascribe to "life." Our inputs affect the systems, surely, but not in a predictable or even repeatable way.

A cow is an "inefficient means" of turning grass into tasty beef, by some accounting. But the cow is, as far as it is concerned, fully efficient at being alive. our notion of "what is the internet" fell out of sync with the realities sometime around '94 i think.

dukeyukey• 5 years ago

Hah, that's a good way of seeing it.

tata71• 5 years ago

IPFS ENS

Nevermark• 5 years ago

I would call the Internet resilient, in that lower level failures are often reliably re-routed, healed, etc., but at higher levels the same kinds of failures happen over and over.

Anti-Fragile would imply that each doh-inducing failure resulted in internet wide improvements. Something that famously is not happening.

The Internet: "It's Anti-Fragile!" (looking up)

The Internet: "It's Duct Tape all the Way Down" (seen from above)

TheMagicHorsey• 5 years ago

What are some other/better proposals for how to organize a world-wide network? This is one area where I have not seen many articles. But admittedly, I'm probably not looking in the right places. Any suggestions?

teddyh• 5 years ago

You could ask Cloudflare or Google and they would probably say that they should run it all.

hunterb123• 5 years ago

Which funny enough, are the only parts that really go down all at once.

When the internet has troubles it's a mistake of a giant centralized service.

Most of the time only that service is affected by their own mistakes, but sometimes that service hosts a lot of others, or is so massive that they cause DDoS attacks like when Facebook went down and their clients spammed DNS servers.

Seems the internet is fine. Centralized services not so much.

cblconfederate• 5 years ago

blockchains are one

throwawaysea• 5 years ago

I am not a security expert, but this article seems to just be saying that the Internet is held together by trust, convention, and an ever-evolving set of technologies. In the case of Level3 (now Lumen), it seems they did not deprecate an insecure method that others already deprecated. And it seems that better technologies are on the horizon (RPKI) but not yet fully in use. To me this doesn't feel as bad as "spit and baling wire". We could be more secure by holding everyone to a stricter standard on adopting newer, more secure technologies. But is it really as broken as the title suggests? I don't think so.

oblib• 5 years ago

I've been leaning on my app's users to do what they need to do on their end to implement offline/local first use of the app and they just do not get it. For them the only issues they've had were connection issue on their end with their service providers so they don't feel this is an issue of concern.

But I read stuff like this, and in this case it's Krebs, so I have to expect these kinds of issues will pop up. The article mentions the FB outage and most everyone on my FB feed was freaking out over not being able to access it, and for the most part it's not a critical service. And when they came back online some of the conspiracies they were sharing about what/why it happened were way over the top.

From my perspective it feels like everything on the internet is just one missed tap on a keyboard from breaking.

imchillyb• 5 years ago

Has no one here seen:

https://www.reddit.com/r/cablefail/

There are 1000s of these pictures.

The entire world's IT infrastructure has been held together with spaghetti-noodle-cabling and bodged patches for over three decades now. I'm even guilty of it. Most IT guys are guilty of it.

Don't even get me started talking about how bodged together corporate codebases are. Banks are perhaps the worst offenders. Old hardware running with bandaids and bubblegum. Software that more than 5 different teams have had their fingers in, mucking about, and some codebases have orders of magnitude more than that.

Anyone that didn't know this, doesn't pay attention, or just started internetting.

jl6• 5 years ago

Don't even get me started talking about how bodged together corporate codebases are. Banks are perhaps the worst offenders. Old hardware running with bandaids and bubblegum. Software that more than 5 different teams have had their fingers in, mucking about, and some codebases have orders of magnitude more than that.

And yet, perhaps counterintuitively, the main downside of such systems is that they are slow and expensive to change, not that they are unreliable.

dragonwriter• 5 years ago

And yet, perhaps counterintuitively, the main downside of such systems is that they are slow and expensive to change, not that they are unreliable.

They are slow and expensive to change because they are maintained because they have been, at great expense and cost in both failures and remediation efforts, made tolerably reliable (but still extremely fragile) so long as things are exactly within certain expectations (which often have been narrowed from the intended design based on observed bugs that have been deemed too expensive to fix), and it is inordinately difficult to modify them without causing them to revert to a state of intolerable unreliability.

They are systems that generations have been spent reshaping business operations around their bugs to make them “reliable”.

charcircuit2• 5 years ago

How "pretty" your cable routing is doesn't effect performance / reliability.

SahAssar3• 5 years ago

Pretty in this context mostly means predictable, which does have an effect. If you are called to fix a broken connection and you get a rats nest it will take a lot longer and might impact other users.

cgh2• 5 years ago

I had a friend who worked in IT in a major Canadian bank some years back. He said they’d lost the source code (Cobol, I would imagine) for certain jobs and had taken to editing the binaries directly with a hex editor.

SahAssar3• 5 years ago

If you haven't read it https://www.stilldrinking.org/programming-sucks beautifully describes this and has both made me want to quit and start a farm and want to stay and improve things depending on the day.

If you (the collective you, everyone reading this) haven't, read it.

walrus012• 5 years ago

Have literally seen major things taken out because somebody tripped over the $5 power strip to the sole authoritative nameserver and nobody realized/cared for years that the authoritative ns2/ns3 weren't really receiving zone transfers.

winternett6• 5 years ago

And Google requiring (costly and/or time consuming) SSL certs to be applied on all sites to "ensure security" was also a big industry money making nightmare for many independent (non-income-driven) sites that is still playing out badly, and not providing much more security.

Two factor authentication and account verification is really an elaborate corporate sham to get people's phone numbers and PII for free. It doesn't do anything new or good for consumers in terms of security over time. There, I said it.

I prefer the old Internet. All these new fangled "fixes" are only makin it worse, more expensive, and overly complicated. :/

fulafel4• 5 years ago

Deprecating unencrypted HTTP is a big systemic improvement even though some individual sites may not benefit much. It's a network effect. (What's the money grab given free let's encrypt certs?)

switch0072• 5 years ago

Setting up, monitoring and maintaining LE isn't free

winternett6• 5 years ago

Lets encrypt from what I understand require time consuming updates every few months. My host provider also does not allow me to install them manually, further complicating the process, and conveniently they sell certs for $125 a year... Per site. It's been a thorn in my side because we're too big to easily move now.

oneepic2• 5 years ago

Honest question (IT/security noob) -- why does it not provide that much more security? I like verifying that my traffic is going where I want.

winternett6• 5 years ago

With Encryption being applied to every site as a requirement is relatively new since google made it a requirement in Chrome.

Previously it was only required for secured transactions like purchases and working on health care records etc... And very rightfully so.

Now Google Chrome flags even simple (informational) sites for not being encrypted, and (quite possibly) rightfully so because of the potential for tracking/abuse, but adding encryption to a site is costly for independent sites (not hosted on social media or corporate platforms like blogs etc...

You shouldn't be required to encrypt a baking recipe site if you don't want to... Ultimately laws should discourage data abuse, and/or encryption should be inherently provided for every site/app uniformly by all web host providers (natively and inherently, and at a far lower price than it is now, generally speaking).

Too many people are running widely varying encryption measures, and implementing security in too many different ways to ensure that it is stable across the Internet. Security is best when it is uniform, fortified by rules and regulations, and updated ritually.

ratorx• 5 years ago

I’ll reply to this and some of your other comments in this reply.

In a lot of cases, SSL is not expensive or time consuming. It is a single line in cron. I appreciate that this is not the case for your hosting, but economic pressure is one of the main ways SSL can be more utilised. The fact that you’re considering moving away from them, suggests that their business will suffer in the long term, if they don’t make integrating SSL easier/less expensive. This is good economic pressure, and its likely the best pressure that can be applied right now, considering the glacial pace of technology laws in almost all countries. You seem to be generalising your situation and applying the blanket “it’s too expensive” argument to everyone, even though it’s mostly a non-issue for people who have better hosting providers or not as much legacy.

Arguably, building a website with a login is a LOT easier and cheaper now than it was 10 years ago, because Let’s Encrypt is such a well known option. If they wanted to do so 10 years ago, they would have most likely had to pay through the nose for an expensive certificate. You seem to also have forgotten about these people with your blanket statement about hosting websites being more expensive for everyone.

Is the security provided significant in simple sites? Probably not. However, having SSL be a default is good overall. It gives less chances for operators to screw up because non-HTTPS raises very user-visible alarm bells. If your site is small and non-revenue generating, then why does the security alert even matter? It doesn’t prevent anyone from accessing the website.

Your 2FA argument is wrong. Sure, there may be multiple reasons for mandating it, but for regular users, 2FA is good defense in depth, that offers protection against password compromise. Again, the average consumer doesn’t necessarily have strong passwords or unique passwords across services. 2FA is good protection for them.

Also, if mining user data was the main reason for 2FA, big tech wouldn’t support hardware security keys for 2FA. Mobile 2FA is a usability compromise because it targets a lowest common denominator that (almost) everyone has.

snoopen• 5 years ago

Sure, certificates can be time consuming at the moment but that will only get easier. Just like hosting the underlying website.

The number of sites that should have had SSL but didn't was laughable and justification enough for browsers to require SSL.

I don't know if you're being deliberately alarmist, but 2FA is a huge peace of mind when done correctly with one time codes. Those don't require phone numbers and is the properly secure method.

Sure the old internet was a bit more fun and carefree, but it became far less fun when you had your online accounts compromises because of weak or non existent security.

rufus_foreman• 5 years ago

Is there a shortage of spit and/or baling wire?

krisrm• 5 years ago

I'm a bit shocked that MAIL-FROM auth was ever accepted, let alone until 2012. Even the other auth methods via email seem somewhat dangerous, though I sincerely hope these registries follow extremely strict policies for key management.

throwaway9843932• 5 years ago

That's how validation of TLS certificates and domain registration still works.

oneepic2• 5 years ago

...and routers

SahAssar3• 5 years ago

“LEVEL 3 is the last IRR operator which allows the use of this method, although they have discouraged its use since at least 2012,” Korab told KrebsOnSecurity. “Other IRR operators have fully deprecated MAIL-FROM.”

I'd prefer if we kept deprecated and removed as two different terms. It sounds like level3 deprecated it, and everyone else removed it. To me (and most definitions I can find) deprecated basically means "don't start using it, if you are using it stop using it, we will remove it soon but have not done so yet for compatibility reasons"

a-dub• 5 years ago

reminds me of the old email interface run by network solutions when they were the sole registrar for everything that wasn't government.

throwaway9843932• 5 years ago

Whenever people are predicting the end of the world because of some political or cultural upheaval, I think about the internet, or airport security. There are really simple ways that any idiot could totally fuck up either of them and cause catastrophic problems. But it doesn't happen. What that shows is that the potential for catastrophe has nothing to do with catastrophe actually happening. Even if the world could fall apart around you at any moment, it's probably not going to.

Ekaros2• 5 years ago

I have kinda wondered how robust Internet would truly be if the proverbial stuff really hit a fan. I mean proper nuclear wars or economies including large ISP going down with admins dying or being fired on left and right. Specially with all of the talk of global crypto and so on...

As I understand it is continuously almost hand-tuned machine and enough misconfiguration or entities simply stopping to exist much of the transit capacity would be gone. Works well enough when it is not touched and there is someone on call to fix it, but specially if later was gone? How robust and self-correcting is it really? My guess is probably not at all...

ronnier• 5 years ago

It wouldn’t work. Just look at what happens during bad storms.

tim333• 5 years ago

I've never know the internet to go down in a storm. Does that happen? I mean I've had the wifi router stop if there's a power cut and had to use cell data but that's about as bad as it's got for me.

yazanobeidi• 5 years ago

You’d be surprised what goes on in Level 0… who works there, how they work, the hour-by-hour. Even at a big company that handles most of the world’s credit card transactions at their biggest data centre, for example. It would absolutely blow your mind. It’s amazing the internet works as reliably as it does!

mmaunder• 5 years ago

BGP is how the most critical infrastructure on the Net communicates and BGP4, the current version, is 15 years old.

pmlnr• 5 years ago

Age, alone, doesn't make something bad or unreliable.

quickthrower2• 5 years ago

Could the -MAIL-FROM requests somehow be tunneled through a VPN so that they are only accepted through that VPN and not the wider internet?

illwrks• 5 years ago

Value less comment... but I've always had a thought in the back of my head that it will only take some catastrophe for us to be back in the dark ages. I hope I'm wrong.

a9h74j• 5 years ago

As a less all-or-nothing scenario, one relevent concept comes from Joseph Tainter as a theorist of collapse. He proposes that eventually complexity reaches a point where it not sustainable (either in general or at the margin), and a society collapses back to a configuration of less complexity.

thenthenthen2• 5 years ago

'spit'...nice way to put it I guess, Krebs still living with his mom init?

rswail• 5 years ago

The term "spit and baling wire" is a well known (US/AU rural at least) expression to mean something that is working well enough but is fragile.

thenthenthen2• 5 years ago

I never knew! Excuses for the terrible joke

pjf• 5 years ago

Come on, who in 2021 actively drops routes because of an entry in the Level3 IRR? I mean, IMHO, it's not close to impacting the whole Internet.

FredPret2• 5 years ago

Every complex system - including the human body, the global economy, the ecosystem, and large software systems - grew one functional hack at a time

walrus012• 5 years ago

Krebs is not wrong but he also doesn't get into how many things are "verified" through an even older mechanism, the LOA (letter of authorization/letter of authority), which is literally just an actual letter on one corporate entity's letterhead, specifying some CIDR-notation prefixes/ranges that are permitted to be announced by that entity's upstreams, signed by a corporate officer. Often in the form of a raster scanned PDF.

BGP4 in general dates back to a time when everybody trusted each other and the global routing table/network engineering community was a much smaller thing. We've been trying to glue better things on top of it for 25+ years.

chasd00• 5 years ago

You know what? I get the feeling that _everything_ is held together with only spit and baling wire.

balaji1• 5 years ago

I get the same feeling too. And maybe that's fine. We are led to believe that we are over-reliant on a particular things like the internet or the shipping supply chain but we probably are not.

Most modern, massive systems seem that held with only spit and baling wire. This might not be true for infra like roads, bridges and dams. And there are things, buildings and structures that are 10s or 100s of years old that still work. Power plants and cars for example.

jrochkind1• 5 years ago

However, the additional integrity RPKI brings also comes with a fair amount of added complexity and cost, the researchers found.

I wish our government spent billions on funding open source software to reduce that complexity and cost, instead of on introducing/hoarding vulnerabilities.

This is an appropriate task for government funding. (even better yet, multi-lateral/international funding). Nobody else is going to provide the resources to get it done. Clearly.

solotronics• 5 years ago

Some of this has to do with IPv4 and massive carrier NAT schemes. If technical people could get an ASN easier and route with each other on IPv6 it theoretically could provide backup paths for neighborhood level traffic. I guess the transit/ISP would have to enforce something like BGPsec with these little peers to prevent mistakes or malicious ASNs.

alexshendi2• 5 years ago

Yes, but why did he only realise this on November 26th, 2021 AD?

alexshendi2• 5 years ago

Of course FidoNet is much better.

cainxinth• 5 years ago

So is society. When you’re a kid, it seems like everything is incredibly well organized and orchestrated.

As an adult, you realize the world is a patchwork of semi-functioning systems and it’s a miracle the whole thing works as well as it does.

iancmceachern• 5 years ago

I've been thinking along these lines a lot recently. When young I thought everything was all carefully architected, thought out and planned. I've come to realize it's all chaos, interactions between the disparate desires of various individuals and groups randomly acting constructively and destructively with eachother creating random results.

The internet is held together with spit and baling wire

Related Stories