Make It Ephemeral: Software Should Decay and Lose Data

https://lucumr.pocoo.org/2024/10/30/make-it-ephemeral/

by BerislavLopac • 2 years ago

31 37 2 years ago

lucumr.pocoo.org

maxbond2• 2 years ago

The physical world does impose itself on software in some ways. Software is prone to going out of fashion, becoming disused and unmaintained, and eventually forgotten. Peering at a long dead document format in a hex editor is the digital equivalent of wiping dust and grime from the wall of a forgotten tomb.

Arainach• 2 years ago

This is a strange and awful take. Physical objects decay to the fundamental forces of nature, but that is a flaw, not something we should emulate.

Software is a tool. Software is infrastructure. Degradation is desirable in neither.

Data has metadata - creation time, access time, etc. - that can power workflows to identify and deal with old data if that is desirable, but in most cases they are not desirable. I haven't looked at the directory full of my wedding pictures in a decade, but I don't want my photo editing software to decide they're irrelevant and delete or hide them.

Having access to old documents and data is often important and beneficial. It's why people look at inscriptions in old books, why they keep photo albums, why finding a forgotten item when moving furniture can be delightful. It's why we're able to identify trends.

Most crucially, it is incredibly difficult to predict in advance which data will be useful in the future. Proactively discarding things when storage is incredibly cheap is a net negative.

rpmisms• 2 years ago

You can view software as tools or art, it is arguably both.

TZubiri6• 2 years ago

Disagree, I'm with Kay on this one, life is much more powerful than current machines.

When was the last time your mind ran into an out of storage issue and crashed?

You talk about metadata being important, yet more than half of the storage used today isn't used to store books, or medical records or whatever business case, but traffic logs and virtualized package dependencies and shit like that.

fragmede• 2 years ago

It's called a senior moment, and you'll have them too if you've never had one before. In a middle of a conversation, you'll just forget what you were saying because you got distracted.

castwide3• 2 years ago

The disagreement here seems to be about what kind of data should be considered ephemeral. Traffic logs from a decade ago? A ten-year-old package lock? I'm sure I don't care anymore. Documents I wrote in the 90s? Yeah, I might still want them. I might not need them everyday, but I can say from experience that I've needed to track down files that are more than a decade old, and I'm glad I was able to find them.

the_mitsuhiko• 2 years ago

Author here.

Physical objects decay to the fundamental forces of nature, but that is a flaw, not something we should emulate.

That is explicitly called out in the article and I do not agree with you. Useful information is replicated, less useful information tends to die out. That we threw away that concept entirely in the digital world without a good replacement has some pretty strong downsides from my experience.

And if you read what I wrote in more detail you can see that I'm very open to the idea of soft deletes and intentional information hiding.

erik_seaberg6• 2 years ago

Information that is known to be useful is sometimes replicated. The https://en.wikipedia.org/wiki/Rosetta_Stone#Rediscovery was abused and damaged; only four incomplete copies were gradually found by luck after two millennia. There are entire academic fields trying to learn things about our history by digging up stuff that was thrown away or abandoned.

Barrin92• 2 years ago

It's why people look at inscriptions in old books, why they keep photo albums, why finding a forgotten item when moving furniture can be delightful. It's why we're able to identify trends.

I think it's often the opposite. Drawing confusing or bad analogies to the past rests on the false assumption that the future resembles the past, and the reason people love old photo albums or forgotten items is usually a kind of nostalgia, not because they hold relevance towards the future.

This is also why forgetting or deliberately wiping the slate clean is fundamentally a good thing, it forces people to rebuild things from first principles without being bogged down in false assumptions carried over from the past, it's why the people who build new things tend to be young rather than old. If past information was so valuable in determining the future most visionaries would be 60. Hanging on to old information is a form of digital hoarding.

totallykvothe• 2 years ago

Methinks the mindfulness cult has gone too far

TZubiri6• 2 years ago

Thought about this. I agree 100%. The incumbent belief in software that machines should always remember (embedded in postgresql logo for example) has a dystopian sense of inmortality desires.

I would like to play with the idea of a filesystem layer solution that tiers out information based on staleness. The closest I've seen to this is AWS S3 tiers, you've got standard S# at 2.2 cents per GB month, then infrequent, then glacier and some tiers in between.

tracerbulletx• 2 years ago

I mean, retention policies exist.

donpark• 2 years ago

Decay and losses are just two of the many constraints that could be mixed like paints to create new systems.

Limitations, artificial or not, are not always bad. Walls, for example, can be seen as limitation or protection depending on how it's used.

paulproteus• 2 years ago

For this reason, my browser's Downloads folder is /tmp.

TacticalCoder• 2 years ago

For this reason, my browser's Downloads folder is /tmp

OK. But my desktop reaches uptime of six months very easily (its not common that a security update mandating a reboot happens with Linux).

Once in a while I clean my browser's Download folder. When I do that the very first thing I do, before the mess creeps in everywhere in "proper" directories is to deduplicate the browser's Download folder.

The amount of time I'll have bankingstatement20241103.pdf, bankingstatement20241104-1.pdf (which should be really be "-2" btw instead of that zero-based non-sense but I digress), bankingstatement20241104-2.pdf, etc. in my Download folder is insane.

orbital-decay• 2 years ago

That's a very feel-good "why" that doesn't answer on "how", which is 99% of the work. How exactly do you select what to forget?

batch12• 2 years ago

The attempts I've seen at this are usually in support of a corporate data classification and retention policy and are pretty hard to pull off successfully. The attempts I've seen usually involve adding metadata or tagging the data either manually or using automated classifiers.

crabmusket2• 2 years ago

What are some concrete examples of this?

- Linear automatically cancels tickets which haven't been touched in 6 months

- Trello would visually "age" cards, making them increasingly yellow and tattered

- Datadog dashboards have a "popularity" score based on usage

erik_seaberg6• 2 years ago

As jwz points out in "The CADT Model," abandoning reported bugs is just telling your users not to bother reporting anything.

crabmusket2• 2 years ago

In my experience, things don't get done just because they are hanging about in the backlog.

Most auto-closed issues in my small team are of the varieties "the PM had a nice idea one day" and "long standing annoyance which we would need to re-architect to solve and we'll get to it one day".

And I don't even necessarily disagree with the CADT post, but... most users are not, for better or worse, so easily dissuaded.

bsnnkv• 2 years ago

This was an excellent read and it resonates with me a lot.

I have written here before[1] about the tyranny of "read it later" apps and how liberating it felt to ditch them and move to a "read it now or read it never" (RINORIN) mindset.

Although this articles talks about digital ephemera in different contexts such as shared spaces, wikis, runbooks etc., I believe the underlying idea is the same.

[1] https://news.ycombinator.com/item?id=41991376

erik_seaberg6• 2 years ago

Not everything important is equally urgent. https://en.wikipedia.org/wiki/Getting_Things_Done recognizes the need for a trustworthy "I know I am going to get back to this at the right time" process, so that I don't continually reserve a bit of my brain worrying about forgetting.

keyle• 2 years ago

It's already ephemeral and decays by itself.

Honestly as a polyglot, only a Go project can run without being touched for 5 years.

The rest of them get vulnerabilities discovered in the packages, which requires you to upgrade, which changes the feature set and takes re-work.

Also design ages, UX paradigms change over time, look at the "web 2.0" look today and the "web 1.0" of before. Skeuomorphic design aged as fast as Garage and Jungle music did in the 90s. This will be the same with flat design and neuomorphic design in 5-10 years, if there is even an UI then.

All in all, software does decay and we lose data every day. Try go through your oldest bookmarks, many sites will be gone. Or run software that used to run on windows 98 or DOS or amiga or classic mac, you need to jump through hoops, the UX will be terrible, that's aging.

Software doesn't need artificial aging, it erodes well all by itself. Even fast moving code in a startup changes so much from week to week, your merges can be difficult. That's like a river bed redefining itself during a rain storm.

As humans go through time, so does software as an extension.

dig1• 2 years ago

Honestly as a polyglot, only a Go project can run without being touched for 5 years.

Go CVE archive [1] clearly indicate otherwise. What distinguishes a Go project from any other programming language?

[1] https://pkg.go.dev/vuln/list

im_dario• 2 years ago

What distinguishes a Go project from any other programming language?

My experience with Go, as user and open source maintainer, is that many libraries just work even if untouched for several years.

There is a culture around building small libraries, with minimal usage of dependencies beyond the standard library. This leads to most CVEs existing in the standard library, which combined with Go's commitment to backward compatibility, makes improving your security posture simple by just upgrading the toolchain.

booleandilemma• 2 years ago

I don't think I could disagree more. He compares software to "the physical world", but photos in photo albums don't just disappear because they get old, and if I put something in a filing cabinet I know it'll be there in 1 year or 10 years.

maxbond2• 2 years ago

That's because you took the care to put them in the photo album where they would be protected. If you left it sitting naked on your desk it would fade, accrue coffee stains, etc. Same goes with the filing cabinet. That's what Ronacher is suggesting - save the things that people care about, discard the things that are no longer relevant.

I don't think it's a dogmatic position either, I don't think the suggestion is to always do this but to do this far more often than it's done now (almost never). And they are pretty explicit that they think this should be a default which can be overriden, and that you should be able to save things indefinitely if you so choose.

tqi• 2 years ago

I'm referring to making a deliberate design decision to discard data at a later time. I'm willing to bet that your cloud storage or SaaS applications likely serve as dumping grounds for outdated, forgotten files and artifacts. This doesn’t have to be the case.

IMO there is no way to do this programmatically that satisfies all of your users. What the author describes is organizational debt, and the only way to deal with it effectively is to have the organizational will to periodically clean up that debt. It's not easy, and any shortcut comes with it's own drawbacks (see corporate data retention policies that inevitably nuke documentation or context).

Make It Ephemeral: Software Should Decay and Lose Data

Related Stories