Designing Low Upkeep Software

https://www.jefftk.com/p/designing-low-upkeep-software

by mhb • 5 years ago

402 248 5 years ago

Using a filesystem is underrated these days with web this or that. I'm debating relying only on the disk for my Adama platform (http://www.adama-lang.org/) where part of my thesis is that by combining mem, cpu, storage into a single node that ill get decent utilization once I figure out a durability story sorted out.

antoinealb• 5 years ago

I personally find myself reaching more and more often to SQLite as an in-between. I like having access to basic SQL and not needing to care about filesystem details for reliability is nice. And it is part of Python standard library so I need literally nothing added.

z3ugma• 5 years ago

Slight alternative: let smart people who design rock-solid software handle serialization for you, use SQLite to get the benefits of files-on-disk, portability, and make it easier if you use different datatypes in your scripts.

nemetroid• 5 years ago

I think SQLite fits the bill described in the post. It's included in Python's standard library, stores data on the file system, and doesn't require an external process.

phkahler• 5 years ago

But it IS an additional dependency.

KronisLV• 5 years ago

This really reminds me of an article that i read a while ago, called "SQLite as an application file format", which seemed to express some ideas like that: https://www.sqlite.org/draft/appfileformat.html

ellimilial• 5 years ago

For https://biokeanos.com (biomedical data catalog and discovery tool), keeping things on disk and versioning by pushing to repo saved me months already on dubugging and maintenance only.

warpech• 5 years ago

I have been thinking of such setup lately for another piece of software. Is your sistem distributed in any way? If so, how do you resolve conflicts in version control?

drivers993• 5 years ago

I've been thinking about a bare metal Forth system and one of my goals for it would be that anyone could recreate it without having any specific dependencies to build it. Something fully explained like jonesforth but without a linux and assembler dependency. Part of my motivation for that is when I try to learn bare metal programming, the tutorials are rapidly outdated because of changing dependencies/tools, broken links, outdated information, etc. Maybe just some hex codes of the machine language which you could use xxd to convert to a binary, or if you didn't have that you could use any programming language that can write a binary file to create it, or a hex editor. The ultimate step would be that the system would be able to compile the next version of itself. (Self-hosted forth implementations are really common; you can find a bunch by googling that.)

senkora• 5 years ago

Look into CollapseOS if you haven’t already!

https://collapseos.org/

drivers993• 5 years ago

Wow that's cool. I'll definitely be checking it out. Ironically though, it illustrates one of my points: the documentation link is already broken (although it also exists in the download). It says it runs on a POSIX environment and uses cc so it still has more dependencies, but definitely worth digging into and a step in the right direction. Thanks.

Karrot_Kream2• 5 years ago

Bare metal programming in C99 hasn't changed in... Ever really, so that's another option if you want something less esoteric than Forth. (Though Forth is a lot of fun and will give you a dynamic experience that few other environments at this level of the stack can offer.)

akkartik• 5 years ago

I was just telling someone[1] that I explored Forth for this exact purpose a few years ago, but ended up with a more imperative, statement-oriented approach: https://github.com/akkartik/mu

[1] https://merveilles.town/@akkartik/106978783913724906

drivers993• 5 years ago

Just saw this. The live update of the stack at each step with the function definitions on the left is really cool!

OneWingedShark6• 5 years ago

Look into "SeedForth".

mtlynch3• 5 years ago

I definitely agree with OP on this and do my best to minimize maintenance burden for most of my projects. This has worked out pretty well in that almost everything I've published[0] can chug along in the background for months or years without me touching them at all (some of them earn a few hundred dollars per month[1])

On the server I run Ubuntu LTS until it's near the end of its supported lifetime. Every two years I run do-release-upgrade and move to the next one. This cost is shared over many jefftk.com projects, so it isn't too bad.

I was surprised by this part because running your own server seems like a big maintenance burden. I'd expect that you have to regularly upgrade your packages to protect yourself from security vulnerabilities.

I've avoided this by using PaaS solutions like AppEngine or static hosting + cloud functions. The closest thing I have to a VPS is publishing Docker containers, but that's basically my last resort.

[0] https://mtlynch.io/projects/

[1] https://mtlynch.io/retrospectives/2021/09/#legacy-projects

christophilus2• 5 years ago

Application of security patches can be automated, and depending on your application’s dependencies, can be done without worry. I run about 10 bare metal transcoding servers, and it’s zero hassle.

mtlynch3• 5 years ago

Can you share more about your setup? Is it cronjobs to just apt-get update && apt-get upgrade -y? How do you handle packages that require reboots?

pnt12• 5 years ago

I wrote a static website for a family business using vanilla html - it has been up and running for several years.

I also have an (abandoned) website using a markdown to html converter. I've switched converters two times because they eventually broke over the years, and finally moved to plain pandoc + script the last time. Even then, the CSS somehow broke and I haven't fixed it yet.

Maybe my next project will be to convert it to html-only and actually add updates instead of just dreading to pick it up and find another broken tool.

iveqy• 5 years ago

Very nice writing except the python part. Just 2to3 really have been a maintenance pain for me. The hardest part with script languages to actually know if somethings is wrong or not without extensive testing.

Writing something maintenance free should be done in a compiled language.

bArray3• 5 years ago

Very nice writing except the python part. Just 2to3 really have been a maintenance pain for me.

I agree. I could have been talked into getting onboard with Python for long-term low-maintenance projects if only they didn't burn everybody with the backwards compatibility. I understand of course why the changes were made, but it doesn't make it any easier. It's not impossible that 3 -> 4 will do the same.

I was wondering the other day whether something that is already dead might make a better scripting language. For example, you haven't got to worry about any random changes to the COBOL language! I suspect something like JS is pretty much at maturity now with WebASM poised to replace it.

iamevn• 5 years ago

I doubt 3->4 will be too much of a shock, iirc the only planned breakage will be switching annotations to have delayed evaluation so annotations in things like classes can reference the thing that's not yet defined

joshuaissac• 5 years ago

I think it is not necessarily about being compiled versus interpreted. It is more about static typing, and the type-related issues being revealed and fixed right at the beginning instead of only when the user invokes the problematic branch of code at runtime.

jhgb3• 5 years ago

On the other hand, one of most long-lived and apparently low maintenance pieces of software I've ever used so far has been GNU Maxima, which is not written in a statically typed language.

vonadz• 5 years ago

I don't know about waiting so long on updating Ubuntu, or essential server software. As an Arch Linux user, I've come to loathe outdated packages that break, which is ironic because Arch is notorious for breaking on new updates. From personal experience, the latter happens less frequently than the former.

hsn91510• 5 years ago

People think if a project on github hasn't been updated since 3 months then it has been abandoned!!

Like, why don't we just let projects be "done"? Things don't need to be maintained and updated for eternity.

In my mind, the best software engineering is where you solve a problem once and your solution just works and needs no configuration or maintenance or updates.

This of course has a very low chance of happening if your system has to exist as a part of "ecosystem" where you expect/assume the presence of some external service that can change its API on a whim (or just disappear).

Yes, to a certain degree it's impossible to design software that does not exist as a part of an ecosystem, which is why I put it in quotes.

Some APIs are stable and are guaranteed to continue to exist for a very long time: CPU instruction sets, networking protocols (IP/UDP/TCP), operating systems (AFAICT: Linux (the kernel) works hard to not break user programs, and Windows is kind of known for bending over backwards to maintain compatibility with old programs), file systems, etc.

What I'm advocating for here requires that your program be compiled into a native executable binary file, and it must embed all its library dependencies (aka static linking).

amw-zero5• 5 years ago

I completely agree with you, but your last point is exactly why recent updates are a proxy for ‘well-maintained.’ Software is so complex that it _has_ to have dependencies, and even if you vendor them / compile to an executable, that doesn’t account for upstream bug or security fixes.

hsn91510• 5 years ago

1) If you buy into this concept, you would naturally limit your dependencies to the bare minimum.

2) Static linking does a good job of protecting against when upstream introduces new bugs or security holes.

umvi• 5 years ago

I disagree. How many times does software use 3rd party dependencies when the standard library would have sufficed with a bit of extra effort? With python, golang, and others the standard library should cover most cases

bArray3• 5 years ago

Like, why don't we just let projects be "done"? Things don't need to be maintained and updated for eternity.

This is generally why I opt for "single-file" libraries that do one simple task well. The smaller the library, the more likely it is "done". For example, do I want some insanely complex image library that handles every file format under the sun, or do I just want some basic one that allows me to output a simple JPEG?

I often find myself referring to "single_file_libs" repository: https://github.com/nothings/single_file_libs

Looking at the open issues, it doesn't appear to be actively maintained but it's still an incredibly good resource for "completed" projects.

bjterry• 5 years ago

Looking at the open issues, it doesn't appear to be actively maintained

I'm not sure if this is intentionally ironic or not, but it does seem like if your small libraries aren't getting updated regularly because they are done, you at least want the meta library (in this case the single file libs repository) to be updated regularly with new small libraries.

bmicraft• 5 years ago

Funny you chose to say jpeg, because jpegxl is just on it's way to getting big. If you're dealing with a lot of images the savings are probably big enough to warrant an update.

There really is nothing that can be considered completely stable.

lr4444lr• 5 years ago

Well, bugs do get found, and continual new demands from humans make more optimizations necessary. SemVer is the compromise between stability and continual improvement, and I think it works pretty well, all considered.

z3t4• 5 years ago

and Windows is kind of known for bending over backwards to maintain compatibility with old programs

Maybe they used to, but good luck running anything made for Windows XP on current version of Windows. But you will have a better luck if you run it with Wine on Linux.

mb_722• 5 years ago

I have an application I developed for a client starting in 2005 using Delphi 7; my VM is based on Windows XP, and the app worked fine on Windows 7 and is now used on Windows 10. I had to update one setting in an external configuration file for slight differences in the ADO driver, but other than that I haven't rebuilt the executable for years.

I was genuinely pleased for the client as they are nice people running an essential service on a budget that is always under threat, and this result means not needing to pay for a silly 'fresh' version of the application.

derefr• 5 years ago

Packaging your software into a container-image accomplishes almost exactly the same thing, without the need to be using a stack that can create standalone static binaries.

I would point out, though, that even when you can just continue to use a single pinned version of a Docker image forever (because the program itself is stable, or is part of a stable system that only uses the program in precise ways), people still value regular releases, and see non-upgrading images as “rotting” — because they can contain vendored deps, and those deps can have security vulnerabilities discovered in them over time.

hsn91510• 5 years ago

I haven't worked a lot with Docker or similar containerization tools, but my impression is they require you to have an internet connection to download stuff.

At least with Docker, when you write or download a docker file for the first time, it's just a description of stuff that needs to be downloaded and maybe commands that need to be executed.

This kind of thing is qualitatively much worse than a statically linked executable binary in terms.

a stack that can create standalone static binaries.

Yea, a language that has the notion of a compiler is a MUST if you actually care about this kind of thing.

dehrmann• 5 years ago

Docker is especially bad for this. A jar file that runs today will probably run fine in 10 years on future Java. The container might, but it almost certainly has a huge number of security vulnerabilities, and maybe no one uses AMD64 and ARM anymore.

e3bc54b2• 5 years ago

Anything that connects to or supports something that connects to the internet eventually gets used as some part of vulnerability, and hence needs to be updated.

But, I agree with your sentiment. Endless feature bloat is not necessary for vast majority of software. The expectation that every software must remain updated forever is just not sustainable.

amadeuspagel2• 5 years ago

People think if a project on github hasn't been updated since 3 months then it has been abandoned!!

This a heuristic that generally works. Most projects that don't have recent commits really are abandoned. And when you open a project on github, one of the first things you see is when every file was last updated. Maybe github should offer other heuristics, or of it already has them, display them more prominently.

hsn91510• 5 years ago

I don't necessarily disagree, but this is the cultural problem I'm sort of pointing the finger at.

tjpnz• 5 years ago

People think if a project on github hasn't been updated since 3 months then it has been abandoned!!

People are too quick to assume that all Github repos are meant for public consumption. Can only speak for myself really but most of what I've pushed there was designed with only me in mind. If others find it useful that's nice but don't expect me to accept pull requests.

TedDoesntTalk6• 5 years ago

This is precisely why I have a private git repo.

ValentineC• 5 years ago

I am suddenly reminded of this earlier discussion:

https://news.ycombinator.com/item?id=28263721

david_allison• 5 years ago

Have you considered a "no maintenance intended" badge on those repos?

https://unmaintained.tech/

Osiris• 5 years ago

I have a side project that I haven't updated in years. I haven't needed to because Windows is fantastic at backwards compatibility.

Also the website automatically handles payments and sending license keys.

The only work I do is when PayPal integration breaks because they make some minor change on their end.

Unfortunately, Windows 11 removed the IDeskBand interface I rely on so I might actually have to do some work.

strawhatguy2• 5 years ago

Curious, do you limit yourself to the C win32 api on Windows? I hear that low of level never really changes, and you can do all the stuff the newer windows gui libs can do.

3pt141593• 5 years ago

Unless you're doing something mathematical, where you can exhaustively test every input and output and ship something that's essentially close to the platonic ideal of the thing, you're going to need to update.

Protocols change, programming languages change, human languages change, boarders change, definitions of time change, laws change, sensors change, and on and on.

It's like asking why a person can't be just done learning so they can live their life. Well, they can, but pretty soon the world is inaccessible to them. They can't use a computer or a phone because they stopped learning in 1995 and they're utterly dependant on others to do things for them.

But I will say this, I've long been thinking that there should at least be a programming language and OS that does its best to not change. A sort of whole environment where every single piece that's out of beta commits to minimal interface changes over time. Fixing security bugs and supporting new emojis, fine. We have to do stuff like that. But everything else is just as frozen as can be. It would be useful for super long term software. If we have buildings still around from 100 years ago we should be able to build for 100 years from now without a team around for constant maintenance. Though I do not think it can ever be maintenance free.

strangetortoise• 5 years ago

But I will say this, I've long been thinking that there should at least be a programming language and OS that does its best to not change. A sort of whole environment where every single piece that's out of beta commits to minimal interface changes over time

At the risk of this being interpreted as trolling; don't we have this, for crucial OS interfaces at least? Linux is famous for "we do not break user space" (yes, I know it's not 100% true, but it's closer than anything else I've seen), and afaik the posix standards are pretty stable too.

The C language also has a stable ABI, which is basically the ffi for most languages, and most static libraries that were developed decades ago can link just fine.

In my mind, the problem is caused by at least two things:

- Languages that are not compiled to machine code have no incentive to have a stable API or ABI. This has the effect that code reuse now necessitates replicating the state of the machine it was developed on (Which might be using a different language version, runtime, etc).

- Programming culture has progressed to a "get it done quickly, just grab a library" culture. This is not to say you should develop everything in-house, but on a spectrum, I have the hunch that this easy accumulation of dependencies induces a culture that does not vet the stability of the dependencies. Once one of your dependencies is unstable, the project on which it depends cannot be stable.

estaseuropano• 5 years ago

No building stands for 100 years without maintenance. 100 years ago heating and light by coal or gas were still common, so every single old building has had upgrades to electricity (or first gas, then electric). The roof leaked, the wood rotted, a bunch of people died in a fire and now all buildings in a similar style need to add a fire escape...

prox• 5 years ago

We perhaps need a LongNow Smartphone.

jcelerier2• 5 years ago

But I will say this, I've long been thinking that there should at least be a programming language and OS that does its best to not change. A sort of whole environment where every single piece that's out of beta commits to minimal interface changes over time. Fixing security bugs and supporting new emojis, fine. We have to do stuff like that. But everything else is just as frozen as can be. It would be useful for super long term software. If we have buildings still around from 100 years ago we should be able to build for 100 years from now without a team around for constant maintenance. Though I do not think it can ever be maintenance free.

you can fire up QBasic on dosbox just fine

sagichmal• 5 years ago

Unless you're doing something mathematical, where you can exhaustively test every input and output and ship something that's essentially close to the platonic ideal of the thing, you're going to need to update.

You only need to update dependencies that interact with things that change autonomously.

For the types of programs I write, that describes very few, sometimes none, of my dependencies.

Continuous updating of dependencies is valuable in some circumstances. Not all.

brian_cloutier• 5 years ago

But I will say this, I've long been thinking that there should at least be a programming language and OS that does its best to not change.

Believe it or not this is a core design goal of the Urbit ecosystem. The core bytecode should be simple (and stable) enough that some future archaeologist can implement an interpreter in a weekend and run programs. They take it so far that their version numbers run _backwards_, with the idea that once they reach version 0 nothing will ever change again: https://urbit.org/blog/toward-a-frozen-operating-system

FpUser• 5 years ago

"Protocols change, programming languages change, human languages change, boarders change, definitions of time change, laws change, sensors change, and on and on."

I have 17 years old Windows desktop product that I've long abandoned. But it is exactly this: single exe with everything compiled in. It uses DirectX 9 and Directshow. It still runs fine without any tweaking. It even compiles fine from source code. So from a practical standpoint it is maintenance free.

cxr2• 5 years ago

there should at least be a programming language and OS that does its best to not change. A sort of whole environment where every single piece that's out of beta commits to minimal interface changes over time

A passable implementation of Lorie's UVC concept[1] is sitting right in our lap, it's just hardly ever put to use in this way, and it's considered by some not to be very sexy—or rather, it's popular to dump on it for various reasons. But you're probably using it right now to read this comment, and the reason is that its availability and reliability is a foregone conclusion by every stakeholder in this conversation—whether that be me, you, or our hosts involving Y Combinator &co. And with respect to the article, it's a foregone conclusion that it will also be involved in the tech tree of the example projects given. Jeff even mentions its longevity explicitly—in a comparison to Python, where Python is considered the worse of the two. How about just not depending on Python at all?

If you click through to Jeff's "makerss.py" script and look at the kinds of machinations that it does, consider whether this stuff could instead be taken care of directly in a "README.html" in the repo root. No implicit[2] assumptions about what software is installed on the machine in question—since it's using the UVC after all—and none of the Python 2/Python 3 fuss. (It'd have to forego the symlinking that makerss.py does near the end (currently line 1717 and 1719), but that should arguably already be handled elsewhere, anyway.)

1. https://news.ycombinator.com/item?id=11512892

2. https://news.ycombinator.com/item?id=22991033

closeparen• 5 years ago

Software would be incredibly "low upkeep" to last as long as HTTP. You don't have to use a new programming language just because it exists. Time zone info is a data file managed by the OS and has no business being baked into specific programs. Translations are likewise data files and should be modular so that you can add or replace them without invalidating the program. Sensors change much slower than software; the oldest and crustiest operating system image you'll ever meet is the one coupled to a multi-million-dollar sensor in a lab or a hospital.

nobody0• 5 years ago

"Art is never finished only abandoned"

speed_spread2• 5 years ago

"Klingon software is not released, it escapes"

oakfr2• 5 years ago

I evaluate the liveness of a github project based on the responses to questions (latest post, time to reply). Not by when the last change was made.

nonoesp• 5 years ago

It'd be useful to have a way to tell visitors this project still works without having to push commits to refresh the last updates timestamp.

Buttons840• 5 years ago

I do this in the README. Include a "state of the project" section.

Seeing no updates in 3 months is one thing. Seeing no updates on a project that describes itself as a work in progress is another thing.

nly• 5 years ago

Isn't this why we have CI/CD and scheduled pipelines?

fiddlerwoaroof5• 5 years ago

This is one reason I really like Common Lisp and Clojure: both communities have developed a mindset that allows libraries to be done and, as a result, I notice a lot less churn than in other languages.

eigenhombre• 5 years ago

I also like them for the same reason. The languages themselves evolve slowly (Clojure) or not at all (Common Lisp). A library that emphasizes simplicity and does One Thing Well can be stable and continue to be useful indefinitely. There are network effects too -- if my library uses stable libraries and is built on a stable language, it need not change at all.

On the other hand, there are the big libraries that, for convenience, pull one or two small things from several others -- their surface area becomes so large they are practically guaranteed to generate more work in terms of security updates, dependency conflicts, and the like. I experience this at times developing in Clojure, whose hosted nature means issues can emerge from the Java ecosystem, which is obviously enormous and dynamic.

We recently had a defect at work relating to a conflict between transitive (Java) dependencies from a third party library used to serialize exceptions prior to transmission to their service, and another tool used only for generating "literate"-style code documentation. While not a production defect, it cost us some time and could have been avoided (both by us and the libraries' authors) by more hard-headed evaluation of the tradeoffs involved in introducing certain dependencies in the first place.

cubano2• 5 years ago

I would use Occam's razor with this idea and posit that it is much more like that its a lack of overall mindshare with functional programming than with the nature of language abstractions.

Don't tell me that Lisp ever achieved cool-kid status that a thousand different libs wouldn't pop up basically overnight.

randyrand• 5 years ago

Yes, to a certain degree it's impossible to design software that does not exist as a part of an ecosystem, which is why I put it in quotes.

I’m doing a big embedded project and all the code is self contained! The only inputs are physical user inputs.

5faulker• 5 years ago

"Evergreen" might be a word we can use here.

paulddraper3• 5 years ago

But what doesn't change?

Network protocols? No, HTTP/3 is here.

OS? Kernel, sure, but everything else no. E.g. most Linux OSs recently changed their entire init system.

CPU architectures? ARM was irrelevant a few years ago. Faster CPU instructions are arriving every year. You say, "okay, well just recompile." Which brings us to....

Build tools? These are a nightmare to make stable in the best of circumstances.

It's all fiction. Code rot is real.

hsn91510• 5 years ago

http is not a networking protocol in the sense I meant. It's an application level protocol. Although you can still write a server that implements only http1.1 and it should still work fine.

A lot of the things that you describe as changing don't really need to change.

Operating Systems? Windows XP was probably the best windows, and it's from nearly 20 years ago. Looking at Windows 10, what exactly does it bring to the table? It's a much worse experience overall.

Compilers don't need to change either.

remram2• 5 years ago

How often are changes in your web app directly warranted by a new init system or HTTP protocol? Does your code directly depend on what init system is running it, or somehow become unusable if it doesn't understand the latest HTTP version?

If yes, just what the hell are you making?

huijzer• 5 years ago

The answer is in Lehman's laws of software evolution. Most systems "must be continually adapted or [they] becomes progressively less satisfactory".

The argument is simple. The world changes all the time and most software interacts with the world. Therefore, software needs maintenance in order to keep up with the changing world.

jlundberg• 5 years ago

Software close to the user have this need for adaption.

However, my experience is that the building blocks for making even large software systems are increasingly more stable and Debian GNU/Linux is a prime example of this.

TeMPOraL4• 5 years ago

Yeah, but the rate of that change makes all the difference.

Consider biological systems: they're constantly evolving, constantly in flux - and yet, all living things we can see with unarmed eyes are fixed in the scope of a human lifespan. Most of those don't show meaningful changes withing lifetime of a human society. Cooking recipes of our great grandparents are still valid today, because fruits and vegetables and meat didn't change that much[0] over the last century. Wood still works the same way it did 1000 years ago.

Being able to change much faster is good - as long as the changes tend to lead in useful direction. Software seems to be changing way faster than it's needed, because it's changing for the wrong reasons - instead of improving the value it provides, it's usually about one-upping competition, or forcing recurring payments, or forcing competitors to waste money (e.g. by introducing incompatibility on purpose). Software industry resembles a rain forest, in terms of both diversity and frequency of senseless murder (er, "survival of the fittest"). But as humans, we've dominated this planet by being smarter than that, by being more efficient than natural evolution.

[0] - Despite being optimized directly by our industrial processes.

buu700• 5 years ago

That's a nice, succinct way to put it. Software that doesn't interact with the world can be finished. Software that does ends when the world does.

I suppose then that a benefit of pure functional programming is that you can statically analyze whether a given piece of software is plausibly "done" or simply unmaintained, depending on whether and to what extent it uses monads.

wpietri• 5 years ago

Like, why don't we just let projects be "done"? Things don't need to be maintained and updated for eternity.

As the Buddha taught, desires are numberless. I don't think I've ever even heard of a piece of software that hit 1.0 and people were like, "Welp, it's perfect, never change a thing." I've certainly never worked on one. Indeed, the biggest sign of project success to me is releasing an initial version and people being shocked, shocked at the obvious features I missed. And I love it, as that's proof that they're really trying to use it for their real-world needs.

dheera• 5 years ago

Like, why don't we just let projects be "done"? Things don't need to be maintained and updated for eternity.

I agree with this but

native executable binary file

This is hard because new architectures, instructions, optimizations show up every few years.

1vuio0pswjnm7• 5 years ago

"In my mind, the best software engineering is where you solve a problem once and your solution just works and needs no configuration or maintenance or updates."

https://cr.yp.to/software.html

TameAntelope• 5 years ago

Can you elaborate about this mindset in the context of vulnerabilities and bugs?

grimgrin• 5 years ago

What I'm advocating for here requires that your program be compiled into a native executable binary file, and it must embed all its library dependencies (aka static linking).

the super example of this is justine's project

https://github.com/jart/cosmopolitan

it's pr fucking cool

Cosmopolitan Libc makes C a build-once run-anywhere language, like Java, except it doesn't need an interpreter or virtual machine. Instead, it reconfigures stock GCC and Clang to output a POSIX-approved polyglot format that runs natively on Linux + Mac + Windows + FreeBSD + OpenBSD + NetBSD + BIOS with the best possible performance and the tiniest footprint imaginable.

pirate787• 5 years ago

Related to this -- many popular packages churn through changes for change-sake. For example LTS on Symfony is just 3 years and on Doctrine, a single contributor is renaming a bunch of standard PHP/DBAL functions for no serious reason other than feels. It would be nice if LTS was more like a 10 year commitment but I know that's not fun for developers and it is a serious drawback to a lot of open source projects. The frameworks have to change because PHP is changing and then you get a stack that's broken and needs multiple upgrades and major refactoring.

jagged-chisel• 5 years ago

No one wants to commit to 10yrs because of income concerns. Software developers tend to not be business folks. I'd give a 10yr commitment on a project that was paying me and required minimal maintenance.

jbjbjbjb• 5 years ago

This is a pet peeve of mine too. It helps to have a small public api and encapsulate the parts we don’t need and to just wait until the api is stable before going to v1.

userbinator• 5 years ago

It would be nice if LTS was more like a 10 year commitment but I know that's not fun for developers

I'm surprised that some developers think constant breaking changes is "fun".

Symmetry• 5 years ago

Maybe from a "If it hurts do it more often" sense.

brundolf3• 5 years ago

Re: dependencies, for "low-upkeep" projects I also like pinning specific patch numbers instead of doing any kind of semver pattern

Also +1 to using files as a DB for small-scale projects. Something I've done a few times is keep a single in-memory JS object as the source of truth, and just write it to disk as JSON on a certain cadence/read it from disk at startup. It's wonderful having your little one-off server not need to be a distributed system from day 1.

y4mi2• 5 years ago

SemVer was specifically for libraries. It's really pointless to adhere to it if your code won't be imported into software projects of other people.

brundolf3• 5 years ago

I meant for my dependencies, so they don't automatically change underneath me, even in ways that are harmless according to semver

mst3• 5 years ago

The "essentially no dependencies" thing always fascinates me.

Certainly, complex C code is a risk across OS versions but ... perl5 version 8.1 was released just over 18 years ago and after multiple years of annual major releases of perl we're now on version 34. Many of my chosen cpan dependencies produce the exact same output when on a 5.8.1 install as they do on a 5.34.0 install, and I can move them to a new machine/OS via rsync or tar of the local::lib structure (virtualenv like thing configured by setting a couple of environment variables) and it Just Works.

Am I weird here? Is the author? Is python unusually bad? Is perl unusually good? This all seems quite strange to me.

oakfr2• 5 years ago

I am with you here. That being said, I believe the author refers to « weaker » libraries (higher-level, more recent etc. )

liveoneggs• 5 years ago

perl is unusually good in this regard, as I'm sure you already knew.

mst3• 5 years ago

I do suspect that, but my fingerprints are on enough of cpan these days that I would worry about bias if I simply asserted it.

TedDoesntTalk6• 5 years ago

I guess it’s the language maintainers who decide if they want to keep backwards compatibility. And you can’t even say that’s due to language age, necessarily. PHP 8 (Nov 2020) had many breaking changes over past PHP versions and that language is from the mid 1990s. On the other hand, Java also dates to the mid 1990s and code written for Java2 still compiles and executes in Java 17 (2021).

mst3• 5 years ago

Java is not my cup of anything but I do absolutely respect that about it.

I have a long standing semi-shitpost opinion that if faced with a choice of technologies, it's often best to pick the one that's been "declared dead/dying" because that means it's stable and in wide use.

ziml77• 5 years ago

Maybe some code does, but I've run into JARs that I've had to grab an older version of the JRE to run.

areyousure• 5 years ago

I have noticed the same thing regarding Python and Perl. It's sometimes difficult to get the same program to even run on adjacent minor versions of Python. There are multiple xkcds about sorting out your Python environment: https://xkcd.com/1987/ https://xkcd.com/2510/ (see title text)

Meanwhile, I have Perl programs deployed that have been running with no changes for over 15 years.

samsquire• 5 years ago

I find that my old JavaScript projects and tools such as Jsbin break whenever dependencies are upgraded or I update to the next version.

My python code without dependencies just works. My ruby code breaks all the time. My Java code never breaks but my Java code has very few dependencies.

I wish there was a way to abstract the interface of mechanism in a promise that other people can implement and evolve and outsource decision making to someone else who has more skill or nuance in a way that is looser and more reliable than a particular API.

APIs want to evolve over time but a promise to fulfil a high level goal is different. An API is a mechanism like a particular bottle whereas a bottle mould captures the essence of a bottle but is not a bottle itself. Interfaces (such as in Java) are just APIs themselves so they don't really solve the problem. For example, I want to specify routes that correspond to the URL of my React app or I want to encrypt data with the most secure algorithm. These two example mechanisms evolve all the time. I kind of want to abstract the intent and input. An API doesn't have to be a function call, it can be a data structure. The mechanism doesn't matter to me, but I want the mechanism to adapt with evolution. How do you decouple the method/mechanism with the goal? Evolution in nature tries to find a variable to mutate that can produces better survivability. Like higher body temperature in cats which must have an evolutionary advantage for the mutation to have survived.

amadeuspagel2• 5 years ago

I don't like the solution for comments. I think there's real value in having a discussion that is directed at and moderated by the author. For example, if an author writes a post that people on the forums from which he aggregates comments generally disagree with, I don't want to see one comment after another denouncing him. I want to see comments from people who at least respect him enough to write a comment on his blog, directed at him.

Would a wordpress blog really be harder to maintain then this? If so, I think a better solution would be a mailto link, so that the email contains some basic text like this:

My comment on blog.com/article3:
...

Then maybe you could automate adding the email as a comment to the article in some way.

marcosdumay• 5 years ago

If you accept comments, you will have to deal with spam. That is a very-high maintenance problem.

jefftk4• 5 years ago

I've considered building my own comment system, and I think I could do it in a low-maintenance way. The main reason I haven't is that I actually like the pattern where discussion happens on facebook/lesswrong/hn/etc. I get wider and more interesting feedback that way.

haolez• 5 years ago

A few years ago, I made an extremely simple chat bot for Telegram that gave artists access to Spotify For Artists' API. I've used a simple Lambda and that's it.

I've never touched it again and I've found out recently that I now have a user base there and everything still works :) my longest living software ever was made in a couple hours. A little frustrating, but awesome nonetheless.

Havoc• 5 years ago

One of the reasons the whole server less philosophy appeals. Stuff can still go very wrong but at least there isn’t a server that is your problem when things go sideways.

015a• 5 years ago

I've been at my current company for maybe 6 years now, in an engineering role.

In that time, the service-oriented code which has significantly outlasted anything else, with nearly no maintenance, has maybe surprisingly or unsurprisingly been: AWS Lambda functions.

We have quite a few Lambda functions (written in Node) that I believe pre-date even my tenure at the company; the only modifications in the interim, maybe a few lines of obscure bug fixes, updating the lambda node version, we switched from JS to TS everywhere a few years ago, and keeping the CI updated on those repos (which is a rather ironic and wasteful investment of resources given they've only changed a half dozen times in something like six years).

Checking in on those dozens of functions handling millions of invocations every month, and seeing them just chug along with no maintenance, has convinced me, personally, that if a problem domain can be solved in a FaaS-like way, its the way to do it. Not every problem can, and I'm not specifically advocating for Lambda vs Azure FaaS vs an open source kubernetes-based solution.

More-so that, every deployment artifact has to draw an "operations" line, to the left of that is the rote, daily operations banality, and to the right is the ever-changing business problem space. The further right you can draw that line, to push as much as possible into operations, the better; it gives you optionality to outsource that left part, maybe to automated software, maybe to AWS, maybe to an in-house operations team. FaaS draws that line roughly as far right as I think a general purpose programming system can; further right would approach things like Excel, Retool, etc.

MathMonkeyMan• 5 years ago

I think the answer lies in your last paragraph, but your comment made me think: Perhaps Lambda functions are resilient because they're:

- scope constrained

- surrounded by APIs only (e.g. no cheating by having a whole POSIX system)

- infrastructure is maintained solely by a giant company that you pay to not have to think about it

I haven't used any "serverless" stuff, so I don't really know.

Per the article's recommendation, I have noticed at work that resilient software is often a 400 line python script with no other dependencies; or, a medium sized piece of C++ infrastructure that depends only on the OS, the compiler, and a hand-written build system.

Hendrikto• 5 years ago

hand-written build system.

Please don‘t. It will be fine as long as a project is very small, but get out of hand quickly. Rewriting the build system after the fact will not be budgeted and likely never happen.

codesections• 5 years ago

I've been at my current company for maybe 6 years now, in an engineering role. In that time, the service-oriented code which has [lasted the longest is] AWS lambda functions. We have quite a few Lambda functions (written in Node) that I believe pre-date even my tenure at the company.

Interesting, AWS lambda only launched in November, 2014[0]. So those more-than-six-year-old functions must have been written for a brand new platform.

[0] https://en.m.wikipedia.org/wiki/AWS_Lambda

bombcar3• 5 years ago

Things that get rarely touched but depend on external systems are some of the best candidates for extensive and continuous test suites - as they’re most likely to silently break.

Chyzwar2• 5 years ago

AWS Lambdas are bane of my existance. Many of issues I was involved over years*. From issues with performance, deployments, cold start, api gateway integrations, arbitrary limits or just random failures. Debuging is difficult and whole thing is dog slow. Even today I have two outstanding issues that I need to investigate, lambdas fail to connect to redis and pino not logging correctly response times.

Some of that can be blamed for misusing lambdas, but I hate FaaS concept. AWS ECS, google app engine and k8 where you deploy container are imo better and give you more control.

Edit not hundreds but probably close to 70 in total.

dd444fgdfg• 5 years ago

they can be quite usable with something like the Serverless framework.

really though, in my opinion that should be a part of the core AWS solution. Lambda itself is otherwise "low level".

rapind2• 5 years ago

I've been moving more expensive operations into postgresql functions, Initially writing them has taken a good chunk of time, but they just kind of do the job, much faster too and with low to zero maintenance so far at least.

I don't think I can only credit postgresql functions with the low maintenance though. I suspect that the additional overhead / time to write them has forced me to really make sure they are written correctly (and yet I'm sure there's a lot of optimization I could still do).

brian_herman• 5 years ago

I noticed on one of the files he uses lxml, that is not part of the standard library is this one of those do as I say not as I do?

https://github.com/jeffkaufman/webscripts/blob/b6a004790b464...

jefftk4• 5 years ago

When I started this script ~12y ago there wasn't any good html parsing option in the standard library, and that was worth taking a dependency for. I don't see a conflict there with "minimize your dependencies and limit them to ones that value backwards compatibility"?

ChrisMarshallNY• 5 years ago

I've written SDKs that have lasted for decades.

Number of reasons why. Most don't stay static, though. People mod them. They need to be something that can be handed off to a new team.

I find documentation is key.

I write about that here: https://littlegreenviper.com/miscellany/leaving-a-legacy/

bogdanoff_2• 5 years ago

I wonder if there's a way to do server-side javascript programming with this philosophy.

For client-side I can just throw together some html and javascript to make a simple app. But if I want to use javascript on the server it seems like I need to download a bunch of npm dependencies to do anything useful.

WA• 5 years ago

Sure, you can do it on the server with Node. Node has `http` as part of its standard library and you can write a simple webapp without third party dependencies. Check this tutorial for example [0].

The problem is that Node's http lib isn't super productive and you gotta write some "solved" stuff like cookie handlers. Also, there is no database standard library. If you want to persist data, you could use files. If you need a database (even SQLite), you gotta use third-party stuff.

In comparison, PHP has a lot of stuff build-in and you can be more productive with zero dependencies in PHP.

But then, maybe it doesn't really matter if you include a well-maintained npm package or if you have a more powerful standard library that has some features deprecated after a while. It's still "other people's code", although npm packages require some kind of installation, whereas a standard library usually doesn't.

[0] https://www.section.io/engineering-education/pure-node-js-no...

tored2• 5 years ago

There was a trend a few years ago (maybe still going) that languages with a large standard library, like PHP, is bloated. That we should strive against minimalism and then just share dependencies when needed. But then the nodejs left-pad debacle happened.

I used to be more on the minimalist side, but over time I have switched to the opposite, today I can really appreciate the bloated nature of PHP, everything from large projects to small scripts just becomes easier to handle.

Karrot_Kream2• 5 years ago

I follow a lot of these practices myself for hobby code, but if I had to offer one change, it would be to use SQLite instead of the filesystem for data. It does add some complexity in languages like Go that don't "easily" link with C libraries, and you do need to remember the table schemas that you're using, but for me the benefits are worth it. I never worry about locking files, naming some key badly, or any of the other issues that come with flat file dbs.

bob1029• 5 years ago

SQLite is an absolute powerhouse. Filesystem interop is the best way for it to start proving itself in any codebase. Loading assets for a AAA game straight from disk? You would have a really hard time doing better than SQLite - especially if you have millions+ of unique items to contend with. In many cases, SQLite is faster than the actual file system it resides upon [0].

We use SQLite for nearly 100% of business object persistence in our product today. The performance & scalability arguments fall on some very deaf ears after years of experience optimizing and succeeding with this stack in production.

  [0] https://www.sqlite.org/fasterthanfs.html

phendrenad2• 5 years ago

Good advice for personal projects. Sadly you can't use a lot of free libraries out there, because more often than not you'll be forced to upgrade (possibly due to a security vulnerability that the author refuses to backport to "old" major revisions), and encounter a backward-incompatibility when upgrading.

TedDoesntTalk6• 5 years ago

This is not how I approach design at work, or necessarily how I would recommend doing so. Having a budget where initial creation is essentially free (fun!) while maintenance is extremely expensive (drugery!)

titzer3• 5 years ago

The first backend that I achieved self-hosting of Virgil was the JVM backend. I targeted version 5 classfiles. These classfiles are still accepted by JVMs but there have since been several significant additions to the format. In particular, classfiles are required to have stackmaps, which are pretty much a major distraction for me to add to my backend (so I haven't!).

The second backend that I bootstrapped on was x86-darwin. That worked great for about 7 years, until Apple decided to deprecate first 32-bit, and now, I guess, x86 altogether. Thankfully I ported to x86-linux soon after, as that syscall ABI is solid as a rock.

But you know, things keep changing. To keep up, I finally had to bite the bullet and write an x86-64-linux backend. And I guess at somepoint now arm64 if I ever want to work natively on a mac again.

It's an endless cycle. Thankfully now, emulators are pretty good, but I am still struggling with getting QEMU to run fast on MacOS on the M1, so it's just more futzing with enormous emulation stacks just to run old stuff.

Datenstrom• 5 years ago

Beautiful code denies its existence so perfectly you forget you wrote it.

gfxgirl2• 5 years ago

The biggest piece is minimizing your dependencies, and limiting them to ones that value backwards compatibility

Three.js, one of the most popular JS libraries, has the explicit policy of ignoring backward compatibility.

https://github.com/mrdoob/three.js/wiki/Migration-Guide

I find it not only irresponsible but arguably mean to other devs time and lives but apparently I'm in the minority.

Every time I have to spend an afternoon or evening updating stuff instead of doing something new or visiting friends etc I silently curse the devs

jefftk4• 5 years ago

You curse the devs who build excellent libraries and give them away for free, because they do not accept the additional burden of never making changes that would require you to adjust your usage?

You could choose not to use such libraries, but as long as they're up front about it I don't see how you can hold it against them.

gfxgirl2• 5 years ago

it upsets me because they are the only library of 100s I use that do this. They consciously break things every month. they don't give a fuck how many man hours it burns of other people's time. No other project I know of does this.

It's not just time using the library. Every volunteer that wrote docs or a tutorial their work is now out of date. every answer on S.O. is more than 3-4 months old is wrong. Every new user now has the burden of trying to learn where any content they find is practically guaranteed not to work correctlt with the current version.

it's mean in the extreme to piss on so many people

fulafel3• 5 years ago

I'm sure you know this but for the record, they don't actually ignore backward compatibility, it features often as a concern in the dev discussions and they're conscious of when they break it, and there can be deprecations[1].. it's a conscious tradeoff allowing development to go faster.

I'd argue Three.js beats native to make low upkeep & long lived end user (=portable) 3D apps, given all the churn, constant driver breakages necessitating new workarounds, proprietary-ness and fragmentation in 3D APIs. With Three you update to newer versions on your own time (if ever) and stuff doesn't break from under you. And in the decades long view WebGL 1 / WebGL 2 / WebGPU is used behind the scenes.

[1] example: https://github.com/mrdoob/three.js/pull/21777

AtlasBarfed2• 5 years ago

Well, it probably starts with a stable secure cross-platform language and ecosystem.

Things like this are best explained by a very grey neckbeard who's been through a lot.

Author is not particularly old, author programs in python (something that has broken compatibility once)... "stick to the standard library" is about at ground level an opinion on this subject that a typical HN probably thought of as they clicked the link.

1) need a stable, mature, secure language

Stable eliminates Rust, Python, Perl. Go hasn't been around long enough.

Secure eliminates C, C++, probably D and most compiled languages

No dependencies: yikes there goes Java because of the JVM dependency.

Assembly isn't portable.

...Seriously, are we left with Ada? Secure, strict, mature, reasonably portable. And virtually unknown outside of government/military.

2) UIs age, but CLIs really don't

So make it a CLI, and use unix philosophy

3) OS churn

Hopefully with the practical cessation of gigahertz scaling, the industry will settle down. Then again, the ARM hordes are approaching to wipe out all the x86 stability in desktops and servers.

Perhaps the best thing for this in the industry has been Windows subsystem for Linux. If we could get an OSX subsystem for Linux, that would be a real boon.

Even Linux on it's umpteenth release is still changing quite a lot, and NIH reinvention like Wayland and Systemd and toss-it-all-out Gnome 3. Well, it kind of keeps us employed. I'd adhere to assuming a unix-ish os, like maybe POSIX.

So: Ada, CLI/text, POSIX compatibility. That's not limiting at all, is it?

Sigh.

jhgb3• 5 years ago

1) need a stable, mature, secure language
Stable eliminates Rust, Python, Perl. Go hasn't been around long enough.
Secure eliminates C, C++, probably D and most compiled languages
No dependencies: yikes there goes Java because of the JVM dependency.

Common Lisp, then?

AtlasBarfed2• 5 years ago

Almost, but Java has a VM. Yes you can technically compile Java to an executable but it's not done much. CL has a VM too, does it not?

anyfactor• 5 years ago

I think the idea of we need continuous improvements to maintain a steady userbase is a myth. And it is disturbing to me that we talk about browser support and the global reach internet services yet I struggle to access many popular websites on my galaxy S2.

karmakaze2• 5 years ago

The most important things I've learned is that the system has to be (1) self-maintaining (e.g. security updates, rotate logs, alerting monitors so safe to ignore), (2) super easy to make and deploy a change. This one consists of a README that word-for-word works so you don't have to think. Minimizing steps with push-to-github auto-deploy is great, as is an easy setup for a hot-reloading local dev.

Following those two points, I've been able to use whatever languages/tools tickle my interest with little impact on maintenance effort. Dependencies are few, but not minimal, pinned to exact versions. FYI, these have run the range of Vue/TypeScript, Node, Go, Java/Kotlin, Clojure, F#, and Php, with typically MySQL (sometimes multi-master).

shoo• 5 years ago

(2) super easy to make and deploy a change. This one consists of a README that word-for-word works so you don't have to think. Minimizing steps with push-to-github auto-deploy is great

Great advice. Minimise the energy barrier to being able to ship a patch or a new feature after neglecting the service for months/years and forgetting how you set everything up.

I always have a bit of a laugh after patching my hobby project and doing a handful of iterative automated deploys while making small changes, then switching back to the day job where aspiring to do a single deploy will typically require hours of meetings and general bureaucratic overhead.

Another possible tip is: understand your deployment environment and learn how to take advantage of what it offers out of the box without adding additional layers or dependencies. E.g. I deploy a hobby webapp to my own debian server. Debian comes with apt for package management and systemd for managing services, so I use both of those (yes, even apt for python package dependencies).

karmakaze2• 5 years ago

Totally agree. Systemd .service unit files are super simple to understand and setup. Well worth the small time investment.

unknownOrigin• 5 years ago

Designing for minimum maintenance is nice when you're working on something for "yourself"... not so much if you're creating something for someone else. Since designing for minimum maintenance takes additional time (and money) and most stakeholding entities (contractee, employer) will want the job to be done as soon as possible (and only THEN worry about upkeep), this can easily prevent you from "doing the right thing" simply by the nature of competition. Not to mention the elephant in the room - that minimum maintenance often means less money for the person doing the maintenance, i.e. "you". Vendor lockin is an issue only if you're not the vendor.

cblconfederate• 5 years ago

php+mysql has been low upkeep for at least 15 years is that enough

milofeynman• 5 years ago

An interesting take-away from this is designing low upkeep business services. I have been dealing with trying to onboard to a service at work, and there are so many dependencies! Several are pinned to versions that require python2, but I can't install python2 because another service I use calls "python" and assumes it's linked to python 3!

And sometimes someone will add a new feature, and include an entire library for that single method call, instead of trying to write a little JavaScript.

I almost think there should be a non-trivial process at most businesses for including new libraries so that web devs would try to do it without adding a new library, so they didn't have to deal with red tape!

forgotmypw174• 5 years ago

Lindy Effect says that software will continue working for as long as it's already been working so far.

I use this to my advantage by writing software AS IF it has already been working for 20 years, meaning I use only languages and technologies which have not introduced breaking changes in 20 years or so.

It's worked out quite well so far!

est31• 5 years ago

Which language or technology has not introduced breaking changes in the last two decades? Windows has good backward compatibility, but even Windows can't run really old programs. C++ has evolved dramatically in the last 20 years. Even C had changes. Do people still use tables to create websites, or jquery?

forgotmypw174• 5 years ago

Now, to be fair, I'm talking about carefully writing code with the knowledge of the previous 20 years, which is a little bit like "cheating", whatever that concept means here.

I use ASCII/textfiles, HTTP, CGI, SSI, access.log, Perl, Bash, and small subsets of PHP, HTML, JS, CSS, and almost no third-party libraries. Certainly no JQuery or anything similar.

I'm not hardcore enough for C or C++, but I know that a program written 20 years ago in these languages would most likely compile today.

rapind2• 5 years ago

SQL. Well mostly.

tomerbd• 5 years ago

Sounds good for personal projects.

oftenwrong• 5 years ago

On this topic, what happened with https://www.jefftk.com/nextbus/ ? Seems to be a 404 now. Was there a breaking change or disruption in the nextbus API?

I loved its bare-bones UI. It was definitely one of the most usable webapps. I assume its a11y was nearly perfect as well.

If anyone would like to see it, here's an archived snapshot:

https://web.archive.org/web/20170622033104/https://www.jefft...

I also admire https://diskprices.com/ for the same reasons.

tored2• 5 years ago

Few weeks ago I wrote a lengthy post here on HN about how to write low upkeep software in PHP

https://news.ycombinator.com/item?id=28232146

6510• 5 years ago

For fun personal projects I like using php to manipulate static html documents. I have quite a few projects with broken admin areas written in ancient php but (besides from user contributions) the html/css/js/xml/json just continues to work as if nothing happened. Sometimes I just delete the admin tools.

XCSme• 5 years ago

I am not sure writing your own build tools is the way to go, if there already exists something that does that thing even better.(as the author mentioned, he wrote a templating engine that updates all static files when a shared component is edited)

Designing Low Upkeep Software

Related Stories