Programming is hard

https://myme.no/posts/2023-09-19-programming-is-hard.html

by ingve • 3 years ago

121 114 3 years ago

simonblack• 3 years ago

That's why programming is more of a challenge and more exciting than ordinary old computer games.

Games only hold a challenge until the game-writer's secrets are broken.

When I first started out in computers, I thought the games would be thrilling. They weren't. But programming .. trying to pit my wits against the very fast moron who insisted on deliberately doing no more and no less than I told him to ... that was thrilling.

TheRoque• 3 years ago

Games only hold a challenge until the game-writer's secrets are broken.

Games ultimately are code themselves, and you can manipulate it to some extent, creating other infinitely complex interactions (source: advancements in speedrunning or various game challenges like the A press challenge [1])

[1] https://www.youtube.com/watch?v=yXbJe-rUNP8

yetihehe• 3 years ago

Nice observation, never thought myself in such way about programming. This compares nicely with my experience with sudoku. It was fun until I've found and implemented algorithm to solve them.

ninechars• 3 years ago

The end user is the final boss, and your job is to keep them alive.

behnamoh3• 3 years ago

At the end of the day code has no obligation or enforced requirement to resemble the problem it’s supposed to solve

Unless you do language oriented programming (in Racket) or create DSLs.

jongjong2• 3 years ago

After 15 years of software development experience, I've been thinking about this a lot lately. It's like you don't stop learning to be more careful. Even after 15 years, my mind is expanding. Sometimes I find code that I wrote 2 years ago which, at the time, was produced with far more care and extreme pedantic attention to detail than the average developer would have given it, yet it was still not correct for ALL POSSIBLE edge cases. It's insanely difficult to produce correct code. I'm starting to have thoughts like "Has anyone in the world (aside from Knuth haha) written a single program which is over 10K lines of code and which is 100% bug free right now? Does such program exist?" I don't even believe that Bitcoin is bug free; it just has too many lines.

behnamoh3• 3 years ago

which is 100% bug free

define "bug free". In Python, I've been reimplementing monads recently. It's such a shame that Python still doesn't have Null/None propagation. After each obtained result and before using it in the next chain, I have to run a series of sanity checks (is it Null or not, etc.) which should be resolved automatically by the language. Alas, most people don't care.

Something like this would be nice:

    result =? foo(bar :? str)

meaning that foo should receive a str bar, but maybe bar is not str. Then foo should return something, but maybe it returns None (the =? part).

unnouinceput• 3 years ago

Something like that would fail the code review at any large organization. Why? Because ternary operator is hard for maintenance purposes so the reviewer would simply ask to use if..else syntax instead. I gather you never worked in such organization.

benrutter2• 3 years ago

This is an interesting idea- I'm not sure I understand it fully though. Do you have much luck with static type analysis? I find in Python, nothings ever certain, but mypy with good type hints solves a lot of these kind of None based shenanigans.

fiddlerwoaroof• 3 years ago

I don’t know about bug free, but Code Complete claims that the space shuttle project shipped a million lines of code with no bugs in production.

irjustin• 3 years ago

Bug free code isn't pragmatic at all.

For any system that is reasonably complex; that solves a real world issue, there's simply no way to handle all the edge cases.

One thing that I've found myself doing in recent years is to simply let the program blow up and have the end user experience a 500 error. I know this isn't ideal for so many orgs but I've come to the conclusion that the worst thing you can have happen is a feature that breaks but returns a "200".

indemnity2• 3 years ago

Another almost worst thing is having production environments with significantly different stacks to what you are using locally, with branches in the code to account for this.

This message brought to you by a day of commits dealing with edge cases that only manifest in the cloud environment which uses a "strategic" database technology that is impossible to run locally.

And it deploys from one of our old crappy Jenkins pipelines that decided 30 minutes ago to start having a failing volume.

Should have just used GitHub Actions + Postgres.

lelanthran2• 3 years ago

One thing that I've found myself doing in recent years is to simply let the program blow up and have the end user experience a 500 error. I know this isn't ideal for so many orgs but I've come to the conclusion that the worst thing you can have happen is a feature that breaks but returns a "200".

Acceptable for webapps. I can't really think of anywhere else that it becomes acceptable to simply say "We had an oopsie. Try again later".

Mostly what you want to do is give the user an actionable error message - i.e. "This is what you can do to fix this error".

"cd /somedir" gives errors of the form "permission denied" (i.e. change your perms level, or ask the owner of the dir to change its permissions), "not found" (i.e. that dir does not exist, so create it if you need it), "i/o error" (i.e. your disk may be bad, run fsck), "Not a directory" (i.e. specify a directory, not a file), etc ...

With most programs written in HLL these days, all I get is something along the lines of "Operation could not be completed", or, worse an exception handler that has absolutely no clue what is needed to fix the error (a real one I got last week: "expected integer, got string". I have no idea where in the 3k lines of input that field actually is, or what it is called).

The only good error messages I see are written in programs that don't have exception handlers, so the error is printed out where and when it happens, and the programmer who wrote the code that generated the error is also forced to handle it, and so has all the context to generate a reasonable error message.

wiz21c• 3 years ago

For any system that is reasonably complex; that solves a real world issue, there's simply no way to handle all the edge cases.

There's a spcetrum between 'all the edge cases' and 'most of the edge cases'. A web app for a blog is not exactly like an automat onboard the JamesWebb telescop...

myme2• 3 years ago

OP here. This is one of the exact things that, in my mind, makes programming hard. It probably wasn't covered in the post, but striking the right balance between "bug free" and "pragmatism" towards accepting lack of functionality and error tolerance, is crucial.

Within certain domains like healh-care (my area these days), space travel, military, edge case management play a significant role in the valuation of where to place your effort (although it doesn't exclude the ability to fail fast and hard during development cycles). In other domains exposing users for these hard failures is considered perfectly fine. It depends...

It's just one other aspect - when thinking beyond just the code - that makes the decisions and consideration I have to make every day "hard".

benrutter2• 3 years ago

This is so true. I see a lot of bad APIs and returning something like a 200 response + a message body saying something like "Psych! Actually it'd broken" is a surprisingly common anti-pattern.

Reminds me of Petet Gabriel's Worse Is Better[0], one of the insights of which is that it's often better to fail for edge cases than try to handle every single one.

[0] https://dreamsongs.com/WorseIsBetter.html

PH95VuimJjqBqy9• 3 years ago

As long as you're fixing and preventing the 500 in the future, this is how you build stable software.

it's not possible to predict everything, so instrument, let your software be loud, and listen to it.

lmm• 3 years ago

Somewhere Edward Kmett has a great post about how he realised he was just writing buggy code that would have to be thrown again away and away, and that was the point where he switched to writing Haskell libraries that were perfect and could live forever.

raincole• 3 years ago

Bug-free is more about how narrow the spec is.

Can your GUI app works on a Windows 95 with Polish as the system langauge while using a Chinese input method, and the user holding Alt all the time? Probably not. But the question we should ask isn't how could we make it work, but why the hell do we need to support this use case?

lelanthran2• 3 years ago

But the question we should ask isn't how could we make it work, but why the hell do we need to support this use case?

I don't even think it is necessary to ask that question. The only question that should be asked for something like this is "How do we handle all unsupported use-cases?"

Because then it becomes simpler, when an edge-case causes the software to freeze, to detect for that edge-case and hand it off to the "not supported" handler.

In the best case scenario, we simply detect whether the use-case is supported and then either proceed or hand control to the not-supported-handler.

In the worst case scenario, when the bug report comes in, we don't even bother trying to reproduce it, we just add a general check for it and move on to the not-supported-handler.

In this particular case, the detection can do any one of the following:

1. Check for english, and if not, display "unsupported language". 2. Check that input language and system language match, or display unsupported". 3. Check that the OS is supported, display "unsupported OS" if not. 4. Check that the input is valid (i.e. no special codes, whether by holding down halt or otherwise), or display "unsupported input".

Whether the spec is narrow or wide, the developer needs to do the work to ensure that the user is running within spec.

Not checking results in more "bugs" that are, as you said, use-cases which shouldn't be supported anyway.

63• 3 years ago

I think about this pretty often from the opposite perspective. As a frequent user of software, I can't get over how much it all sucks. Nearly every nontrivial piece of software I've ever used has had annoying and frustrating bugs and it feels to me like that's only getting worse. I agree that there's no correct software and I think the software engineering discipline needs to be blamed more for it. I don't think we deserve these salaries just to churn out broken code. I love programming but you can bet that the second there's an AI that can write perfect code, I'm advocating for it to replace us all because I'm tired of broken software.

pvdoom2• 3 years ago

Well, there is a massive force that acts directly against not doing broken code - people(read managers and stakeholders) don't want to wait, they want the value for their money now, and from their perspective a certain degree of brokenness is fine

ozim2• 3 years ago

What software are you using and what were last 10 frustrating bugs you encountered?

I find it fascinating because my use of software seems bug free in last 5-10 years. It is hard for me to run nowadays into a frustrating bug.

When I was in high school there were just loads of bugs and everything seemed broken it was 2005ish and before. Windows was crashing now and again, had to reinstall frequently, software was crashing taking my work with it so I still have compulsive ctrl-s tick.

jongjong2• 3 years ago

Fully resonates with me. The complexity and bloat of many modern software has gotten out of control to the point that even users end up being victims; in spite of all the additional effort, workarounds and automated testing undertaken to try to shield them from it.

Developers are fallible humans and almost every single developer tends to underestimate the complexity and difficulty of what they're dealing with. Even those developers who are naturally good at coding are nowhere near where they need to be to produce merely correct code. Not outstanding, readable code, just correct code.

Modern software has become an inseparable amalgamation of people and code because the code on its own is just not good enough. DAOs which can operate on their own with minimal to no human intervention are still a pipe dream.

dkersten2• 3 years ago

I recently watched a video about craftspeople who spent a lifetime mastering their particular craft and one was a Japanese calligraphy brush maker who had been working on a particular brush for 25 years. He called it “no compromise craftsmanship”.

It made me wonder what a “no compromise craftsmanship” computer or software would look like. For one, it would never truly get done, there’s always more to do, but then again I guess that’s the same with the 25 year calligraphy brush.

PH95VuimJjqBqy9• 3 years ago

The goal isn't to produce perfect code, the goal is to evolve to perfect code.

This gets missed by so many developers, once you change your mindset on this, you'll find you write better software.

One of the questions to ask when writing code is "how long will it take to discover if this goes wrong?". Make it quick to find, let someone else in the future deal with the imperfections.

I'm not saying don't try for perfect code, I'm saying not having written perfect code isn't a problem and it isn't a failing. code evolves to be stable, it is not born that way.

Aperocky• 3 years ago

Having to translate real world requirements into abstract constructs that – when evaluated by a computer – model our problem domain to a satisfactory degree.

I tend to think of it as starting from abstract constructs and matching it to a known real world problem in an satisfactory degree. From that sense it didn't seem and feel so hard.

In mathematic theory it starts by defining 0 and 1 and the whole thing comes together shortly after and can be used to model the real world to an amazing degree. Software engineering in comparison are still quite straightforward even in its highest complexity.

heroku• 3 years ago

i once bouth chessishard.com not good marketing value.

usrbinbash3• 3 years ago

"Not so much on how we can more effectively understand requirements and translate these into code."

The first step to understanding requirements: There needs to be a requirement worthy of the name.

Because let's face facts: programmers are quite often confronted with "requirements" cooked up by some "stakeholder" that don't even describe the problem domain, much less a path to how it should be modeled.

A lot of the spaghetti that contemporary production code ends up looking like far too often, is programmers having to play a stupid game of "chase the nebulous and ever changing 'requirements'".

Once you develop for A but then B is required as well, and the architecture didn't anticipate that, you may be able to go back and change it ... or not, because usually the requirements change, but the deadlines don't, and good luck doing the same, or more, work all over again, in half the time.

Yes, programming is a hard problem. It really is. But it is also quite often being made harder than it needs to be.

atoav2• 3 years ago

Programming is a lot like design

- the customer tells you they need a design for their product, but they don't know what design they need, just what design they want. Now of course it falls back on you if you give them what they want and it doesn't work well.

- there are many ways to tackle the same problem, many of them work okay, some of them work well and look nice, some of them work well, look nice and are easy to manufacture and maintain. But it also has to be the solution they need and one where you can convince them they want it. Finding a good solution is a lot like finding the biggest common denominator of a large set of numbers.

As someone who worked in design before I can say that the most work and stress for me was bringing the clients up to speed, so I can communicate to them about the designs on one level. That meant explaining to them why their primary-color-powerpoint-comic-sans-poster communicates things they do not intend communicated.

saghm3• 3 years ago

I suspect that this sort of problem is present is a _lot_ of fields; it's not a programming problem or a design problem specifically, it's a human communication problem, and those can make just about anything more difficult.

esskay• 3 years ago

I've just spent the last two days building out a system based on an API created by our clients internal dev team, only to have both the project manager and client throw in a massive curveball they knew about but neglected to mention - the API's been rewritten and replaced with a new one with a vastly different architecture.

That's ~12 hours of work down the drain and one demoralized developer.

It baffles me how this kind of thing continuously happens everywhere. Part of it seems to be that despite how many times you explain it to stakeholders, they don't seem to grasp that development takes time, and altering the plan mid way through is not only incredibly distracting but comes as a mental blow to the developer.

imetatroll• 3 years ago

I think it is more that they don't respect developer time.

PH95VuimJjqBqy9• 3 years ago

which is why you get in contact with their development team directly.

I've seen internal teams that thought the communication channels couldn't go dev to dev and it was a nightmare.

If you're experiencing this, consider that you have options if you're willing to take them.

Scarblac• 3 years ago

But all that is part of programming. It's not made harder than it needs to be, it is that hard. Because exploring the whole problem space with the stakeholders and deciding together what the software should do is a central part of the problem of programming.

And it's ignored by everyone, the stakeholders don't realize it's a problem, the programmers know but think it's not their problem, and nobody includes it in estimates or plans the required time and meetings for it to happen.

dragonwriter4• 3 years ago

But all that is part of programming.

Nah, its systems analysis, a field which is largely either forgotten (in the “cross-functional means everyone is a nobdifferentiated developer” school of Agile teams) or replaced with “business analysis”, whose practitioners have less relevant skills (particularly, systems analysts were experienced programmers with additional process engineering skills, while BAs are nonprogrammers in a role that is often structured more as stenography than engineering) and “software architects” (who have a closer skill set to systems analysts, but have a role with a simulta!eously more abstract and narrower, implementation-focused orientation.)

Systems analysis requires (or at least benefits from) an understanding of programming, but its not the same skill

saghm3• 3 years ago

exploring the whole problem space with the stakeholders and deciding together what the software should do is a central part of the problem of programming

I think the situations that the parent comment is addressing don't really fit the description of "exploring the whole problem space with the stakeholders and deciding together what the software should do"; the programmer is often only looped in _after_ the stakeholders decided the requirements, and if there was any "exploration" done, it was deemed complete before the programmer had a chance to give input. If management actively chooses to work in a way that doesn't facilitate programming being done well, that's not programming being hard; it's just management being bad.

bryanrasmussen2• 3 years ago

that's nice but if the "stakeholders" tell you a particular set of requirements at one point, and then a different one later that's not really anything to do with exploring the problem space.

usrbinbash3• 3 years ago

the programmers know but think it's not their problem

I disagree. We know it's our problem. But we quite often don't get a, or too little, say in the decisions made to manage that problem properly.

Because yes, it would be nice if we would "explore the whole problem space with stakeholders", and it would be great if we could "decide together".

What ends up happening quite often however, is some "stakeholder" saying yes and thank you to whatever he-who-holds-the-money (or his career-advancement) says, with little to no regard whether the promised addon/feature/whatever is feasible and/or accomodated by the existing architecture. Suddenly, the space that needs to be explored may be galaxy-sized, or moebius-strip-shaped.

What also tends to happen: The "exploration of the space" is outsourced to some consultant or architecture astronaut, who wont write a single line of code, and certainly won't have to deal with any unforseen difficulties.

It would also be nice if, once the "exploration of the space" is done, devs get to implement based on what was explored. Which, given the aforementioned unforeseen properties of the problem domain, is laden with unforeseen change as it is. What tends to happen quite often is that in addition to that, requirements, and thus the problem domain change mid project...sometimes multiple times.

PH95VuimJjqBqy9• 3 years ago

But all that is part of programming. It's not made harder than it needs to be, it is that hard. Because exploring the whole problem space with the stakeholders and deciding together what the software should do is a central part of the problem of programming.

strongly agree, but I've found a lot of developers want to make a 6-figure salary and not have to do that part.

Furthermore, often times the stakeholders themselves don't have a concrete idea of what they want, it's nebulous in their head because they've identified a problem and a general direction and are relying on conversations with the technical team to help nail down the details.

That back and forth is the meat and potatoes of software development and its what separates senior from junior.

sancarn• 3 years ago

I feel like communication of requirements is difficult too. I'm currently a business end-user, with developer experience, who is maintaining legacy VBA technology, trying to get an official system produced by a 3rd party contractor.

The requirements started out truly quite simple - we want a database and a nodejs web server.

But Technology are gate-keepers. We're not being trusted with what we want, or with that level of control over the system. Which then starts this cat chasing mouse thing of what the Technology will allow and what the business wants.

I've always vouched for "Internal IT" - specialised developers who work closely with the team, even doing the jobs users do, to self-identify what technology is required and kickstart these projects.

Edit: in this particular project, I handed over numerous diagrams of how the system should be architectured, and Technology BA's came back with a proposed set of technologies (SASS) that simply wouldn't be technically feasible... It's really made me lose hope.

c222• 3 years ago

I think there's something to this. When I write programs to solve my own problems I find the process easy and enjoyable.

sebastianz• 3 years ago

How big are the programs, how often do your requirements change, and how many years or decades do you develop and support them?

Because tiny one time programs are indeed fun for a while until they “grow up”. :)

Hermitian909• 3 years ago

This actually touches on one of SV's durable advantages in the software world: the realization that if you want good software your engineers must be involved in the requirements gathering, design, and scoping phases. Critically, the business understands that engineering is the most authoritative voice on cost and since all business activities should pass through an ROI analysis filter engineering needs to be in the room.

Keeping engineers in the room then adds a positive feedback loop: as they understand the business better they start to offer up new opportunities to the business that have surprisingly low cost.

(I will caveat that for sufficiently solved problems you can generally involve software engineers less)

brabel• 3 years ago

I thought I was going to find this far more entertaining post when I read the title: http://www.stilldrinking.org/programming-sucks

nologic01• 3 years ago

Programming is hard mainly because its open ended. Its not a well defined finite problem. The domain of application keeps expanding and the logic required to organize information flow keeps getting more complex and varied.

It all started as the serial, offline processing of numbers emulating some aspect of the world. Since then you have the major but still largely incomplete revolutions with real time, interactive and concurrent applications.

The metaphors have long stopped being numbers. LLM's pretend to encode human language itself. Our entire semantic universe, to the degree it is externalized, is slowly codified and mapped into bits.

You also have the deepening of the stack. It started with simple machine code but now you can have like four layers of abstraction to help our poor brains manage torrents of information flows between very complex pieces of hardware.

So yeah, programming is hard because we are pushing hard to find the limits of what can be done with digital devices.

I dont think we are even halfway there yet. Its going to get much harder :-)

myme2• 3 years ago

Author here. Thank you for this and from what I can tell you're nailing my frame of mind and where I was coming from with this.

Writing valid computer programs - from "Hello, World!" and the likes - is safe to say "easy". What programming has turned into - and what a large percentage of professional programmers have to deal with on a daily basis - is not.

So it's an article sure to stir up some comments, because we're all coming at it from different angles and people considering themselves programmers fill a wide variety of roles in their companies and in the profession. If people feel they're having an easy time, good on them! But neither feel no shame when the imposter starts creeping in on you or you feel the mountain you have to climb is too tall.

nathias2• 3 years ago

Compared to learning other skills programming is easy, it is made to be optimized for human use, legibility and learning, it has the most resources on learning by far. Yes, you might not be able to create a perfect and complex computer program, but this has more to do with the nature of the world and complexity than programming.

quonn• 3 years ago

Your definition of programming is too narrow. It‘s like saying writing is easy, every 6 year old can do it.

nathias2• 3 years ago

You assume I mean just learning syntax, I don't, I mean the actual skill of developing programs in production.

zelphirkalt• 3 years ago

Learning computer programming is easy ... mastering it is a life-long journey.

yarg• 3 years ago

No, programming is complex.

karmasimida• 3 years ago

It is not hard.

It depends on what do you plan to use programming for. Creative and sophisticated programming is hard, but CRUD one isn't

emmelaich2• 3 years ago

It is hard. And isn't. How hard is it? In number of years / hours. Relative to what?

“When you can measure what you are speaking about, and express it in numbers, you know something about it; but when you cannot measure it, when you cannot express it in numbers, your knowledge is of a meagre and unsatisfactory kind.”

-- Lord Kelvin

user39393822• 3 years ago

So Walt Whitman would express his mastery of poetry by what, the number of poems he's written?

It sounds like something worth considering only in some contexts but is phrased as an absolute.

emmelaich2• 3 years ago

Perfectly reasonable and expected response. I think if you asked Lord Kelvin he would say it meant only in a particular context.

If you said Whitman was more prolific than, say, Frost, but didn't have the numbers then it is an empty assertion.

pixl976• 3 years ago

How can you assign a number to an unbounded problem?

vb-8448• 3 years ago

Programming was hard in the past, nowadays with all high level languages and advanced monitoring/debugging tools programming is easy.

Today, the real challenge is correctly managing and delivering changes and integration with other programs or external systems.

throwaway71271• 3 years ago

For some reason programming is not getting any easier, I can even make an argument its getting worse despite all the improved tooling and andragogy investments we have done.

Programming is in some sort of local minimum and we keep going in circles. Few years ago I watched this talk by Gerald Sussman: "We really don't know how to compute" - https://www.youtube.com/watch?v=HB5TrK7A4pI and it made me question my views on what I think programming is.

I dont know if its because people are afraid to try new things, or there is no funding to try new things, we are kind of stuck. People thinking "don't reinvent the wheel" and for some reason think we have built a wheel (as Casey Muratori says: https://www.youtube.com/watch?v=fQeqsn7JJWA). We have a square at best. There are so many problems in the world, and a lot of them need robust and resilient programs to be solved, e.g. self cleaning public toilets or worm farm that you feed bio garbage and it produces protein flour.

The challenge is exactly the same as it was 80 years ago, manage complexity in chaotic and emergent systems: chaos, complexity and entropy as the poet said. And to be honest, I think we have gotten worse at managing all of those, most bugs that we encounter now is like a murder mystery from Detective Conan and it requires Shinichi Kudo level of deductive reasoning to figure out what is going on.

Programming will remain hard if you keep thinking it is just a translation problem from a "business requirement" to a "computer language".

PH95VuimJjqBqy9• 3 years ago

I can even make an argument its getting worse

I agree and disagree.

We can do a lot more with a lot less people, but the issue is that the developer ergonomics have taken a straight up nose-dive due to the cloud.

nine_zeros2• 3 years ago

The biggest challenge in programming is dealing with stakeholders and management that expects things to be done yesterday - as in, they can't accept the concept of unknown unknowns - mostly because they have promised ambitious timelines to others.

brynbryn• 3 years ago

Don't really like the trend of articles "quoting" themselves. If people want to quote you, they will.

snvzz• 3 years ago

Hard as in NP-Hard?

another2another2• 3 years ago

"Programming is hard" ... let's go shopping instead?

GuB-42• 3 years ago

I've seen somewhere that computer programs are the most complex objects that humanity has created. Nothing we did before computers can compare. Clockwork, cathedrals, electronic circuits, math proofs, novels, etc.. None of these have as many interlocking "moving parts", as medium-sized software projects.

And in fact, programming is all about managing complexity. Really, it is all there is to it. The operations that computers do are simple, anyone who can operate a light switch can understand them in minutes. But how to piece millions of these to do something useful takes a lifetime, and it involves layers of tooling, custom or written by others, among them things like compilers and text editors, libraries, OSes, etc...

Programming is hard

Related Stories