95% of Programming Isn’t Hard (and other random thoughts)

https://macwright.com/2020/08/01/recently.html

by chmaynard • 6 years ago

36 72 6 years ago

Once you’re settled in, 95% of programming isn’t hard. It doesn’t involve algorithms, hard design problems, performance or storage constraints. It isn’t worth blogging about.

Once you're settled in ... having dealt with architectural design in a way that allows you to continue doing something with the code for 20 years or more.

OK, sure, 100% of programming isn't hard. But if you think that the hard part is only 5%, you're not doing the right job.

3pt14159• 6 years ago

I think you're right, but in a different way than I understand that you mean.

Whatever project I'm on I try my best to take on the hardest tasks. This allows me to code against things that are actually a challenge. Large refactorings, risky features, hard we-arn't-sure-if-this-is-possible tasks. This makes 40% of what I do hard. But it isn't because I'm writing software for ICMBs or satellites. I'm the dude that takes on the tough stuff and makes it work, and I think more developers would do themselves a favour if they did the same thing. Maybe you don't need to quit your job in order to change the ratio. If there is work at your present place that seems hard then volunteer for it. It will make you a better software developer.

ricardobeat2• 6 years ago

every time that someone tries to use JavaScript objects for data, they write a security vulnerability by way of prototype pollution

I would love to hear more on this, as it makes little sense as written. You never touch prototypes when using data objects, is the author talking about constructors pre-ES6? Or the fact you can shadow methods from the Object prototype? I don't remember seeing any exploits based on that (vs "that’s, really often, a problem").

tmcw3• 6 years ago

Fair point, I should have cited that. From snyk: https://www.google.com/search?hl=en&q=prototype%20pollution%...

You'll see prototype pollution CVEs in such libraries as jQuery, Lodash, handlebars, ajv (and transitively, request). A pretty good set of modules that are heavily used.

Scroll through the latest vunerabilities in the database and you'll see prototype pollution popping up very reliably https://snyk.io/vuln?type=npm

rbg246• 6 years ago

May I ask for clarification?

Is my understanding correct in saying that this is a peculiarly JavaScript issue because you can overwrite language constructs on the object prototype?

Whereas in other languages I can't affect the language core constructs?

mordechai9000• 6 years ago

I was wondering about this, too. An attacker who can run arbitrary code in your JavaScript environment can overwrite properties on base object types. But they have to be able to run arbitrary code already to do this. Running untrusted code is always a problem, in any language or environment, so I guess this would just be a way to take advantage of it. But you already assume the browser client is potentially compromised, since you never know what the user will do with it. And if you have an attacker running their own code in your server process, you're already hosed. So I'm not sure why this more of a big deal.

redmorphium2• 6 years ago

Concretely, at least on Node:

    const myData = {};
    myData[rawUserInput()].key = value;

This is vulnerable if rawUserInput() returns `__proto__`: `key` on the prototype may be set to value if `myData` had `__proto__` set. If the hacker controls `key` and `value` then they can effectively modify `key` field on all objects.

https://github.com/HoLyVieR/prototype-pollution-nsec18/blob/...

A common way is as described by that paper ^

Recursive merge functions, unless if they check for this vulnerability, are often susceptible, since JSON.parse defines __proto__ on its return values.

ricardobeat2• 6 years ago

Indeed, though you'll only manage to break or DDOS the server by doing that.

It seems that the Ghost vulnerability explained in the paper relied on the handlebars engine calling the equivalent of `eval()` on a string value, in the global scope, plus a path traversal vulnerability allowing loading a template from node_modules.

Are there are any other examples of this being exploited in a meaningful way? Even if you end up passing raw user input to key and value, user payloads will never be able to define a function, so the possibilities are very limited. I think having every NPM library that does object assignment using a user-provided key be marked as 'vulnerable to prototype pollution' is quite different from it 'being a problem very often' in practice. Happy to be shown otherwise.

bawolff• 6 years ago

They're referring to how in js, the for...in loop includes properties from the object's prototype.

There used to be much concern around the idea that you might load some third party js that adds thing to the base Object prototype, which will then be enumerated with for..in loops on all your data objects.

I always thought this was a stupid vulnerability because if you're executing malicious javascript, you are already screwed in a much more direct way

[Edit: i was mixing this up with a slightly different vulnerability that is a bit sillier than the one they are reffering to]

greg7mdp• 6 years ago

I kind of disagree. When programming isn't hard, it is because it is not done right, and the coder uses repetition instead of higher level abstraction.

Regardless of the programming task, it is usually hard to find the best way to do it, and not trivial to implement it correctly. This is demonstrated by the fact that most code sucks.

dmitriid• 6 years ago

When programming isn't hard, it is because it is not done right, and the coder uses repetition instead of higher level abstraction.

Aren't higher level abstractions supposed to make programming easier?

And yes, for the vast majority of programmers programming is easy thanks to existing high level abstractions, and it's usually the business domain that is hard.

ozim2• 6 years ago

They are supposed to ... but finding correct abstractions is hard. After a year or two abstractions usually are not correct anymore. That is why "favor composition instead of inheritance" is important because hierarchy of inheritance is fragile abstraction and if you compose things it is easier to swap one thing with other if they have the same interface.

wwright3• 6 years ago

IMO, abstractions aren’t easy, but programming with the wrong abstraction is significantly worse than with the right abstraction.

I think that because everyone starts with few visible abstractions, they get the sense that adding abstractions adds ease. Removing or changing them is often just as valuable.

ozim2• 6 years ago

When I read what you wrote it seems like the opposite is true. When programming isn't hard it is done right. Using higher level abstractions is not "golden bullet". If it is easier to have repetition and it easier to do find and replace and maintain and change code, why even make an abstraction?

Just like true developers write all their code in notepad and real developers etch bits on disk with magnetic needles.

cheez• 6 years ago

Had the same reaction.

NicoJuicy• 6 years ago

The hard thing about programming is making it easy ( readable)

humbleMouse2• 6 years ago

The first part of your comment reads like a talking point a meeting hopping noncoding archetect would use.

Simple repeatable solutions are better and more maintainable than constantly abstracting things to a higher level. I write systems and code with operations and developer ease in mind, not some academic idea of abstraction.

gnusty_gnurc• 6 years ago

I write systems and code with operations and developer ease in mind, not some academic idea of abstraction.

I'll be quite honest, if this is the same sentiment behind my experience at work (convoluted tangles of conditionals everywhere, 100+ line functions doing simple things - just to name a few), abstraction isn't just some academic idea.

I work with people who follow the path of least resistance and leave everyone else with the path of most resistance when we have to figure out what the hell is wrong with their code and fix the glaring oversights and errors they left behind.

AnimalMuppet• 6 years ago

The obsession with citing facts in paranoid literature that Hofstadter mentions is no longer a thing: modern paranoia is untethered.

The paranoid now cite "facts" that are stated somewhere online. They still cite them, though, as though they definitively settled the case.

Grimm13• 6 years ago

My one friend writes code for satellites. My other friend is a security vulnerability researcher working on the Linux kernel. I write code to process terabytes, potentially petabytes of data consistently and performant. I don't really know people who do frontend professionally in my personal life but I can't imagine keeping a webapp serving millions of people on many different devices performantly is easy.

My point is it really depends on the scale or depth of the work you're doing. It's pretty easy to keep things running smoothly for a smaller company or one not doing hardware etc etc but I think this really misses the mark for a lot of work, there's a scale or depth where you can't get away with not using what the author says is only used in interviews.

cosmodisk• 6 years ago

What you described here falls within 5% of all programming jobs.The rest 95% aren't that demanding.

rtlfe2• 6 years ago

I would guess that most satellite and big data code falls into the non-hard category. In my experience working with things that sound similarly challenging, it's usually the case that one person or a small group of people do most of the "hard" part early on and then a much larger team spends years doing relatively simple work expanding on the foundation they built.

Grimm13• 6 years ago

I just don't think you're correct. I think that idea is why we have companies on their third rewrites and most codebases look like sphagetti with performance issues abound. I think many domains are much more demanding but the people working on them don't know any better to avoid the pitfalls.

Thaxll• 6 years ago

I don't think programming for satelites / kernel is more demanding than modern frontend, for example the Linux kernel did not change that much in the last 15 years, it's pretty much the same old C with the same tools.

On paper it looks cool and complicated, in reality it's not that hard. Lot of fields in CS don't move that much especially things based on C / asm, kernel, embeded ect ...

humbleMouse2• 6 years ago

I like the article title and agree, the other points the author make are valid but seem like randomly picked issues.

qayxc• 6 years ago

Glad I wasn't the only one who thought that.

chmaynard• 6 years ago

notes on thoughts, in sort of a Matt Levine-esque style

Matt Levine is a columnist for Bloomberg.com. His columns are typically a random collection of thoughts on current events, from the point of view of economics.

wwright3• 6 years ago

He says at the start that he just kept some notes for the last month and was publishing them all

cle• 6 years ago

95% of programming isn’t hard
It’s easy to screw up the 95%, though

To me, those just don't seem like compatible assertions. If something isn't easy to get right, what else do I call it but "hard"?

MetallicCloud• 6 years ago

I've been walking for a good number of years now, yet every now and again I'll trip on something, or bump into a wall. I wouldn't call walking hard.

akvadrako• 6 years ago

I would call walking hard - most animals are just very good at it.

No programmer is good enough to build a walking algorithm that is 1% as good.

specialist• 6 years ago

TLDR: People are hard, computers are easy.

The only hard part about programming, and computers in general, is other people.

The OC mentions JavaScript, something insightful about data and objects. Just imagine: where would JavaScript be today without people like Brendan Eichmann?

I imagine other industries are more or less the same wrt the work vs the people.

alexashka• 6 years ago

The author should ask himself if he is doing 5% writing (and thinking) or 95% writing.

An honest contemplation of this question would likely result in a very different set of ideas and likely a different presentation format to go along with it.

tmcw3• 6 years ago

I commit my full force of being to craft the highest quality posts on my personal blog for you, this is really 1% stuff.

awake• 6 years ago

Thank you for your effort. It clearly shows both in your work which is prolific and creative as well as in your writing.

rtlfe2• 6 years ago

Once you’re settled in, 95% of programming isn’t hard. It doesn’t involve algorithms, hard design problems, performance or storage constraints.

It sounds like OP thinks the hard parts are algorithm design and architecture. I wouldn't even really consider those programming. They're things you need to do before you can start programming.

fish45• 6 years ago

5% or less of programming is hard: it’s the stuff that interview questions refer to. You might need to think about algorithms, might need to try out multiple strategies before deciding on the one.

The author is correct that most programs don't need complex or clever leetcode type algorithms, but IMO program architecture is just as difficult to get right for any program longer than a thousand lines.

Consultant324526• 6 years ago

What percentage of the population is even intellectually/psychologically capable of programming at a professional level? My guess is significantly less than half. That's not a we're smart and they're not. There's many jobs I'm psychologically ill-equipped to perform. For programming to be easy you have to have just the right amount of autistic/systemizing brain type, which appears to be significantly genetic. Everything in programming is abstract, so you have to be able to hold very large completely abstract systems in your brain. That's a different skill/aptitude than, for example, holding all the working parts of an internal combustion engine in your mind.

Programming is "easy" for only a tiny percentage of the population. That's just regular programming problems, not particularly difficult concepts/tasks I see many posts referring to.

enchiridion4• 6 years ago

I disagree with a few points here.

1. Most people could be trained to code. The most important trait in a programmer is persistence. Momentum built from solving past problems is a powerful force.

2. The idea of needing to be atypical in order to be successful is damaging and wrong. What matters is practice.

I understand what you are trying to get at, but there a lot of very smart people outside of the realm of software development. The ideas you are perpetuating raise the barrier to entry.

Consultant324526• 6 years ago

The blank slate theory has been thoroughly debunked at this point. I explicitly said it wasn't about being smart vs not smart.

ExtremisAndy2• 6 years ago

I pretty much agree here. I am an example of someone who simply does not have the genes for it, so to speak. I have loved programming since I was a little boy but am horrible at it. I can learn a language's basic syntax just fine (and usually in very little time), and have written many little tools/small apps that I have personally found helpful in automating certain aspects of my life and work. So I certainly value what little bit I can do. But every time I've tried to study a serious book on data structures and algorithms, it's been a disaster. I simply lack basic problem-solving skills. I try but I'm just not good at it. I mean, I can usually come up with working code to do whatever I've ever wanted to do, and I'm certainly thankful for that since it's ok if my personal projects aren't terribly efficient. But professional programming would never be a possibility for me. I'd get laughed out of an interview b/c of the convoluted/entirely-too-complicated ways I solve problems. I just wish I had a good sense of what algorithms/data structures to use where. That would certainly help me a lot! But I'm just not sure I "have what it takes", if you know what I mean, to develop that skill.

stuartd2• 6 years ago

I can usually come up with working code to do whatever I've ever wanted to do

Been working as a programmer for 25 years, and I’ve never read (or considered reading, or felt the need to read) a book on data structures or algorithms.

Also, I’ve seen plenty of people who are employed as programmers who simply cannot come up with working code. I don’t see how you can write working code without problem-solving skills, as by definition you’re solving a problem by writing working code..?

(I don’t really have a point, sorry, this just resonated with me)

Consultant324526• 6 years ago

I just wish I had a good sense of what algorithms/data structures to use where. That would certainly help me a lot! But I'm just not sure I "have what it takes", if you know what I mean, to develop that skill.

I wouldn't worry about it. I had a guy I mentored who couldn't code his way out of a paper bag in spite of having had a programming job for 5+ years at that point. I taught him how to program and while he did improve significantly, he realized he wasn't going to make it as a programmer. Now he works in infosec where he's largely administering software with occasional scripting type stuff. He gets to do the scripting/programming bits he likes and makes a ton of money. He's actually my boss now. There's a lot of this kind of work that I think is really rewarding. I hope you find a good fit for you.

ehutch79• 6 years ago

There was a story that i'm having trouble finding now, where researchers figured out a test for who would succeed in a compsci program.

It didn't matter how many questions you got right on the test, but rather, if you answered the questions in a consistent matter. even if you were wrong, as long as you were wrong consistently, you'd do well.

If you saw a question, and then a very similar problem 3 pages later, and answered them wildly differently, you wouldn't make it.

the conclusion was that if you were building patterns in your brain, or using logic to build up answers, you were good to go.

trustfundbaby• 6 years ago

if you do find it please post it. that is an entirely fascinating idea.

untog• 6 years ago

I really don’t think that’s true at all. Just think of all the spreadsheets out there in the world: they aren’t programming by the strictest definition, but they have most of the components.

To me, programming is simply an acquired skill. The vast majority of the population could do it.

PaulDavisThe1st3• 6 years ago

Creating a solution to a defined problem is one thing.

Understanding what the problem is ... that's something else.

stuartd2• 6 years ago

you have to be able to hold very large completely abstract systems in your brain. That's a different skill/aptitude than, for example, holding all the working parts of an internal combustion engine in your mind.

Is it really different though? “Why isn’t this engine working?” [traces through combustion path to find problem] doesn’t require ‘holding all the steps in you mind’ but it does require knowing them and how they interact, whereas “how do I make this engine more efficient?” does require a mental model of the whole engine and how the parts interact?

These seem broadly to me as analogous to respectively maintenance programming and systems development.

Edit: I think the part I don’t understand is why you think the abstract system (the code) is qualitatively different from the concrete system (the engine).

Consultant324526• 6 years ago

Yes, abstract thinking is different than concrete thinking. Thinking about an engine part is concrete thinking. Manipulating that part in imaginary 3d space in your mind is abstract thinking. 100% of programming is abstract thinking. Abstract and concrete thinking are done in different parts of the brain. Here's the first google result I found on the topic. I think this stuff is really interesting.

https://www.goodtherapy.org/blog/psychpedia/abstract-thinkin...

grosales• 6 years ago

At the end of the day, we all implement systems - large and small. The more moving parts a system has, the more complex its implementation tends to be. Decomposing large systems with moving parts takes many tries to master.

boris-ning-usds• 6 years ago

The rest of the 5%... well, that obviously is reserved for naming things.

codr7• 6 years ago

It's as hard as you want it to be, right at the limit of what you're currently capable of dealing with if done right.

I find that with increased experience, writing (any kind of) code is more intense. Because I'm pattern matching 35 years of experience at every turn.

As a result, I tend to write code in short (2-4h) but very intense bursts with plenty of rest in between lately.

odyssey7• 6 years ago

I tend to write code in short (2-4h) but very intense bursts with plenty of rest in between

At any level it’s a great idea to ruminate and plan what you’re going to do. Alas, current management trends are about keeping developers busy typing — which is something of a vicious cycle.

brian_herman• 6 years ago

I feel like there are two articles here one about politics and the other one about programming. I don't see how they are related in the way the author presented?

ZinnZirconium• 6 years ago

The hardest part of programming is finding workarounds for undocumented bugs. And I don't mean the kinds of details people have to be told on stackoverflow after refusing to read the manual. I mean genuinely showstopping bugs with nonobvious workarounds like HTML5 audio didn't fire the ended event in Safari until iOS 13.4. There's a guy who complained about it on his blog for eight years until Apple allegedly fixed it. And I say allegedly because I consider the fix to be broken and I still use a workaround which I had to discover for myself through original research.

The only part of programming which is easy is copying example code from the documentation. Once you build a nontrivial project that solves a real problem you're already working beyond the trivial. Real work gets real challenging real fast. If you're staying within the percentage of programming that's still easy then you're just doing copypasta and not really programming.

phendrenad2• 6 years ago

Almost everything I've ever attempted to do on my own, once I did enough research into how to do it, turned out to be extremely easy. The only time it's been difficult is when some other person put arbitrary and/or unexpected rules or limitations in place. Examples:

"Oh you want to make a website? You knew how to make a website in 2004? Well too bad, the tools you used in 2004 are now hopelessly broken. New tools have arisen to take their place, but you have to learn everything from scratch."

"Oh you want to make a video game? You learned OpenGL? Too bad that won't work on iOS. Looks like it's back to the studying phase for you..."

"Oh you want to compile Hadoop from scratch? Too bad it's a 9,000-step process, and anything could go wrong along the way."

"Oh you want to add a feature at the company you work for? Too bad someone wrote that module in an unnecessarily confusing way, so have fun tracing through that mess to figure out how it works before you can add the feature."

I've found these cases account for 95% of programming. So I guess I'm saying that 95% of programming is hard.

95% of Programming Isn’t Hard (and other random thoughts)

Related Stories