Prefer duplication over the wrong abstraction (2016)

https://sandimetz.com/blog/2016/1/20/the-wrong-abstraction

by rafaepta • 1 month ago

541 353 1 month ago

sandimetz.com

cryo323• 1 month ago

You can do both with microservices!

zephen2• 1 month ago

But wait! There's more!

For $19.95, you can replace your single single point of failure with multiple single points of failure!

flawn• 1 month ago

Or for 100$, get a 5x increase on all failure points - maximum vibes, maximum excitement.

DJBunnies• 1 month ago

Except 9/10 times microservices end up wildly dependent on each other, yielding a distributed monolith. Better to use service oriented architecture and just ship the monolith, you can test easier and skip the extra layers of serialization / deserialization.

loevborg• 1 month ago

I think you missed GP's point

mystifyingpoi3• 1 month ago

end up

So it just happens, right? There is no remedy to this? You know the answer :)

BTW I'm all for monolith.

mohamedkoubaa2• 1 month ago

Please, stop it

mystifyingpoi3• 1 month ago

I get the joke, but in ideal world, in microservices, there is no such thing as code duplication across services. As a maintainer of a service, I should not give a crap about code present in some other service - it's some other team's code, why would I care? I don't have to even know that the other team exists. In big systems, it happens that I can't even feasibly know the existence of all the applications.

cryo323• 1 month ago

You mean you don’t have 300 versions of your badly developed rapidly evolving platform services rotting away underneath the bit you didn’t duplicate?

throwaway203714• 1 month ago

In [an] ideal world, in monoliths, there is no such thing as code duplication across subsystems.

antonymoose• 1 month ago

Twice a coincidence, thrice a pattern.

anon-3988• 1 month ago

The problem with coming up with a rule that works for everyone is that everyone have a different idea of what makes a good abstraction.

Do you want to iterate using for loop or using .iter().step(2).map()?

I would rather have consistency than a mixed bag of levels of abstractions.

metaltyphoon• 1 month ago

Do you want to iterate using for loop or using .iter().step(2).map()?

I don’t think it matters, specially for sort sized loop scopes

doix• 1 month ago

Do you want to iterate using for loop or using .iter().step(2).map()?

This isn't really a good example, assuming both can be used to represent the same thing.

The problem with the wrong abstraction is when your abstraction doesn't let you represent something. Then, because of you've already invested so heavily into it, you start contorting the problem to fit your abstraction and it becomes a shit show.

christophilus2• 1 month ago

Yes. I’m dealing with a graphql, urql, Next, Prisma stack at the moment. Something that would be a handful of lines of code in a different stack ends up being hundreds in this one.

The Node ecosystem is full of wrong abstractions.

Rohansi2• 1 month ago

The problem is self-inflicted. You do not need to keep jumping to the next trendy framework.

RussianCow• 1 month ago

I don't know about you, but I generally don't write code in a vacuum. Other people may have touched it before me. Those other people may have made poor decisions.

Not that I'm immune from choosing the wrong abstraction sometimes. More than once the "other people" was me. We all make mistakes.

em-bee3• 1 month ago

my paycheck needs me to.

christophilus2• 1 month ago

I inherited this one. My preferred SPA stack these days involves Porsager’s Postgres library, a simple RPC stack with Zod schemas, and Preact.

Even better is an old school MPA with progressive enhancement.

cpursley• 1 month ago

Nextjs in particular is a dumpster fire, it's a shame that it's the default stack many LLMs slop out.

KHRZ• 1 month ago

This is the biggest lesson I got from LMMs. I have a 1 million LOC vibe coded project that I can only imagine would fit in a few hundred thousand lines. But it's still holding up, I expected some kind of development collapse long before this point.

cassianoleal• 1 month ago

I don't think that's a good lesson.

OP is right that code duplication is far cheaper than the wrong abstraction, but the opposite is also true - the right abstraction is far cheaper than code duplication.

gavmor2• 1 month ago

Well sooner or later I would expect a developer who intimately understands their code base to feel compelled to start refactoring and extracting fitting, meaningful well-leveraged abstractions.

imhoguy• 1 month ago

I don't think that will happen anytime soon. Prompts are the code now, and programming languages code is compilation product. Almost nobody optimizes compiled assembly code.

Perhaps "recompilation" - rewrite by replaying all prompts in strict code quality context (linters, complexity & dedup checks) would make better abstractions.

The only problem now is that LLMs are non-deterministic.

gb2d_hn2• 1 month ago

It's made me wonder the same, but most LLM generated codebases haven't been around long enough to judge maintainability. I have noticed issues in some of my more LLM heavy code when I expect a change to be replicated in multiple areas, assuming common code / styling was reused, only to find it wasn't. It's for that reason I can't use LLMs for client codebases without heavy scrutiny of every line generated (for my own hobby projects I'm a lot more lenient)

platz2• 1 month ago

2016 (up to 2018 or so) may have been the peak of such varied activity in the developer ecosystem, including articles like this, whether it was discussion, ideation, OSS variety, language development.

There has been growth since but it's been concentrated into fewer channels and somewhat industrialized.

agentifysh• 1 month ago

i recall very early in my career i did exactly this. i took what worked duplicated it—my reasoning being that it was far safer to reuse what has been battle tested and leave refactoring at a later stage

it wasn't received well and senior developer told me that 'good developers know exactly what patterns to use all the time before writing any piece of code and that he will clean up my mess'

long story short his refactoring caused what was otherwise a stable system into a complete mess and it reminded me of Nassim Taleb's book

nicoburns• 1 month ago

It's definitely an "it depends" thing. It's easy to overabstract. On the other hand, I've also met junior developers who just didn't know how to use function parameters.

dimgl• 1 month ago

long story short his refactoring caused what was otherwise a stable system into a complete mess

Yeah that totally happened

rf15• 1 month ago

"use the right pattern" coming from a senior smells like a senior who can't freely design new patterns. Established wisdoms are a starting point, not the go-to solution.

bhouston4• 1 month ago

I used to struggle with abstractions back in my OOP days but since moving pretty much to a purely functional approach I find that code duplication is rare. Just have a function and call it in two parts. The main abstraction issue is then data structures but with TypeScript interfaces being duck typing essentially I run into few problems there as well.

So code duplication because of abstraction issues is rare. Code duplication because of siloed developers is so much more common.

platz2• 1 month ago

what exactly is 'calling a function in two parts'

lysium• 1 month ago

I read it as „calling it from two places“

saghm4• 1 month ago

I assume they mean to call the function from two (or more) parts of the code (i.e locations). It's not immediately apparent why this is meaningfully different than what would be possible in Java though, since ostensibly a function is the same as a method by just moving the callee to the list of parameters. (There are some things in a Java method that you can do that don't translate to most functional languages, like invoking the version of the method from a superclass, but there's nothing forcing you to do any of those from the language perspective, so it seems a bit strange to claim that the language itself is the issue rather than maybe the specific patterns that were chosen, maybe by their coworkers or just not common in the ecosystem).

odo1242• 1 month ago

I believe they’re referring to callbacks / dependency injection / higher order functions to customize the behavior of a function?

bhouston4• 1 month ago

Calling a function from two locations is what I meant.

Basically since moving to a functional approach in typescript I find I do not fight abstractions as I used to when I used classes and inheritance.

Akronymus4• 1 month ago

I assume to split the overall behaviour (loop through all elements, transform some value, etc) and the specific one (apply this function to all elements, transform it in this way, etc) into multiple functions and combine those to achieve the actual intended behaviour.

At least that's my interpretation

tosh• 1 month ago

apparently it is not what the author meant but:

using projection you can "call a function in two parts"

  add: {x+y}
  add4: add[4] / gives {4+x} by fixing 4 as first argument to {x+y}
  add4[2]      / gives 6

this is a useful pattern that you can use to first 'fix' data or behaviour to produce another function

https://en.wikipedia.org/wiki/Partial_application

scotty79• 1 month ago

I read it as if your function contains a lot and you'd like to reuse part of this functionality in another place you don't have to figure it out how to abstract this. It's easy to just split your function in two parts and call the part you are interested in far away as is.

But I'm probably overthinking it.

ikety• 1 month ago

For hobby, I use functional languages, and I find the techniques are the important bits to remember. Most modern languages let you easily stand on functional programming theory. You don't need to know Haskell. Everyone's brain works differently, but the idea of small, simple and occasionally flexible parts building a whole works for me. As opposed to the large complex do it all shape shifting machine.

kccqzy• 1 month ago

Developers do not really have to be siloed to experience code duplication. When the team size grows past a certain point such that each person is not aware of what every one else is working on, code duplication is quite inevitable. This is the case even if everyone writes functional style code. In fact this just happened last month at work: I wrote a new functional and pure helper function and placed it at the beginning of the file; a week later a colleague told me a similar helper function with substantially the same functionality with a different signature had been written and placed near the end of the same file.

pramodbiligiri• 1 month ago

It would be interesting to see if LLM's can scan and surface such issues. Might have been probable with parsing tools pre-LLM also? Not sure.

throwaway203714• 1 month ago

    > but since moving pretty much to a purely functional approach

What language?

yCombLinks• 1 month ago

Not seeing how this makes sense in terms of the article? A function is an abstraction. Extracting duplicate code into a function is the same concept.

atmanactive• 1 month ago

No it's not.

znkr• 1 month ago

+1 The worst code I had to maintain was code that tried to follow DRY (without the trying to understand what the original intention of that principle was). The only way out of that mess was widespread code duplication.

zadikian3• 1 month ago

Yep. Keyword "tried," as in they did it for a while then hit a point where it's impossible to faithfully follow the abstractions because they're wrong.

tomjakubowski3• 1 month ago

It'll be fine, don't worry about it: just add a couple more obscure boolean parameters to that reusable function to support your new use case and ship it.

bazoom42• 1 month ago

Depends. If the abstraction is just a level of indirection, then it is usually pretty simple to eliminate - just hit “inline function” in the refactoring tool a few times.

On the other hand it is pretty difficult and error prone to consolidate duplicated code which have drifted apart over time.

If in doubt, chose the approach which is simplest and least risk to revert if you discover in the future you made the wrong choice.

I do agree a bad abstraction can cause huge problems. But it’s usually not the kind of abstractions introduced to eliminate code duplication, but the kind of top-down “architecture astronaut” abstractions, where a model is chosen which does not fit the complexity of the problem.

strongpigeon• 1 month ago

Echoing the article, anyone who has experienced both will agree: it’s far easier to work with an under engineered code base than an over engineered one.

tarcon• 1 month ago

Contrary to that. The saying - Better to have a bad abstraction than none - was born from spaghetti code pain.

HeavyStorm2• 30 days ago

Even better to work with one that isn't shit.

jstimpfle• 1 month ago

Code duplication is the wrong abstraction too -- unless it's not really code duplication but code that only happens to be similar for some really "unstable" reason.

dofm19• 1 month ago

I would agree that there are good "de minimis" reasons not to abstract code that isn't ready to be abstracted at all. If the pattern has not settled it shouldn't be forced into an abstraction (beyond those that make sure it is e.g. not vulnerable)

But beyond that, any stable abstraction is better than duplicated code.

dofm19• 1 month ago

No it's not. This has always been a needlessly iconoclastic rather than sensible suggestion.

At the very least it is not once you're working at the wrong kind of scale.

Once you have an awkward number of customers (more than five and less than a hundred), maintaining duplicated code that should have been abstracted and modularised will only seem cheap if you don't mind that you burn through even junior employees at a pace.

And in the LLM era the wrong kind of scale appears in different ways; code generated and duplicated without proper abstraction and then maintained by an LLM that cannot be trusted to do the same modification each time it encounters a pattern or to have enough of an overview to slowly rescue duplicated code through good abstractions.

I would go as far as to say that any abstraction you can maintain (that is in active maintenance, I mean) is better than code duplication once you are past a de minimis threshold.

mawadev• 1 month ago

I think you applied this idea into the era of LLMs but consider an abstraction that takes in multiple god structs for branches it may or may not call in the case you are looking at and has a lot of if conditions that explode in combinatory complexity across a deep call chain. Now the bottle neck is that you need to call this function 144 times a second. That is where you start to have clusters of hot code paths where the latency stacks depending on the angle the god structs come in. Not sure what LLMs do here, I don't vibe code

dofm19• 1 month ago

I am applying it to LLMs on the basis of twenty years of seeing smaller programming shops tie themselves in knots by using duplication to avoid developing an abstraction that would help them because they were unsure of it.

Everyone always thinks duplication is fine when you can bill the modifications by the hour. But they never think to understand that the reason they've had so many employees is that they've turned their change process into firefighting all the different versions of the same code and all these young developers burn out from the sheer anxiety of not knowing where all the little fires are.

I once had to rescue a site that had become a victim of its own popularity, that was written by subcontractors who clearly believed that duplication is better than the wrong abstraction.

Until one day, along came a change — MySQL 4 to MySQL 5 — and a significant duplicated query no longer worked due to its new, proper strictness.

The problem was compounded; not only was the broken pattern in hundreds of places where it had sat, stable and predictable, but the pattern was broken because it, itself, was avoidance of another abstraction that would solve it.

They quit: they said they couldn't and wouldn't fix it. It had always worked how they had done it, and it would have to stay on MySQL 4 (which the hosting provider refused to accommodate).

I don't think it helped that they were severely misguided in their understanding of SQL, but the code had become beholden to duplication and then crippled by a new problem in the duplicated pattern.

I had to first find all the contexts in which that pattern appeared (which required me to spend half a day on a bespoke script) and then work out a new pattern and as few variations of it as possible to fix the duplicated code in each place, because there was no proper budget to rewrite the whole thing. And then I sat at my desk, for days, working through each one, figuring out how to change it to fit the slightly different expression of the pattern.

Even a total bullshit abstraction would have saved that client both time and money. And this is only one of dozens of times I've seen small firms simply duplicate and change code that would later become unmaintainable because of a straw breaking a camel's back.

fny• 1 month ago

Code duplication is cheaper than the wrong abstraction. If you have a good abstraction, you should run with it.

If you haven't figured out a good abstraction at 5-100 customers, God help you.

dofm19• 1 month ago

I disagree.

But also it's very possible to not realise you needed an abstraction until it catches fire in multiple places.

And quite often it's not you that got the codebase to a hundred customers, is it? Sometimes it is a sequence of fresh-faced young developers who didn't have the authority to say "this duplication is bullshit" and were instead compelled to repeat it.

I think a lot of these discussions happen in nice little blog-post vacuums of progressive thinking, where people can go "mmm, object oriented coding obscures intent and clarity, mmm", blog posts with "an X is a Y", "the unreasonable effectiveness of foobar" etc.

In the real world, every duplication that works sticks for good; there is rarely budget to electively replace code that isn't broken. Until one day it doesn't work. And then… how many times is it actually duplicated? How many of the duplicates diverged? How many of these do we no longer need?

feoren• 1 month ago

A good abstraction? As in one? I'd go so far as to say the process of discovering and refining abstractions is the most important part of software engineering. A large project has dozens of abstractions, and some of them are "wrong" at any time, as you discover over time. None are ever perfect. If you wait to stop duplicating code until you have the "right" abstraction, you are just putting off the hard part of developing software and taking on tech debt.

Half of your abstractions are wrong. The hard part is knowing which half.

enos_feedler• 1 month ago

What if there is no good abstraction for the entire stack of software on each of computers? What if we built a common one because we had to? What if now we get to all make our own with natural language?

stymaar• 1 month ago

Code duplication is cheaper than the wrong abstraction

This is tautological though, it's like saying “starving is much better than eating the wrong food” (for instance: eating quick lime).

Of course you'll always find a way to do things wrong in a way that is costlier than not doing anything.

locknitpicker• 1 month ago

If you haven't figured out a good abstraction at 5-100 customers, God help you.

This blend of opinion is very naive. Every single project is a business requirement away from having the wrong abstraction in place.

https://xkcd.com/1425/

Capricorn24814• 1 month ago

I would go as far as to say that any abstraction you can maintain (that is in active maintenance, I mean) is better than code duplication once you are passed a de minimis threshold.

Pretty much everyone arguing for duplication has argued what you are saying, which is wait to see a few instances of it before committing to an abstraction. No one is saying duplicate everything 100 times. So I don't think this discussion was ever iconoclastic.

dofm19• 1 month ago

The point is it sounds all smart and sophisticated and principled in the abstract environment of a code discussion in a blog post.

In the real world, duplication happens in an emergent way, there isn't the time each time to judge whether it's really time to just quietly abstract that code, you may not get the permission, budget or window to do it, and if you don't stop the rot really early you are locked into the pattern.

coldtea5• 1 month ago

Hardly iconoclastic, it's a very sensible suggestion.

It would be iconoclastic if the common sense basic approach would be to start with abstraction. It's not, the common sense default is to write possibly duplicate behavior until you actually discover several cases to abstract away, until you bevalop a sensible idea of which functionality unites them and which doesn't carry over all of them.

Once you have an awkward number of customers (more than five and less than a hundred), maintaining duplicated code that should have been abstracted and modularised will only seem cheap if you don't mind that you burn through even junior employees at a pace

Maintaining the wrong abstraction, or, god help, abstractions, would be even worse.

bluefirebrand2• 1 month ago

Yeah, "Write Everything Twice" is a pretty common and sensible direction for any codebase

dofm19• 1 month ago

Maintaining the wrong abstraction, or, god help, abstractions, would be even worse.

Hard disagree. When you've had to chase through a change in untold and actually unknown numbers of duplications of code in different permutations and fix them because they are all on fire simultaneously, you'd disagree too. A bad abstraction would at least have had one fire in one place.

shinycode• 1 month ago

At work there’s been a huge number of duplication in the start of the company and no solid abstraction. So no tests as well. We introduced tests in the current architecture but rewriting code has a huge cost to make sure there is no regression. When we talk about a saas it’s non-trivial with many customers relying on this tool daily as part of their workflow, regressions because of rewrite could be really painful for them. So we must give a greater budget to take the time to make sure nothing major breaks. So there is a debt that is compounding over time because code is added. Duplication is bad and weird/purist abstraction could make the architecture so rigid that rewriting things could generate hard to understand and catch bugs. It’s hard to find a good balance and it depends on the kind of business and scale of project. Hard to make that a generic advice.

pfannl• 1 month ago

The real rule is probably: duplicate until the abstraction stops looking like a horoscope.

agumonkey• 1 month ago

You seem to have experience, I dont mind factoring / unifying logic, when done sensibly with enough history in the trenches. It pains me more whenever a young dev comes in and barks "we must merge these two things!" repeatedly without planning for more than two cases and starting to add more and more boolean variables. Crystal makers. Then the obvious issue comes, the two variants weren't that close and now there's one god class trying to handle all forces in one big state.

I agree that LLMs are naturally anti abstraction machines.. I'm often trying to find way to reverse that.

dofm19• 1 month ago

I agree that LLMs are naturally anti abstraction machines.. I'm often trying to find way to reverse that.

I am a bit of an LLM cynic but I am trying to learn it all, and I have to say I have spent most time trying to work out: how do you explain how a brown-field codebase actually works, in such a way that the LLM won't pervert it through misunderstanding.

It does encourage you towards the "conventional" coding standard for any new project, because you want to use a pattern that it will have seen in its training set.

But for example there are differences of opinion in how wordpress plugins (which have a very complex control flow) should be structured. LLMs are incredible at knowing how WP works, actually, but what is difficult is explaining how your methodology for a large plugin is going to work.

It is a battle — but a useful one because it can be used for, er, studying the comparative belief systems of the LLMs.

mytydev• 1 month ago

It sounds to me like you are describing a good abstraction. This article does not claim that code duplication is better than any abstraction. It claims that code duplication is better than the wrong abstraction. I'm sure this author would agree that a good abstraction is better than code duplication.

dofm19• 1 month ago

I'm afraid this comment reads in a rather gnomic way.

Of course it's a truism if you just say any abstraction that works is a good abstraction.

That is not what I am saying at all. Bullshit abstractions at least let you control the problem. Duplication doesn't.

tracerbulletx• 1 month ago

Huh? If anything having lots of customers makes the argument for duplication stronger. The issue is almost always once you get huge and 5 product teams are trying to achieve 5 different goals by using the same overwrought abstraction instead of just copying and decoupling. The abstractions that are actually stable end up becoming libraries or platform team owned systems that no one ever really touches.

nfw2• 1 month ago

Over-engineering and "abstraction hell" are very much not iconoclastic concepts

ChrisMarshallNY2• 1 month ago

In my experience, the answer is always "It Depends." That's about the only thing that I can hang "always" on.

It really depends on the exact type of code we're working with, and what our objectives are.

In my case, I often use object inheritance. It's a damn cheap way to DRY. However, when people hear "inheritance," they often think "polymorphism." There's a really big difference between the two, but popular culture has jammed them into one ball, and it's not worth the agita, to try to explain the difference.

But if you are doing optimization, long stacks can be your enemy, and inheritance tends to have long, windy stacks.

In these cases, the copy/pasta method may well be the best approach.

Like I said, "It Depends."

tomjakubowski3• 1 month ago

In my case, I often use object inheritance. It's a damn cheap way to DRY. However, when people hear "inheritance," they often think "polymorphism." There's a really big difference between the two, but popular culture has jammed them into one ball, and it's not worth the agita, to try to explain the difference.

I agree that we should think of inheritance and polymorphism separately. If we want to express this intent in object-oriented code, how can we use inheritance to deduplicate code, while preventing misuse of the resulting object hierarchy i.e. the use of base classes in a polymorphic context?

In C++, IIRC private inheritance would do the trick (you cannot static_cast DerivedWidget * to BaseWidget * if DerivedWidget : private BaseWidget), but most OO languages don't support private inheritance. It's also not possible, as far as I know, to "lock down" BaseWidget * so it cannot be used as a base class pointer from any derived class: instead, you have to apply the private inheritance to every derived class to enforce this rule.

Another approach is to use has-a instead of is-a: i.e. instead store a BaseWidget object as a member of DerivedWidget. This allows for re-use without supporting polymorphism.

Thaxll• 1 month ago

So you centralize 3 liners?

dofm19• 1 month ago

I said "beyond a de minimis threshold".

But in one of the scenarios I mention earlier, I earned a chunk of money once fixing an issue that emerged in a subcontractor's four or five line duplication that had ended up rippled through a long-lived codebase. A ground truth (MySQL version) changed, and the pattern broke everywhere, including places where it had evolved.

So I tend towards thinking, yes, any three-line pattern that is likely to appear everywhere should, perhaps, be centralised.

It's certainly worthy of serious consideration. Usually pretty easy to maintain the surface of such an abstraction.

thinkloop• 1 month ago

The key lesson is that duplicate code is not necessarily "code duplication" - it was always really about abstraction duplication. If two unrelated variables happen to momentarily share a value, it doesn't mean that value should be made common between them, they are fundamentally different things. It would be a confusing lie and error-prone if the code implied they were the same and that efforts should be made for them to be in sync.

dofm19• 1 month ago

I guess any blog post can remain true if you can optionally take one of the key terms and redefine it so it can also mean the opposite?

ubertaco• 1 month ago

I'd recommend clicking through the headline to watch the talk. Metz talks a lot about types of similarity: similarity by coincidence vs similarity due to an actual semantic or functional equivalence.

Code that is coincidentally similar very often diverges in either the short or long term, and DRYing it up aggressively tends to result in functions that have many boolean parameters that each trigger disjoint sets of behavior - which is a bit of a nightmare to maintain due to the high cognitive overhead of remembering how all the interleaved-but-actually-unrelated behaviors should work.

This outcome is low-cohesion code.

It's a useful concept to be aware of - worth clicking through to the actual content of the talk rather than just the headline.

dofm19• 1 month ago

I'd recommend clicking through the headline to watch the talk. Metz talks a lot about types of similarity: similarity by coincidence vs similarity due to an actual semantic or functional equivalence.

I've seen this article and AFAIR the video before, and FWIW having been a Rails developer from the very early days and fitfully until maybe even 2014, I now interpret the phrase "my Railsconf talk…" quite negatively.

ETA: nice to be back to disagreeing with people on HN about coding principles again though. Hopefully this is a sign.

tcfhgj6• 29 days ago

"tends to result in functions that have many boolean parameters that each trigger disjoint sets of behavior "

What tends to happen is in your hands.

cjfd2• 1 month ago

100% agree. 'Code duplication is far cheaper than the wrong abstraction' is a very good candidate for the worst programming article ever.

a-dub• 1 month ago

i agree with the author. i argue a preference for loose coupling over centralized abstractions. sure it's pleasing to compress the code, but if the use cases actually are sufficiently divergent (as well as bugs and externally driven changes) ultimately it becomes brittle, littered with edge cases behind if fences and both challenging and daunting to change.

ideal case: support libraries and then very simple duplicated code that is easy to read and modify. critically the core control flow should remain duplicated, but simplified by the support libraries.

cyberax• 1 month ago

My general rule is to start refactoring once you have three copies of the code.

Starting with abstraction when you are only beginning something rarely works well and leads to code bases littered with interfaces having only one implementation.

Abstracting the code when you have two copies does not always pay off, especially when you end up not needeing more than just two copies anyway.

But once you have three copies, it's indeed time to start generalizing.

yowlingcat• 1 month ago

One of the most challenging kinds of thought to work through with my engineers in professional communication is nuance. For example, they may say something like this, but actually mean "For a particular situation, this is wrong."

The context a decision is evaluated is particularly important for "rules of thumb" like this. There's the rule of 3 (which many senior engineers imparted to me earlier on in my career) - don't refactor until you've actually duplicated it thrice, but even so, what they speak of is a catch-22 that's pretty important to reason about carefully.

On one hand, if you overcorrected on the fear of abstraction, you could easily end up with 500 duplicates that are slightly different and need to be maintained 500 different ways, slowly causing slightly wrong behavior some of the time, data corruption, combinatoric explosion. Surely, once there is such a situation, some degree of abstraction is the only right decision.

On the other hand, if you overcorrected on the fear of duplication early on, you could easily end up with a premature optimization and complexity -- complexity which, most importantly, could be rooted in a gap of understanding of how the code will be used and what direction it may go in over time (often based on which direction the business will go over time).

The only answer that actually works, of course, is "somewhere in the middle." Obviously, that's pretty vague and not very useful. Where, exactly, in the middle IS the right place?

As the years have gone by, I've become more and more steadfast that the answer to that question is and must be an art and not a science. Of course, it must always be rooted in practicality, the actual context of the code around it and where the code/business was in the past and where it will be in the future.

But just as importantly, some of it must be based around beliefs in the face of imperfect information about what you want to invest in for the sake of the technology, the team that develops it, and the business that relies on it. It could be that for your team, your values make it make sense to go a little bit further than "good enough" on normalizing your data modeling, because the way you like to run your business requires that normal form to do the analytics and make decisions productively. It could be that for your team, your values make it make sense to go a little bit further than "good enough" on splitting service boundaries and ensuring clean queues and message passing infrastructure, because you have seasonal spikes where you need to scale up to a ton of load and then scale down after without constantly doing a song and dance or pre-provisioning fragile infrastructure.

But the most common thread there is - art, not a science. Every single decision depends on YOUR team, YOUR business, YOUR needs - and like any art, there is no universal rule or discovery or best practice in the industry that will magically work for your needs without working through the details of whether it appropriately fits your situation or not.

So with that said - I can't really agree with you. At any place I've ever worked with a competent team, maintaining duplicate code is just not that hard and follows the same process for being dealt with. Built a robust test suite that encodes the actual differences and the shared structure. Pull out the pieces that have a good reason to be abstracted and redesign the pieces that encode the true differential structure in a way that is intuitive. Lather rinse repeat. It's always straightforward because it's known - by the time you are doing this process, you've had tons of repetitions and data on what is driving you to develop the abstraction, so when you make the decision, you are making it empirically.

Conversely, I have seen many otherwise competent teams slowed to a halt with premature abstraction. Frameworks that were well intended and reduced duplication, but encoded coupling between components that at a certain point in the businesses progression, fought with reality rather than aided, and all because they were frozen into place before anyone empirically had really clear data about whether the abstraction would be worth it long term. Well intended "clean code" refactors that were meant to solve the old "bad duplication" but instead created a far more difficult to reason about "abstracted base" of code that didn't really solve any of the domain modeling problems and was just as difficult to maintain without introducing buggy behaviors (if not more so) than before.

The biggest problem is that premature abstraction is sexy and fun. There are incentives and dopamine hits from doing it extraneously. But fixing legacy duplication is not fun. And so when it gets done, it tends to get done in a pragmatic way to relieve pain rather than to elicit pleasure. That, I believe is one of the biggest confounding sociological aspects of this whole discussion.

nextaccountic• 1 month ago

The trouble with the wrong abstraction is that sometimes you really do want to change one and not the other. It's code that superficially look the same, but only temporarily (taking a snapshot at the current time) - they are meant to be distinct in the long run

bob1029• 1 month ago

If you work backward from the schema these sorts of things tend to evaporate before they can become a problem.

Some of the biggest rabbit holes come from naming conventions not aligning across the business and technology silos. If everyone agrees that Customer has exactly 34 attributes, then it is possible to move to the next step of sharing libraries of types across the team. Getting your POCOs/DTOs 1:1 across the board is when the duplication really starts to melt away.

sebastianconcpt• 1 month ago

Oh the self-contradiction here...

Generalizing this in the abstract is a wrong abstraction.

northisup• 1 month ago

Duplication is fine, triplication and above is the issue.

mjevans• 1 month ago

Triplication tends to be where it becomes more clear what the correct thing to abstract or de-duplicate is.

It's of course possible to functional-ize segments of logic, but then the question of state mutation must be brought up. How isolated are these changes from other parts of the code / system state. Can this be run in parallel or is it something that must be serial? What potential race conditions exist?

williadc• 1 month ago

The "99 Bottles of OOP" book mentioned at the bottom was an excellent introduction to refactoring. I highly recommend it if you struggle with finding the right data models for the problems you work on.

jbvlkt• 1 month ago

It depends if duplication is accidental or real. I.e. if two taxes are using the same formula, it is accidental. If you use the same physic formula on multipla places, it is real duplication.

originalcopy• 1 month ago

While I see the point, I think I more often encounter the opposite. Duplication, but not exactly duplication. Then the "sunk cost fallacy" is not an issue but there is huge maintenance cost and no-one feels like refactoring it. I'd rather refactor bad abstraction than 10x duplication.

em-bee3• 1 month ago

but those are exactly the cases where the distinction matters. when you have a situation where you can't duplicate the code exactly, then you really have to look carefully if this is actually the right place for a shared abstraction. i tend to wait and see if i can refactor one or the other to get them to be exact duplicates and only then see if i can fit in a common abstraction. and yes, finding that i later need to make the same change in both places is a sign that a common abstraction is probably the right call.

Rendello5• 1 month ago

Two talks come to mind here: Mike Acton's Data-Oriented Design and C++ [1] and Brian Cantrill's The Complexity of Simplicity [2].

Mike's talk argues that code solutions need not be modelled on the real world, and that different data creates different problems, which need different solutions. I can't do the talk justice, but it's had a big impact on me.

Brian's talk is about abstraction generally, and how it's difficult to find the "right" abstraction.

1. https://www.youtube.com/watch?v=rX0ItVEVjHc

2. https://www.youtube.com/watch?v=Cum5uN2634o

saghm4• 1 month ago

Mike's talk argues that code solutions need not be modelled on the real world, and that different data creates different problems, which need different solutions.

I've always found it odd when even fairly smart engineers sometimes prioritize real-world metaphors over the actual needs of the codebase. Years ago when I was only a few years out of school, I was implementing a connection pool in Rust, and the most reasonable way to implement it was to have the connection hold a weak reference to the pool so that it could get checked back in automatically when dropped. My manager (an extremely experienced engineer) didn't like this idea because "a library holds library books, not the other way around". I didn't feel like this was a compelling reason to design things differently, but he refused to engage with the issue in any way other than through the lens of that metaphor. Eventually the impasse was solved by one of the other managers in my department suggested that while library books don't contain libraries, they do have the name of the library stamped in the back as a reference to where they should be returned, and I guess my manager found this to be a reasonable extension of the analogy. If I were more experienced, maybe I would have recognized that I could find a way to engage with the analogy like the other manager did without ceding the point, but even today I still feel that it was completely bizarre to insist on that as the canonical way to frame things rather than just considering the ramifications of the abstraction in the code and the experience of using the library based on it.

Rendello5• 1 month ago

This is somewhat related:

I mention this a lot, but in researching Data-Oriented Design (what Mike was talking about), I came across Richard Fabian's DoD book [1] which talks a lot about database normalization and the like. I found that odd, because the low-level high-performance game code he was talking about certainly wasn't going to marshal data into a DB to run SQL queries on it.

It turns out the relational model has a lot of advantages though. Programmers use trees all the time, in OO, in structs containing structs, in objects pointing to other objects. It's easy to forget that trees are just a special case of graphs (ie. networks), and that there are many ways to represent networks that don't rely on encoding a tree structure directly.

So, I've been doing what Richard Fabian suggested and I lay out my data (on paper) into tables, then attempt to normalize it and see the connections. I really like this way of designing things.

My big issue is that doing DB-like operations is hellish in most programming languages, and if you really want to try and marshal your data into a real DB (say, SQLite or DuckDB via a library), then you have a big messy translation layer where you're trying to match things to SQL types and you have giant SQL strings everywhere.

I see C# has LINQ, which is a query languages embedded in the language. I wonder if that approach is best, and why hasn't it been adopted more broadly? It seems like there's a lot for programming language designers to explore in this dimension, though I wonder if it even matters now with the superintelligence tidal wave.

1. https://www.dataorienteddesign.com/dodmain/

pocksuppet5• 1 month ago

Didn't we all spend years and years learning that OOP is about writing "class Car extends Vehicle"?

lucas_t_a• 1 month ago

this one comes to mind https://youtu.be/17KCHwOwgms

so-cal-schemer2• 1 month ago

See also:

Data-Oriented Programming: Reduce software complexity

by Yehonathan Sharvit

https://www.manning.com/books/data-oriented-programming

and from SICP:

2.4.3 Data-Directed Programming and Additivity

https://sarabander.github.io/sicp/html/2_002e4.xhtml#g_t2_00...

Rendello5• 1 month ago

There's a reason why "naming things" is one of the two hard problems in Computer Science. Data-Oriented Design (DOD) and Data-Oriented Programming (DOP) are two different things which has caused a fair amount of confusion on HN before. Data-Directed Programming (DDP) appears to be a third, different thing.

In searching those threads, I came across a post from the author of the DOP book describing the difference between DOD, DOP, and DDP (a different DDP! Data Driven Programing; a fourth thing!) [2], and I see he also made the "naming things" joke in the first paragraph, so I guess my humour isn't that unique!

There's been quite a bit of discussion about DOD and some about DOP (and much conflation between the two) on HN, it can be interesting to read [3].

1. https://martinfowler.com/bliki/TwoHardThings.html

2. https://blog.klipse.tech/visualization/2021/02/16/data-relat...

3. https://hn.algolia.com/?q=Data+Oriented

ChrisArchitect• 1 month ago

(2016)

Some previous discussions:

2023 https://news.ycombinator.com/item?id=35927149

2021 https://news.ycombinator.com/item?id=27095503

2020 https://news.ycombinator.com/item?id=23739596

2018 https://news.ycombinator.com/item?id=17578714

2016 https://news.ycombinator.com/item?id=11032296

ultim8k• 1 month ago

Nobody wants to listen. Nobody. In 90% of the companies there are some so called senior devs that get ecstatic when they create a new abstraction.

Overengineering, abstractions and premature optimisation are the 3 worst plagues of engineering.

At the same time I’m happy they exist because it means we’ll always have a job.

codemog• 1 month ago

Yep Kubernetes, more micro services than engineers, some complicated protocol that saves a few bytes of overhead, cloud everything, and tons of classes that could have been simple functions.

throwaway203714• 1 month ago

The part that no one want to say out loud: Making boring technology decisions makes you job boring and does not help to build your resume. This is the core reason why over-engineering exists.

skydhash2• 1 month ago

Making the job boring is a great way to get free time to browse HN.

globular-toast• 1 month ago

Remember, everyone else's job is simple and pointless, only your job is difficult and important. Therefore only your job could possibly need abstractions. Everyone else is just over engineering.

DanielHB• 1 month ago

Only MY code is allowed to be clever.

rzmmm• 1 month ago

There are codebases out there with enormous amounts of duplication, filled with implicit dependencies. You just haven't encountered them to appreciate good abstraction.

HeavyStorm2• 30 days ago

I think that all this people defending this post have no idea that this rationale can cause. I'm sitting at a codebase that has the same code for iterating copied around 20 times.

As a senior I will tell you to dedup whether I can to avoid this shit.

MrGando2• 1 month ago

I once had to work with a system that was refactored and abstracted away heavily to use Redux. It didn't work then, the implementation had way too many abstractions, doing any change meant you had to touch dozens of files. It was insanity. Left me with a bitter taste regarding the redux pattern for ever (probably not the pattern's fault).

SoftwareMaven• 1 month ago

Over-abstraction is as much of a problem as under-abstraction. If the abstraction isn’t improving your ability to produce good code, it’s a bad abstraction. I’ve worked with a lot of abstraction patterns in a lot of languages over the 30 years of my career. Any of them can be good or bad. Unthinkingly applying them is always a problem.

Unsurprisingly, that goes for just about any idea in software development. I worked in one code base that heard small functions are best, so every function was less than three lines long. You don’t gain anything by replacing `lst.get(0)` with `get_first_item_in_list(lst)` (in fact, understanding becomes much more difficult), but breaking down functions into the smallest units that make sense independently within the business domain can be very helpful, both for understanding and testing.

MrGando2• 1 month ago

100% agreed. It's interesting since I see over-abstraction often abused by "clever" engineers (sometimes quite experienced actually). Sometimes I wonder if they do that to make themselves indispensable on purpose and create their silos in the codebase.

aappleby• 1 month ago

The smallest amount of simple code that solves the problem wins. Everything else is irrelevant.

hedora• 1 month ago

I’ve seen code bases that evolved like that. The problem is almost always outside the abstraction that has a pile of conditionals.

Usually, some moron decided to copy paste things a few levels up and then the top half of the system metastasized into two parallel universes of broken garbage.

For instance, one might decide to perform auth later in the flow so unauthorized handlers can run and set a “this requires auth” bit that defaults to false, and the other flow could add a forged auth header before the auth step.

Now, the auth handler needs a “allow forged header” flag and a “already authenticated” flag.

I’ve seen that grow to a half dozen cases until massive production dataloss occurred. A buggy client tried to delete something local to their account without specifying a userid as a parameter (this codebase was garbage!) and deleted the something for all users instead.

I can’t remember how the dataloss was “fixed”, but it definitely wasn’t “all requests go through a simple auth check, and all handlers declare/implement their auth requirements in the same way”.

Getting a design approved to require a user id be specified exactly once for account-level operations was fantasy land for that team. (Most hires with any sort of engineering talent bounced in under a year.)

Anyway the “abstractions are hard so copy paste” approach did provide job security for the lifers on that product. I can’t imagine them holding a job elsewhere, but they were completely immune to layoffs (hostage style).

This is a pretty valid approach if you’re an agent hired to perform industrial sabotage, or if you keep replacing keyboards after you knaw through the corner.

fjfaase• 1 month ago

I once used code duplication to implement a fourth type of dialog that looked somewhat similar to the others, that were sharing a lot of code, because I felt that although it looked much the same as the others, there was some fundamental difference. Took me about a day to implement. When some other engineer saw this, he spend the next three weeks trying to integrate all of them with some shared class. His work was not completely worthless, because he did find some small bug during all his efforts to avoid any possible code duplication. I already had predicted that it would take a lot effort, but I did not object, because I hoped that he would learn something from it and the next time think twice before always trying to avoid code duplication.

irishloop• 1 month ago

Too many abstractions are bad. Too many code duplication is bad.

Part of being a good engineer is finding the right balance.

I know engineers who would gladly duplicate code all over the code base to avoid creating a new abstraction.

I know engineers who create polymorphic abstractions for a single caller with a very obvious set of parameters.

So much of wisdom is in finding balance and not being dogmatic about rules.

lokar2• 1 month ago

I feel like the balance has shifted over the last 30 years, and is speeding up. Semi-automatic and fully automatic re-factoring has made dealing with duplicated code much faster, cheaper and safer. Changing abstraction is still high risk.

heisenbit• 1 month ago

Changing abstraction is a high risk unlike agents refactoring scores of almost identical code.

chuckadams• 1 month ago

I have regularly watched agents forget to update one duplicated pattern after changing it somewhere else. If it's within a single file or related class, it'll catch it, but if it's off in some other package in the monorepo, it's a crapshoot.

crazygringo• 1 month ago

Isn't it the opposite?

Automated re-factoring means you can refactor duplicated code only as long as it is exactly duplicate.

Whereas the whole problem is that when somebody changes 3 out of 10 of the duplicate cases in a simple way that they are no longer exactly duplicate, and then somebody fixes a bug in one of the other 7/10 cases, they can update the bug across the 7 "duplicate" cases but they'll miss the 3 that aren't.

The problem with duplicate code is always when some of the instances get changed/fixed but not all of them. And that when somebody edits one instance, they often aren't even aware of all the other instances.

Abstractions are low-risk, because you know where the code is. If it's the wrong abstraction, you can fix that and know what you're fixing. Whereas with duplicated-yet-modified code, you've now lost the connections between them.

andix2• 1 month ago

Duplication is often less harmful than abstraction.

Duplications can often be cleaned up over time, bad abstractions can quickly become a bottleneck, that severely slow down everyone working on the project.

tcfhgj6• 29 days ago

The opposite

baublet• 1 month ago

The most difficult codebases are those with every little thing some bespoke abstraction that went through 3 rounds of committee reviews that results in having to click through 12 files to figure out what anything is doing. Factory factory factories each with their own little frankenframework to understand before using anything.

TexanFeller• 1 month ago

Code duplication is far cheaper than the wrong abstraction

Very true in some sense, but I continue to encourage DRY-bias because I've literally never seen teams duplicate code responsibly and later dedupe it when it's the right time. 95% of the time this sentiment is quoted to justify shipping quick slop and stable reusable bits are never extracted into a shared lib later.

bluefirebrand2• 1 month ago

In my experience if your organization can't commit to doing WET (write everything twice) code then it probably also will fail at doing DRY (don't repeat yourself) code

Maybe this is an area where AI can help identify duplicate code though to show opportunities for de-duping

lg5689• 1 month ago

I believe that "single source of truth" is a principle that should always be followed. If there's duplicated code where it'd be a bug if they diverge, then you should refactor. It creates a long-distance coupling in your code that may be invisible to future developers until a bug emerges.

But with that in mind, I mostly agree with the article: if it's not a violation of "single source of truth", then abstractions are just a convenience. If it starts being inconvenient, then it's not doing its job and there's no reason to use it. It's a serious code smell if a function needs several flags for custom behavior; that means it's probably the wrong abstraction or violating the single responsibility principle. If there is a legit need for lots of customization, an often-good way to handle is to take a function/functor as an argument for the customization. E.g., rather than `solve(f:double -> double, max_iters = 99, x_abs_tol = 1e-15, x_rel_tol = 1e-15, ...)` you can do `solve(f:double -> double, stopping_criteria: StoppingCriteriaClass)`

jonahx2• 1 month ago

I believe that "single source of truth" is a principle that should always be followed

Fundamentally, the article addresses cases where it's not clear yet how many sources of truth there will be. Are the two spots in the code using the same algorithm, or slightly different versions? More importantly, will they change for the same sorts of reasons?

The title adage (correctly, imo) argues that making two different things the same will cause you more pain than making two same things different via duplication. In the latter thing case, the "damage" is just having to make the same changes twice, or doing a refactor to introduce the abstraction. In the former case, you have to keep adding to your abstraction, or undo it. Most crucially, it breaks "locality", which is the only property you really care about when making changes. I just want to make this change and not worry about side effects to unrelated parts of the system.

stanmancan• 1 month ago

The issue with not having a single source of truth is not the fact that you have to update code in 2-3 places, it’s that you have to know to update code in 2-3 places.

Accidental divergence is the problem, not intentional.

kread• 29 days ago

I totally agree with you. If "single source of truth" was possible in every situation, I wouldn't have been confused about first name and last name. When someone asks me first name, I always get confused whether I say first name in Korean or first name in English. I think there are two sources.

fpoling2• 1 month ago

With LLMs the cost of duplication is much lower and LLMs

cluckindan• 1 month ago

and LLMs

… sometimes duplicate things unnecessarily.

mdavid6262• 1 month ago

Of course, in theory this is true. In practice people tend to avoid ANY duplication no matter what. Especially junior developers, as if duplication would be the root of all evil.

jihadjihad• 1 month ago

as if duplication would be the root of all evil

And instead it gets replaced with the actual root of all evil, complexity.

the_af6• 1 month ago

This is something I've seen repeated time and time again as a criticism of (misused) abstraction and DRY, yet I've never seen ONCE -- and this is not hyperbole, I mean it literally -- a junior making an abstraction with any thought to reuse, generalizing anything, or caring about not repeating code. Most juniors I've worked with are content to just churn new code without paying attention to the codebase at all. This all before the AI deluge, mind you.

Very similar with patterns. I've often read people protesting that juniors overuse design patterns, yet I've seldom seen a junior (mis)use anything more complex than a singleton, and when they use any pattern, it's usually forced upon them by an opinionated Java framework.

wellpast• 1 month ago

Definitely the hallmark of junior. Obsession with code deduplication as the highest pri when it’s quite low among others.

throwaway203714• 1 month ago

The hardest part is two algorithms or business logic routines that are nearly identical. What to do? Frequently, all solutions look equally bad!

jackbucks• 1 month ago

I have always believed what the article more or less states. But you have to remember, the primary and maybe only source of duplication in software is situational dependency (the other word escapes me for this). If there was a universal tree of software functions that could be accessed over a network no function would ever be duplicated and every function would be reused from a central tree. When you put 2+2 inside a method or function body you just duplicated code. or any code inside a method or function body.

This is why we have to have programs that duplicate code by doing anything like adding two numbers together or complex logic that is easy to create bugs when someone wrote it 40 years ago better. Because code reuse is mostly done on a very small scale.

Given thats the case when you start on a new React project as an example you are not reusing application code you are duplicating the react framework so you can duplicate every other web app in every sense except maybe the visual.

There is no such thing as full reuse and until we get to a universal network invocable function tree that can be extended only when its truly unique we never will. Maybe AI will do this. People cannot.

At the end of the day code duplication needs to exist to optimize for local correctness (or incorrectness) and speed and abstractions goal is not to provide pure reuse. Its to provide a place to "put your logic" that may be similar and has access to typical state that some kind of widget might typically need.

nnoremap• 1 month ago

https://en.wikipedia.org/wiki/The_Library_of_Babel

pocksuppet5• 1 month ago

universal network invocable function tree

https://www.wikifunctions.org/ btw

alberto467• 1 month ago

Code duplication differs from single source of truth applied to data in the sense that data is data but two pieces of code may functionally be the same (they do the same thing) but they might be semantically different in their usage (they’re advertised to achieve different things), in that case coupling them together with deduplication and forcing them to do the same thing doesn’t really make sense, and may make the codebase more difficult to work on in the future (especially in companies where different teams have responsibilities over different parts).

QuadmasterXLII2• 1 month ago

One killer life hack I’ve found is, if extreme duress pushes software into two sources of truth, add a ci test that wont merge into main till the sources match. The canonical case of this actually being the best solution is pyproject.toml / requirements.txt synchronization, but I suspect it has broader applicability. A precondition is that things have already gone off the rails far enough that single source of truth is unattainable, this is more harm reduction than cure

uberex• 1 month ago

I know it is just an example but I'd generate one of those files from the other in that case.

infinitebit2• 1 month ago

i don’t think anything in the article advocates for not prioritizing “single source of truth”, as in, if we know that there are multiple sources of truth for something, it should absolutely be deduped. the article is more saying “be a bit more skeptical of any two pieces of code actually representing the same thing” and “be more willing to break apart an abstraction that is trying to represent multiple truths.”

jamiejquinn• 1 month ago

it'd be a bug if they diverge

That's a very nice rule of thumb. I've often overabstracted when two pieces of code look similar at one point in time and then they diverge.

shhshahja• 1 month ago

If you knew in advance which source of truth is important to isolate you don’t have this problem.

The problem is not knowing which of the hundreds or thousands of potential truth sources is worth abstracting. The only real way of finding out is not abstracting them and seeing how it works out.

If the problems in SWE boiled down to solve(f -> MagicallyNoProblemAnymore) we wouldn’t have this discussion.

janpmz• 1 month ago

If they diverge

This is the key, if they are very similar but used by different consumers the chance that they will diverge in the future is very high. And once they do they will break the abstraction.

tcfhgj6• 29 days ago

You can still duplicate then

Ctrl+c, Ctrl+v

namelosw• 1 month ago

I believe that "single source of truth" is a principle that should always be followed

Theoretically and conceptually I agree. But in practice there are a lot of programming languages aren’t as expressive. People prefer codebases with duplications rather than visitor patterns everywhere. In essence, visitor pattern is a tool to solve multi-dimensional abstraction problems, just like type classes in Haskell or CLOS in Common Lisp. But it’s so verbose and non-straightforward so more often than not it’s not worth it even conceptually it’s a legit case for “single source of truth”.

gf000• 1 month ago

Visitor pattern is there due to a very simply reason. You have n datatypes with m functions. FP languages makes adding a new row to this nm table easy, OOP languages makes adding a new column easy (that is, without changing every* use site as well).

Visitor pattern makes the row addition case possible for OOP languages, that's it.

joshmoody24• 1 month ago

I've seen the pendulum swing between duplication and abstraction a few times in my career, and I'm currently on team "it's usually not that hard to find a good abstraction up front."

IMO it's easier to inline a bad abstraction than it is to consolidate a bunch of subtly different things that should have been abstracted from the beginning.

But I expect people's opinions on this differ wildly based on their personal experiences. Just my anecdotal take.

luckystarr• 1 month ago

How I see this:

Refactoring code to reduce the number of lines is _compression_, akin to RLE coding.

Refactoring the code to lift conceptually coherent parts is _abstraction_.

Less compression, more abstraction. Then you're fine.

tetha• 1 month ago

I watched a talk by her about this, and this post is missing half of the equation, which is really important:

Having a wrong abstraction means you end up with a class/function/module with a huge amount of configurations through boolean/enum parameters. It's not even clear that all combinations of configurations is even valid. This situation may be simplified by duplicating, and then eliminating code, thus creating more streamlined code for each use case. This may require fixing similar or cross-cutting bugs in multiple places (eg: JSON serialization is stupid, need to hack a workaround), but keeps the business logic changes simple. Maybe a bit more numerous, but the code is able to raise all the scenarios to consider.

Having no abstraction means you may have to change business logic consistently in multiple places, or you have to fix exactly the same misconception (aka a bug) in multiple cases. e.g. tax rate management in a multi-national context. This is also terrible, because you may fix an important problem in one place and forget other places with the same issue. Now you missed 12 potential bugs by fixing one. This can however allow you to discover a true abstraction. Maybe these 12 places should call just one place?

But for code evolving across a team understanding this tension, a bit of duplication while waiting for confirmation that these pieces of code break together and change together is better than just shoving the same 3 if-statements into a function to avoid "line duplication". Concept duplication is more important.

gb2d_hn2• 1 month ago

Interface over inheritance is the paradigm I try and stick to. I'd rather maintain orthogonal code than code with overuse of inheritance because of over adherence to DRY.

dmos62• 1 month ago

If it's duplication, it's the same abstraction by definition. The fundamental unit of programming is intent, not code.

ozgrakkurt• 1 month ago

The discussion around this topic would be nicer if the title had "can be" instead of "is".

Otherwise what is better is better and we don't know what we don't know

slopinthebag• 1 month ago

I prefer the go mantra: a little copying is better than a little dependency.

Abstraction is a vague term when used here. Is a shared function an “abstraction”? It’s more like implementation hiding, maybe some data hiding. But you definitely have a dependency on it now.

Acronyms like DRY are for beginners. Once you get good you know when to break the “rules” (and when not to).

mcculley• 1 month ago

Yes, if your programming language/environment is weak.

omoikane• 1 month ago

Programmer A sees duplication.

This step should also be parameterized by how many times the duplication has occurred. Refactoring preemptively may lead to poor abstractions, but not refactoring after seeing the exact same thing tens of times would also be weird. See also:

https://wiki.c2.com/?DuplicationRefactoringThreshold

https://wiki.c2.com/?ThreeStrikesAndYouRefactor

mohamedkoubaa2• 1 month ago

Duplication is often a small price to pay for isolation

ilvez• 1 month ago

Just three words: rule of three.

LunicLynx• 1 month ago

The generic repository "pattern" is the prime example of this. There might be CRUD operations shared between repositories, but they should not be that base of every repository. I've seen this so many times, because in the beginning the CRUD stuff is what you code over and over again and then suddenly business logic emerges and everything breaks down, but the repository prevails because sunk cost fallacy ...

DmitryOlshansky• 1 month ago

I would argue that _premature_ abstraction is worse than _some_ duplication of code.

Also I’ve seen the kind of codebase that seems to be LZW packed due to the sheer desire to DRY everything out. Not pleasant thing, by the time you goto 10 layers deep on some “helper” function you forgot why you in there.

Verdex• 1 month ago

Cheaper is skipping a step.

Code duplication and 'wrong' abstractions both count themselves amongst the other foibles of programming. But they don't directly produce a cost which can be cheap or expensive.

They produce some other high dimensional intermediate value which can then produce highly variable cost dependent on the domain, goals, and scenario.

As ever, it depends.

The depends is quantifiable, but it doesn't fit in a blog post. Think more along the lines of war and peace.

time4tea• 1 month ago

You dont know immediately if something that superficially seems the same actually is.

Copy and paste once is fine, twice, not so much.

Often I've seen two totally different things exist in one bit of code, no overlap!

Premature generification is bad, and leads the developer to believe that two things are the same, making it harder to see they are not.

Also, can make it much harder to see that a different abstraction would give a cleaner outcome....

nullbio• 1 month ago

Code duplication is terrible in the age of LLMs, unless you want to maximize on drift.

andix2• 1 month ago

I always try to design in a way, that using abstractions/shared logic is optional.

I've worked in too many projects, where every new feature needs to be built on top of existing abstractions, that often lead to severe restrictions if something slightly different is required. I always try to create reusable units/components, that can either be used as intended or replaced by something that behaves slightly different if needed.

Components are not necessarily frontend components, this extends also to backend logic.

johnwheeler• 1 month ago

These are not mutually exclusive.

dang3• 1 month ago

Related. Others?

The Wrong Abstraction (2016) - https://news.ycombinator.com/item?id=35927149 - May 2023 (69 comments)

The Wrong Abstraction (2016) - https://news.ycombinator.com/item?id=27095503 - May 2021 (17 comments)

The Wrong Abstraction (2016) - https://news.ycombinator.com/item?id=23739596 - July 2020 (240 comments)

The Wrong Abstraction (2016) - https://news.ycombinator.com/item?id=17578714 - July 2018 (207 comments)

Prefer duplication over the wrong abstraction - https://news.ycombinator.com/item?id=12061453 - July 2016 (96 comments)

The Wrong Abstraction - https://news.ycombinator.com/item?id=11032296 - Feb 2016 (119 comments)

jeffypoo• 1 month ago

I've always told engineers to duplicate until the abstraction is punching them in the face.

dang3• 1 month ago

I dislike duplicate code as much as anyone, but agree with the OP that bad abstractions can be worse. They add confusion and complexity which compounds over time, since people are forced to build on top of them in ways that (by definition) don't suit the underlying domain and ultimately become self-referential. This leads to contortions, workarounds and even more bad abstractions which ought not to be there—they're reactions to the code not fitting the problem, or as Fred Brooks called it, accidental complexity. You end up in an evolutionary dead end where the system is hard to extend because it's too hard to understand.

I've learned to tolerate a small amount of duplicate code for this reason. If the duplication remains small, it's not that harmful, and if it starts to grow, one has a better shot at finding a good abstraction for it. Bad abstraction is premature abstraction.

One thing I'm not sure this thread has mentioned yet is how LLMs alter the cost-benefit curve of this. They are much better at managing duplication than humans are, and much better at noticing inconsistencies - the sort of small bugs which duplication traditionally leads to. I don't know if this is enough to count as a different kind of good abstraction; I doubt it. It reminds me of a petroleum economist I once knew who had 200 duplicate spreadsheets analyzing different projects and who hired a junior analyst to keep them all consistent. An LLM would be like the junior analyst.

vcryan• 1 month ago

The sweet spot is really duplicating the wrong abstraction: I see you Claude!

stcg• 1 month ago

This is like saying "A slow leak is cheaper than a burst pipe"

Yes, okay. But with both you will have a bad time cleaning up.

There is a third option: good abstractions.

I did see this pattern described in the blog in practice a lot (and fell victim to it myself) and I think that in general this comes down to inexperienced programmers. Object oriented programming makes it worse.

Teaching these programmers that they should not abstract is not the solution. It is blocking their growth.

Teach them how to make better interfaces instead.

dang3• 1 month ago

The OP is aware of good abstractions and is describing a procedure for finding them, or for increasing one's chances of finding them.

zadikian3• 1 month ago

The saying is you build better abstractions if you don't build them too early.

bogrollben• 1 month ago

One thing I don't see talked about often is the fact that not all duplication is equal. Duplicated html/xml/markup does not equal template-based boiler plate, which does not equal almost everything else. I'm far more forgiving of duplicate html/markup because that code is so cheap.

alkhimey• 1 month ago

The disadvantages of duplication are greatly reduced in the world of AI. From my experience it can easily detect the duplicates and refactor code safely. On the other hand, code without abstractions is easier to read and easier for AI.

With AI, we really need to rethink the clean code principles.

more-coffee• 1 month ago

Seems to me like the last thing you want to do is worry whether the LLM has a large enough context window to keep an eye on all duplicates. So I'd argue to deduplicate directly, where possible.

aarjaneiro• 1 month ago

Personally I've seen way more duplication as a result of AI in large codebases

felooboolooomba• 1 month ago

The bad thing with abstractions is when you start it too early in your code base for things. It's also a bad thing when you start it too late, although not as bad. If you start it way, way, way too late it's very, very, very bad.

Of course, the worst abstractions are the ones you don't need at all.

fpoling2• 1 month ago

With LLMs the cost of duplication is much lower both to write and maintain. So abstractions needs much higher justification.

ninalanyon• 1 month ago

You have some evidence for that assertion?

lazide• 1 month ago

Yes, but counterpoint - code duplication is also the wrong abstraction.

Pro tip - which is the least bad abstraction? Answer: it depends!

zadikian3• 1 month ago

I always felt like I abstract and modularize things way less eagerly than other programmers. Was pleasantly surprised to find that LLMs do it mostly my way by default, then again they're also bad at abstracting when it's actually needed.

dan-robertson• 1 month ago

I think LLMs are trained to not refactor. I think it’s either that you would need to do something in training to make them want to do it and the labs don’t do that, or that the labs correctly guess that it would be very annoying for LLMs to go and refactor your existing code as they go. This creates bad effects (eg crazy hacks to avoid refactoring and, much worse, not refactoring the code they only just wrote as required) but I think the alternative would be worse – it’s not something you always want to read and the refactoring is often done incorrectly, restructuring the code to the best shape for the current task rather than something that balances many different needs.

davnicwil• 1 month ago

One way I like to think about is that often abstraction is an automation for a task that doesn't need automating.

You hardly ever change the thing and if you do, changing it in two or three places 'manually' is really not a big deal.

Now changing something fairly often, that affects logic in 50+ places? Then it makes sense to automate with an abstraction so it all flows through the same lines of code.

I know I've personally spent way more time over the years debugging bad abstractions than changing things in a few places.

hakunin• 1 month ago

I think what this advice is really getting to, is that you should prefer everything generally build-time/hardcoded/static rather than runtime/dynamic. Wrote about this in 2013 (calling it a CMS trap[1], back then seemed pertinent).

[1] https://max.engineer/cms-trap

Waterluvian2• 1 month ago

I think about this on occasion. Most recently I ran into an issue during a personal project: 2d sprites for RTS units were packed on spritesheets in a consistent manner: 5 sprites for 8 directions (you mirror 3). Packed in order of: stand, move, attack, die. So I made a loader that understands how to take action + direction and offer an array of sprites to play through.

But then I came across more cases: sprites with no directionality (an explosion), and corpse sprites (which were only 4 directions, 2 mirrors, and most except the first four were shared by both orcs and humans).

I agonized for a little bit on what the hell the common abstraction is for all this. In the end, I factored out some of the loading code, and made a UnitLoader, CorpseLoader, EffectLoader and moved on. Now, there's probably a better abstraction in there because all 3 loaders have to reason about the same things a little bit. But I will discover that abstraction later on and it's easier to just de-duplicate the code then, rather than try to identify the abstraction now and make some complicated EverythingLoader that handles all those cases.

galleywest200• 1 month ago

This is the way. Making games is supposed to be fun. You can do the hard boring stuff when you get to the final 10% of the project.

Besides, sometimes your duplication creates "bugs" which may turn out to be fun features that players enjoy.

andai2• 1 month ago

I like this quote, "things should be made as simple as possible, but no simpler."

I think the natural instinct with programming is to try and simplify the code by means of generalization. But we often over-simplify, and reality is messy. Or as TFA mentions, time passes and new requirements arise, so it turns out that we have simplified prematurely!

Sounds like this should be an aphorism. Premature abstraction is the root of much suck!

dahart2• 1 month ago

You probably already have the common abstraction factored - the code to load pixels for a single sprite, and to display it? It makes sense to me that the level above that, interpreting the sprite sheet layout and modes of playback, come in different flavors and don’t have a common abstraction that fits all cases.

Personally I prefer what you’re doing over trying to come up with a non-obvious abstraction or trying to make an imperfect abstraction fit. Waiting til the abstraction is totally obvious and the need is crystal clear is a good thing.

The flipside (antidote?) of DRY is WET - write everything twice/thrice. More important, IMO, is to abstract only over things I have an actual, demonstrated use case for, usually demonstrated first via duplication, and not speculate about possible future uses I might want. Code written for future use cases we don’t have is so often the code that gets in the way of abstracting the things we do have, and it cracks me up when that happens.

Waterluvian2• 1 month ago

Waiting til the abstraction is totally obvious and the need is crystal clear is a good thing.

I discovered this after a few early years of my career being a bit of a “best practices” zealot. The thing I say often at work is, “let’s get this shipped to prod so we can start learning all the things we don’t yet know about it.”

_pdp_• 1 month ago

The biggest mistakes young engineers make is working out a problem from bottom up... i.e. building frameworks and libraries, rather than exploring the problem space which is more chaotic.

You cannot find the edges of the system with structure you don't understand because once the abstraction are set in place solutions often have the same shape as the frameworks which leads to ultimately really bad systems.

The best way is often not the obvious way. Once you reach the edges then you can think how to program the abstraction but that is many versions down the line from the original.

stevefolta• 1 month ago

"If you have a procedure with ten parameters, you probably missed some." -- Alan Perlis

aftbit4• 1 month ago

Similarly, I've seen some developers who seem to think that any inline string or numeric constant is evil. In one PR, I saw:

    HTTPS_SCHEME = 'https'
    DOMAIN = 'www.example.com'
    
    url = HTTPS_SCHEME + '://' + DOMAIN

I don't understand what they think this is buying, other than just cargo culting "don't embed constants." And of course, the constant definitions were at the top of the file and the url building code was hundreds of lines away.

preg_match2• 1 month ago

I’m a big fan of closeness in code. I prefer defining things as closely to where it’s used as possible. This is a big pet peeve for me!

Do not put regex at the top of the file either! Put it where you use it. Languages are smart, they’ll probably be able to tell that it’s constant anyway.

Also for tiny functions just use a lambda. Please don’t make a one line function a million miles away that you use once or twice.

robotresearcher3• 1 month ago

Amen! The existence of 'helpers.js', 'utils.cc', makes me twitch.

what• 1 month ago

If multiple things use the same regex, which one should it be close to? Or do you propose duplicating it?

mrkeen• 1 month ago

I ran into this as well. If an Event has a name, you can instantly grep across a giant monolith (or a big folder of microservice repos) and find every file that is concerned with that event.

If you pull it out into a constant, you're back to opening up projects one-by-one to 'find usages'

aftbit4• 1 month ago

grepability is underrated. This is one of the reasons I dislike passing functions into interfaces - it makes it harder to follow the call chain when the functions are constantly renamed.

Luckily there are often solutions that just come down to which arbitrary labels that you assign. e.g. compare these two blocks

    do_thing = do_thing_one_way if condition else do_thing_other_way
    process(data, do_thing=do_thing)

versus

    thing_processor = do_thing_one_way if condition else do_thing_other_way
    process(data, thing_processor=thing_processor)

The first is much more greppable - search for "\bdo_thing" all over the code base. The second requires realizing that the processor function renames the internal function to "thing_processor".

dahart2• 1 month ago

Having the constants at the top is more easily customizable, especially should this file get duplicated. If devs need to switch to http instead of https for testing or staging, it makes sense to separate the scheme from the domain and put the constants up top or even in another file. It also matters whether ‘url’ was constructed in multiple places or a single place. Having named constants at the top of the file is a very common style, and sometimes is part of the group coding standards.

Anyway, maybe there are other reasons too, so see Chesterton’s Fence. In any case, it’s never a good idea to assume cargo culting. Someone could easily say the same thing about using inline literals. If it looks weird, ask around and maybe you’ll find out there are good reasons, or maybe you’ll find out nobody cared and that people will like it if you refactor and embed the constants.

aftbit4• 1 month ago

In my opinion (and it's just that - an opinion, and mine - yours may differ), it's better to make the code as stupid simple as possible. When you build it, don't assume future change in that spot, because you could be terribly wrong about exactly what would change, so you're adding complexity for no benefit. The first time you need to change the scheme from `https` to `http`, just change it inline in the URL building code. The second time you need to do it, make a constant or an env var.

Over time, everyone develops their own intuition and opinions on what sorts of change and refactors are likely, and which sorts will never come. It's not perfect - it's part of the art of software, not the science. I'm not going to claim I've never written an interface that had a useless cut-point - I definitely have written many.

In my opinion, it's better to err on the side of simplicity, understandability, and closeness than to prematurely factor out those constants.

It's a judgement call - every situation and every practitioner will have a different response. In this case, knowing this product, and as the code reviewer for this patch made by a newer engineer (senior but new to the team), I was very confident that we would never need to change the scheme, and that the extra line and extra cognitive load of groking `HTTPS_SCHEME + "://"` over just `"https://"` or `BASE_URL + blah` was not worth it.

I suggested that the engineer either rewrite it with an inline constant, or refactor the whole base URL out without separating the scheme.

regularfry• 1 month ago

That particular example doesn't quite fit, but I've certainly seen cases where otherwise perfectly ordinary fixed strings needed to be broken up to meet linting rules.

pocksuppet5• 1 month ago

If your linter is making you do something stupid fire your linter or learn how to exempt a certain line from a certain rule.

aftbit4• 1 month ago

Ah yeah those are bad linting rules then. That's one of my pet peeves. Inconsistent style is better than style forced into jankiness. Also... linters should auto-format whenever possible. I'll make a few exceptions, but broadly I like to set the enforcement of linting rules to only those cases that are honestly confusing or misleading.

andai2• 1 month ago

See also: Muratori, Semantic Compression ("Compression-Oriented Programming")

https://caseymuratori.com/blog_0015

Previously discussed:

https://news.ycombinator.com/item?id=17090319

https://news.ycombinator.com/item?id=36455794

https://news.ycombinator.com/item?id=46183091

infinitebit2• 1 month ago

I feel this deeply. Although abstraction isn’t a one way door, “deduplicating” logic tends to be much easier than breaking big functions back down, and so these days I tend to leave a comment with the date wondering if it is too similar to some other code. then if i come across it again months later and it still is, then maybe it is safe to make it DRY.

I think DRY is the first heuristic for “good code” that most junior devs actually grasp, and so they become very dogmatic about it for a while

threethirtytwo• 1 month ago

Duplication and the wrong abstraction are looking more and more to be implimentation details handled by AI. A possible future may be a place where none of this matters.

hyperpallium2• 1 month ago

prefer semantic abstraction even when it creates duplication

jongjong• 1 month ago

Also, I prefer having all the code inside a single 5000-lines file than split up into many small files representing incorrect abstractions.

The urge to split the code up since they beginning is generally a bad idea; it forces early abstraction; more likely to be wrong.

he0001• 1 month ago

If you can’t fit into the same abstraction, is it really a “duplication” or is it just a slightly, but incomparable, function?

ketozhang2• 1 month ago

I like to think most seniors know to not blindly follow DRY. However, I can tell many of us are uncomfortable with the idea of needing to maintain multiple duplicated sources of code.

To help with that, I think the simple model of two callers depending on a common code needs to be scrutinized. If the common code needs to change because only one of the caller needs it, then it doesn’t belong in the common.

The wrong goal for DRY is attempting to do it with encapsulation. Encapsulation shifts the refactoring work from the caller to the common code. However this is not what you want because there’s a lot more consequence in updating the common code than the caller.

You can avoid encapsulation and still be DRY by having multiple thin abstractions that the caller needs to be aware about is better. In OOP you are taught SRP and IoC for this. In procedural programming, this just comes naturally as code calling series of helper functions.

corysama• 1 month ago

I always liked the advice "Abstract for replacement. Not for reuse."

If you have code that is reusable, you'll want it to have a nice interface. But, you don't need an abstraction on top of a nice interface. Just use it.

For abstraction, what you need to focus on is "What is most likely to change in the future?" You want to put in abstractions that will make those changes low-cost.

Ex: At work there was a small debate about which C++ JSON parser to use with no stand-out winner for our framework's needs. So, we picked one and I put a thin layer over it for everyone to use. We have since then swapped out the parser and swapped it back over the years of a hundred devs using it in our framework and no one noticed the swaps.

ninkendo• 1 month ago

To me it’s distracting to think about duplicating vs creating an abstraction, because the answer is always “it depends”, which is not really an answer.

To me, the question is: can you look at this abstraction and understand why it exists, without knowing who’s calling it? If so, it’s probably fine.

If an abstraction only makes sense because of the particular weird details of these 3 callers that have to pass mutually exclusive arguments to it to get their desired behavior, it’s probably wrong. An abstraction needs “a place to live” in your architecture. It needs to be self-evident in justifying its existence.

If you find yourself repeating code, but de-duping it would create these sort of weird non-self-justifying abstractions, your architecture is probably a bad fit for the problem you’re trying to solve. Maybe that’s because the problem changed since the software started (which is a bit of a pickle: do you re-architect, or do you continue writing weird inscrutable code?) or maybe it’s because you just picked the wrong abstraction in the first place. But you should recognize it: duplicating vs wrong-abstraction is about choosing the lesser evil. If the abstraction was a natural fit for the problem, you wouldn’t need to answer this question in the first place.

codr7• 1 month ago

I certainly learned this the hard way.

When I started writing code 40 years ago, I used to over estimate my understanding and abstraction skills a lot. As a result I created overly complicated and difficult to maintain/evolve solutions.

Turns out I need to see more examples of patterns before making good choices, which means becoming comfortable with seeing and tracking duplication over time.

throwatdem12311• 1 month ago

My company paid to have her do one her day long workshop at HQ. Don’t know how much they paid but it was worth every penny. Changed my life.

gaigalas• 1 month ago

I like the mantra: "prefer duplication over the wrong abstraction".

Combined with another interesting idea "whatever I dislike is wrong", it makes me _always be right_, which is awesome. I can never lose a discussion about abstraction with this powerful combo.

skreem• 1 month ago

Nice to see Sandi mentioned! If anyone liked her philosophy / writing style I highly recommend you check out her books

I read “Practical Object-Oriented Design (POODR)” ages ago at this point, but it reshaped how I approached OOP

Granted… OOP as a default paradigm has fallen out of favor (at least for me), but it’s still everywhere & won’t be going away. She gives a great framework for making it sane

lericzhang• 1 month ago

Of course, as you said it's "wrong abstraction". The real problem is it's hard to tell if an abstraction is correct before you see enough duplication.

danpalmer2• 1 month ago

That's exactly what the article is about.

danpalmer2• 1 month ago

AI: Why not both?

digitaltrees2• 1 month ago

Sandi is amazing. I learned so much from her.

moi2388• 1 month ago

I don’t mind duplication at all. I mind undiscoverable duplication.

But if I have an interface and three subclasses with duplicated or almost duplicated code, this is quite easy to find.

That’s much nicer than an abstract base class where only some children override the methods, because now I need to check which ones actually do.

jwpapi• 1 month ago

Whilst I understand cases in which duplication is preffered. I generally think abstractions are underused. Sometimes I would abstract something away that is only done once, not because i want to have less code, but because it allows me to solve bigger problems and when i look at a function I don’t have to worry about it. It allows to create systems. Obviously your abstractions should be good

esailija• 1 month ago

Another way to put it:

Things that should be tightly coupled but are not is preferable to things that should not be tightly coupled but are.

I agree especially because coupling things is easier than uncoupling things.

so-cal-schemer2• 1 month ago

If you know..

Software Design for Flexibility: How to Avoid Programming Yourself into a Corner

by Chris Hanson and Gerald Jay Sussman

https://archive.org/details/software-design-for-flexibility_...

insaneisnotfree• 19 days ago

That's why you never hire juniors as tech leaders nor "architects"

Prefer duplication over the wrong abstraction (2016)

Related Stories