Solve Less General Problems

https://hacksoflife.blogspot.com/2018/08/solve-less-general-problems.html

by kiyanwang • 8 years ago

193 82 8 years ago

hacksoflife.blogspot.com

RyanShook• 8 years ago

Can someone better explain the leaky abstraction concept? I take it to mean an abstraction that really isn’t.

V-2• 8 years ago

It's basically an abstraction that fails to properly abstract away the (complex) reality behind it, which leads to things being more, instead of less, difficult to grasp.

https://www.joelonsoftware.com/2002/11/11/the-law-of-leaky-a...

salawat• 8 years ago

A leaky abstraction colors your approach to solving problems in ways above and beyond what is minimally needed to solve the problem at hand.

For instance, when the author created an HAL instead of a specific problem specific driver, the HAL meant that he had to solve 2 problems instead of 1 The end product had to work, and had to be compatible with the HAL as implemented. The abstraction leaked.

Just making the driver would have yielded a solution to the problem that would"t have required as much refactoring of old code. It's a way of saying he made more work for himself than was necessary.

A very YMMV observation.

vvanders• 8 years ago

All abstractions leak just in different degrees. Perf/memory is the most common but there are others as well.

Your ivory tower parser/data format/framework is great until you need to fit it in 1/4 the memory or MIPS. That's when you start pulling out the abstractions to get back to where the product actually needs to be.

tabtab11• 8 years ago

One of the biggest industrial-scale examples of abstraction failure is OOP GUI's. They were an abstract interface, meaning in theory they could adapt to ANY future UI interface or standard. But the "the Web" came and took a shot-gun to it by not preserving program state, and by mostly forcing one to let the client determine the final layout (positioning). The existing OOP GUI engines ASSUMED the engine controlled both state and final positioning, because it had usually been that way in the past. But Web-land said, "Sorry, Dave, I cannot do that.", and so the engines were largely abandoned in back dumpsters. (Web UI can turn one prematurely grey, but that's another story.)

ben509• 8 years ago

A leaky abstraction is one that mostly works, but details and assumptions leak through.

For example, many languages attempt to provide a cross-platform API for filesystems; they give you an algebra that lets you construct and manipulate paths.

So if you have a Path, the `/` abstract operator might join path parts, and "basename" and "dirname" are functions that extract parts of the pathname.

Typically, the API designer makes some intentional (and some unintentional) design decisions to keep the API comprehensible but at the expense of being inconsistent with the underlying reality being abstracted.

As an example, some filesystems are case-insensitive, others aren't, so to know if two paths refer to the same file, e.g. if you want to use paths as keys, you need to determine what volume each is on. This gets complicated in Unix due to symlinks (which can be any component of a path) and mounts that can be anywhere in the tree. You could resolve a path to an inode, but then other filesystems might not provide an inode.

tenaciousDaniel• 8 years ago

By this logic though, aren't all abstractions leaky in the end?

edit: not arguing, just curious. I haven't thought much about abstraction but it seems like it could be a truism that every conceivable abstraction could succumb to this problem in some way.

wsy2• 8 years ago

While I agree with the author with respect to his main point, I think the term leaky abstraction side-tracks the discussion. It is simply about wrong abstractions.

nextos• 8 years ago

A good example are SQL queries. You can have two semantically equivalent queries i.e, two queries that will always return the same results.

However, runtime may be dramatically different. One may take a second, and the other one days.

That's because SQL is an imperfect abstraction. It isolates you from the underlying details on how to compute the results of a query, but only to some extent. On many occasions you need to actually understand how the declarative query is translated into an execution plan.

parsnips• 8 years ago

So sorting is a leaky abstraction? Because merge sort and bubble sort can both have the same interface, isolating you from underlying details...

maltalex2• 8 years ago

This is one if those pesky differences between “computer science” as it’s taught in college, and actual software engineering.

I believe that bigger parts of our education should revolve around reading and analyzing professionally written code as opposed to solving completely artificial problems.

wilsonnb3• 8 years ago

Agreed. What really needs to happen is the splitting of software development and computer science into two related but distinct education tracks.

k_sh• 8 years ago

One could say that bootcamps were the first to pave the software development track.

cshenton• 8 years ago

Isn’t this the difference between computer science and software engineering courses?

tabtab11• 8 years ago

From a lifetime of coding I indeed came to a similar conclusion: "Micro-abstractions" are more useful in the longer run than large-scale abstractions. Nobody can predict the future accurately: I've seen many Grand Abstractions byte the dust face first. BUT, with micro-abstractions you can often re-use a decent portion of them when the future doesn't go as planned. Unlike large abstractions, you don't have to marry micro-abstractions: you just date them. If any one doesn't work out, you can ignore and replace it without much penalty: no messy divorce. I've built up a library of "handy" functions and mini-classes that I can borrow as needed per project. I've translated the commonly re-used ones into different programming languages.

Further, one can experiment with different micro-abstractions without investing too much time. They are micro-experiments. If one fails to provide re-use or flexibility, so be it: just stop using it. It's like picking stocks: some stocks will wilt, but if you buy enough, you'll roughly approximate the general market at about 10% annual growth.

I do find dynamic languages make such mixing and matching easier, though, because they are less hard-wired to a specific type or iteration interface: less converting needed to re-glue. I've yet to get as much re-use in type-heavy languages, but not ruling it out. I just haven't cracked that nut yet. Maybe smarter people can.

kjeetgill• 8 years ago

Beautifully put. It resonated with me about not over abstracting too early.

Abstractions usually come earlier in the design cycle than optimization for me and the wrong one makes that later stage harder.

I do find dynamic languages make such mixing and matching easier, though, because they are less hard-wired to a specific type or iteration interface: less converting needed to re-glue. I've yet to get as much re-use in type-heavy languages, but not ruling it out.

I go back and forth. I mostly develop in a mix of Python and Java. Python may give me a lot of room to explore ideas, but I find refactoring a fragile nightmare. Java's type system is fairly simple and lets me quickly pull up all use sites and trivaly catches misspellings.

It lets me back out of and refector the wrong abstraction much more simply.

tabtab11• 8 years ago

I don't know why there are not more code analysis and refactoring engines for dynamic languages. Sure, it takes more guessing on part of the tool, but I see no reason why they can't identify and help with a large portion of such tasks. It shouldn't be an all-or-nothing thing. Sure, the more "meta" your programming, the harder it is to automate analysis, but only a portion of an application typically requires high meta.

adamnemecek• 8 years ago

I’ve been saying and doing the same since forever. What are some of your commonly used functions?

tabtab11• 8 years ago

I'll pick a couple that are easy to describe. First a simple string function: "softEquals(a,b)" that compares ignoring case and leading and trailing white-space, similar to "return trim(upper(a))==trim(upper(b)). The second is to "draw" an HTML button and/or hyperlink because often the user or designer change their mind about whether it's a link, button, or icon: link(title, url, httpType=[hyperlink,submit,get,reset], dispalyType=[hyperlink,button,icon],target="",classes="",otherAttribs="",buttonImg=""); Those with "=" are optional named parameters or optional object values, depending on the language of implementation. I'll usually tune it to app or shop conventions, which can shorten the range of variations or likely defaults, and thus simplify the interface. But the implementation is mostly the same: just the interface needs trimming/tuning. Or just leave the fuller one intact for edge cases, but create short-cut versions better fitting local conventions.

wwweston• 8 years ago

Interesting.

Could we say the much-debated ORM is (usually) a large-scale abstraction?

tabtab11• 8 years ago

Yes, and ORM's are often a painful failure point when they don't work as expected unless you really master their guts. In my opinion it's best to code your ORM queries such that if the ORM is not doing something right, you leave your options open such as using direct SQL, stored procedures, or views. In other words, don't overly rely on ORM's both for the total application stack and individual queries. Have and test a Plan B.

I've built up a series of "SQL helper" functions to do a lot of the SQL syntactic and Prepared Statement grunt-work to avoid ORM's. They are NOT intended to entirely wrap or hide the RDBMS, but rather help with the app's interaction with the RDBMS. Helpers rather than wrappers. If the helpers don't work in a particular case, you can ignore them with minimal code rework. If you depend heavily on an ORM, such is often not the case.

munchbunny3• 8 years ago

I think that's fair.

On the more general point, I think that the larger scale the abstraction is, the larger the number of actually encountered use cases needs to be. You shouldn't write an ORM because you feel like existing ones don't serve your specific needs. You should write SQL, and then replace it with an ORM once it's clear you will benefit from the extra abstraction.

There seems to be a fairly consistent trade-off between the scale of the abstraction, the degree of over-engineering, and the leakiness of the abstraction. So if you're going for a large scale abstraction, then I'd want lots of proof that it's not a leaky one.

groue• 8 years ago

We start to see ORMs like Diesel (https://diesel.rs) or GRDB (http://github.com/groue/GRDB.swift) that radically simplify their domain by removing traditional ORM features like implicit uniquing, lazy-loading, or auto-updates. This of course gives more work to do on the client side. But this work is positive and often simpler than expected. With traditional ORMs, there is still some work to do, which often looks like fights: against misunderstandings, against leaky abstractions, against configuration levels that don't match the needs of application, and generally speaking against the framework.

ORMs are not done yet :-)

abhgh• 8 years ago

This resonates with me. For my long-running Python projects I usually have a file called small.py which has these, as you put it, micro-abstractions. Over time some stuff from the file gets into the larger projects ... and some stuff doesn't. But I like the comfort of "cognitive offloading" this file affords me: I don't have to keep worrying about increasing the scope of my abstractions.

ken2• 8 years ago

I do find dynamic languages make such mixing and matching easier

Yes! I think all of the people I've met who advocate the article's position were using static languages that make it harder to re-use what they've written. This article sounds like it's about a C++ project, which doesn't surprise me.

Where are all the articles about this happening to dynamic data-driven projects? In contrast, people seem to go to incredible lengths to keep old Lisp code running. A lot of people (even non-Lispers!) would apparently rather write a new Lisp interpreter than port their Lisp code to another language -- because the Lisp code is largely a declarative description of the problem and solution.

Data itself is already a Grand Abstraction that has survived the test of time. I don't need a library of mini-classes.

gowld3• 8 years ago

Could be selection bias. People who use lisp today are people who refused to move on over the past decades when almost everyone moved on.

tabtab11• 8 years ago

The boundary between data and behavior is fuzzy, and one can often be remapped into the other. A common example is a "field", such as a screen field. We can represent it as a structure similar to: field(dbName="socSecNo", screenName="Social Security No.", type="string", mask="999-99-9999", required=true...); Whether that's a method call or "data" doesn't make much difference. Ideally we want to define it once, and that's our abstraction.

Lisp does make "swapping" between data and behavior easier, but many say Lisp code is just hard to read in practice. I've been in Yuuuge debates about Lisp, but my summary conclusion is that many just find existing Lisp code hard to read. Perhaps its too much abstraction. Short of eugenics, we are stuck with that problem. Lisp has had 50+ years to try mainstreaming; time to give up.

elvinyung3• 8 years ago

The interesting thing is that the programmer's tendency to abstract and future-proof their code mirrors the late 19th and early 20th centuries' high modernism movements, as James C. Scott analyzes in Seeing Like A State and Michel Foucault discusses in depth in Discipline and Punish and others.

"Modernism" is basically characterized by attempts to predict and control the future, to rationalize and systematize chaos. In the past this was done with things like top-down centrally planned economies and from-scratch redesigns of entire cities (e.g. Le Corbusier's Radiant City), even rearchitectures of time itself (the French Revolution decimal time). The same kinds of "grand plans" are repeated in today's ambitious software engineering abstractions, which almost never survive contact with reality.

fierro• 8 years ago

Amazing parallel

rtsao• 8 years ago

https://www.deconstructconf.com/2017/brian-marick-patterns-f... is an interesting talk on this topic, which is where I learned about Seeing Like a State. I've just finished reading the book and found the parallels between software engineering and the high modernist statecraft and agroeconomic policy discussed in the book to be quite striking. I thought the middle chapters were a tad repetitive, but overall it was a fascinating read and I would highly recommend it.

your-nanny• 8 years ago

Actually, I've thought, for a few years now, that software engineering principles, like loose coupling, have a natural application to just the sorts of problems _Seeing Like a State_ talks about.

panic• 8 years ago

And if these abstractions do survive contact with reality, it's often reality that has to change to adapt to the abstraction. You find yourself saying, "the computer can't do that", not for any fundamental reason, but because the abstraction wasn't designed with that in mind. The unfortunate users have to change the way they work to match the abstraction.

Seeing Like a State describes a similar situation, where people had to give themselves last names so they could be taxed reliably. I wonder how many people have had to enter their Facebook name differently for it to be considered a "real name"? Software that "sees" the world in a particular way is a lot like a state bureaucracy that "sees" the world in a particular way, especially when this software is given power.

thestoicattack• 8 years ago

Don't forget the planned city that actually got built -- Brasilia. Niemeyer was a disciple of Le Corbusier.

twic• 8 years ago

The same kinds of "grand plans" are repeated in today's ambitious software engineering abstractions, which almost never survive contact with reality.

Yonks ago, i was reading an introduction to metaphysics at the same time as an introduction to Smalltalk, and it struck me that the metaphysicians' pursuit of ontology was quite similar to the Smalltalker's construction of their class hierarchy. The crucial difference, it seemed to me, was that the metaphysicians thought they were pursuing a truth that existed beyond them, whereas the Smalltalkers were aware that they were creating something from within them.

It's very likely that my understanding of one or both sides was wrong. But ever since then, i've always seen the process of abstraction in software as creating something useful rather than discovering something true. A consequence of that is being much more willing to throw away abstractions that aren't working, but also to accept that it's okay for an abstraction to be imperfect, if it's still useful.

I, probably arrogantly, speculate that metaphysicians would benefit from adopting the Smalltalkers' worldview.

I, perhaps incorrectly, think that the ontologists' delusion is endemic in the academic functional programming [1] community.

[1] Haskell

derefr• 8 years ago

But ever since then, i've always seen the process of abstraction in software as creating something useful rather than discovering something true.

Software, in my personal experience, is closest to the study of mathematics: there is an arbitrary part (the choice of axioms)—but, once that part is in place, you must obey those axioms and discover what facts are true within them.

If you don't obey your own chosen axioms, the system you will create will be incoherent. In math, this means it just fails to prove anything. In software, this means that it might still be useful, but it fails to obey the (stronger) Principle of Least Surprise.

The regular PoLS is just about familiarity, effectively.

The strong PoLS is more interesting. It goes something like: "you should be able to predict what the entire system will look like by learning any arbitrary subset of it."

The nice thing that obeying the strong PoLS gets you, is that anyone can come in, learn the guiding axioms of the system from exposure, and then, when they add new components to the system, those components will also fit naturally into the system, because they'll be relying on the same axioms.

seppel2• 8 years ago

the French Revolution decimal time

The French were 99% successful with their decimals everywhere (everything except time) in 99% of all countries (everywhere except the US and some freak dictatorships).

The same kinds of "grand plans" are repeated in today's ambitious software engineering abstractions, which almost never survive contact with reality.

Unless these "grand plans" just standardize and compromise the right things. Then they conquer the world (PC, USB, TCP, HTML, ...)

elvinyung3• 8 years ago

Seeing Like A State actually goes into a tremendous amount of detail about what a monumentous effort it was to introduce decimal measures in France. Progress was slow and took generations. It is only in hindsight that it was successful.

The state could insist on the exclusive use of its units in the courts, in the state school system, and in such documents as property deeds, legal contracts, and tax codes. Outside these official spheres, the metric system made its way only very slowly. In spite of a decree for confiscating toise sticks in shops and replacing them with meter sticks, the populace continued to use the older system, often marking their meter sticks with the old measures. Even as late as 1828 the new measures were more a part of le pays legal than of le pays reel. As Chateaubriand remarked, "Whenever you meet a fellow who, instead of talking arpents, toises, and pieds, refers to hectares, meters, and centimeters, rest assured, the man is a prefect."

I would imagine likewise for the other standards that you mentioned. And in a lot of ways they succeeded exactly because they weren't (thinking specifically about how HTTP/HTML succeeded because it was less comprehensive than, say, Xanadu).

zakm• 8 years ago

Other examples of us trying to rationalize and systematize chaos: "the heliocentric model of the solar system", "physics", "economics", and "science."

I'm glad we try to understand the world and bend it to our will.

gowld3• 8 years ago

Feels different when "the world" is other people with their own lives, hopes, and dreams. In that case be ding it to your will called authoritarianism.

zakum1• 8 years ago

This is a fantastic description. As smart, analytic thinkers it is so easy to generalise. My experience is that we should generalise when we are creating conceptual designs, but be very lean and specific in the implementation.

adiusmus2• 8 years ago

Build one. Build another. Now you have built two. Hopefully you learnt something and now you know where things are headed. Now you can write an abstraction. Now you can rebuild the first two using the abstraction along with the next. If you’ve done the abstraction right all three now share and use a commonality that helps clear up and simplify.

chrchang523• 8 years ago

https://en.wikipedia.org/wiki/Rule_of_three_(computer_progra... lines up better with my experience: postpone writing the abstraction until you are actually implementing the third thing. With just two cases, there will generally be a bunch of incidental commonalities that are better left out of the abstraction, and it's frequently not obvious which ones they are.

adiusmus2• 8 years ago

Nice to see it codified. Sometimes you don’t realise you need or decide you want to abstract until you’ve hit 5 or more.

bovermyer• 8 years ago

This is my favorite (and least useful) quote from the article:

I made the right decision, after trying all of the other ones first - very American.

It's not my favorite because it's snarky, or because it makes a generalization I think is accurate. It's my favorite because it has an element of the person behind the technical article.

Too often, people writing about technical subjects get lost in the topic, and forget that the act of writing for an audience is a human conversation.

ken2• 8 years ago

I'm going to risk being contrarian and say that I don't see the HAL as the problem. The problem I see is "the product was canceled, I was moved to a different group, and the HAL was never used again". When the product is cancelled, it doesn't matter what you built, or how good or bad its architecture was. It's dead.

Our industry has an organizational structure for shipping software today that makes every line of code have zero value until it's in a consumer product that ships, and maybe even then it'll be cancelled next month and its value drops to zero anyway. We've long known that most software projects fail. What's wrong with this picture? Why do we continue to work in a system where the most likely result of any day's contribution is literally a useless waste of time?

It seems as though the problem was short-sightedness, and the proposed cure is to be even more short-sighted. (You're looking down as you walk, and you occasionally glance up. While looking up once, you tripped over a stone. The solution is to focus even harder at the 6 inches in front of your shoes -- then you'll never run into trouble!) Is it any wonder that most software projects fail?

We joke about "resume-driven development" but economically that's the only certain gain from any new software project. I'd be foolish to optimize for the lottery ticket of "successful product", instead of the solid long-term investment of "increased personal knowledge".

The Gossamer Albatross succeeded where others had failed when they realized that the actual problem was fixing the process first [1]. They won big by solving the more general problem. What if we fixed the software development process so that everything we built could be useful?

[1] http://www.azarask.in/blog/post/the-wrong-problem/

Sure, some code is just bad, and the trash can is a useful tool, but a DV HAL sounds like a useful tool, too, and I don't think anyone is better off that it got built and then discarded. Could there have been a secondary market for commercial libraries whose products were cancelled? Could it have been open-sourced? Could it have been built as part of an alliance, so other teams could have helped drive the design, and reap the benefit even if one team dropped out? I don't have The Answer but it seems awfully unlikely to me that optimizing one's architecture for the next day or week is an optimal process on any larger timescale. That's basically admitting that we're still at the hunter-gatherer stage of software development. You can't plan for next year's harvest, if you haven't invented agriculture, and don't know where you're going to be in 6 months.

Bonus: "At the time templates tended to crash the compiler, so going fully templated was really expensive." Yes, we need to reduce our scope, because this other program we're using reduced theirs! The limit of this function is a catastrophe. If I were to get a batch of bad steel from my supplier, I wouldn't try to compensate by using twice as many bolts.

gilbetron• 8 years ago

Great abstractions exist, that doesn't mean every abstraction is great. Most are bad.

alkonaut• 8 years ago

I'd formulate the "don't overgeneralize too early" as "make concious decisions about where you solve specific and general problems". It's fine to solve a specific problem knowingly, and while considering the cost of solving a general problem.

Saying "let's cross that bridge when we come to it" is the idea (rather than "whoa we never saw that bridge we should/shouldn't have crossed")

There is also a human perspective to this. Solving general problems is FUN, (at least I tend to think so). The generalization is the fun part of the problem solving. Some people think seeing the working product is cool, no matter how elegant or general. I don't mind if everything is half-finished forever so long as the half-finished product is elegant and general (exaggerating but you get my point about the different personalities). This is also why you need different perspectives (and probably people) on a team.

j7ake• 8 years ago

Richard Hamming or John Tukey said "It is better to have an approximate solution to the correct problem than the perfect solution to a slightly wrong problem."

It might apply to this case here.

childintime• 8 years ago

Jonathan Blow discusses harmful abstractions in several youtube videos, while discussing the Jai Programming language. That language isn't out yet, but an overview is available at: https://github.com/BSVino/JaiPrimer/blob/master/JaiPrimer.md

chaoticmass• 8 years ago

I see the author's point, and it is valid, but I'd like to give a counter example.

I was asked if I could make a program that could 'put a photo on top of another one'-- the idea being they wanted a program where you could load a template image, maybe one that looks like a birthday card, and it would have a spot where someone's picture could be put on top of the birthday card image. Simple enough, but I decided to build a generalized templating system with a built in editor so new templates could be created. This way it could make birthday cards for one person, or make a "Good Job team!" cards for multiple people. It was completely agnostic to what kind of templates you could make with it. It supported layers, conditions, database integration.

Sure enough, over the course of months, I was asked to make the program do more things than it was originally asked to do, and I was able, with few exceptions, to accommodate these requests without needing to modify the source code. Even when I did need to modify source code, it was to extend functionality, not change the basic template abstraction model. It found its way into other uses as well (digital signage, ads, etc).

If I had only solved the least general problem first time, I would have been back at the drawing board rewriting most of the app every time something new was requested.

Maybe this is a rare counter example, but sometimes pursuing the general solution pays off.

commandlinefan• 8 years ago

I actually agree with both of you, _as long as it's up to the implementer_. I see requirements specifications that deliberately try to remain vague in the hopes that the developer will produce a "general" abstraction that can be reused - and always end up shooting themselves in the foot by never saying exactly what they want done. Or the "disruptive scrappy startup founders" who want you to build "legos".

tynpeddler• 8 years ago

I think another way of putting the title is "Right-size your abstractions." My rule of thumb is to solve the problem at hand while accommodating (but not implementing) known tasks within the next 3-6 months. If you're not designing your abstractions to deal with changes you have very good reason to believe will be coming soon, you'll be stuck rewriting all your code over and over again. At the same time, it's very difficult to anticipate requirements more than 6 months out unless you're very experienced with the problem domain and application you're working on.

jononor• 8 years ago

But you did not go and try to implement PhotoShop or a node-editor graphics processing tool, which would have been even more general - and likely too far.

ilovecaching• 8 years ago

I highly recommend following a data oriented approach to software organization. OO loves to abstract data, but why? Data is the main character of the story. I also hate the take on Knuth's "premature optimization". Don't sweat the small stuff, but do architect for performance, it is a feature. If you focus on performance and not hiding your data, the code will fall into the right level of abstraction.

Solving general problems happens when you start thinking about data you don't have, but might, and throw performance away for premature organization.

dahart• 8 years ago

Definitely agree. In ~25 years of professional coding, I've witnessed more money wasted by over-engineering, over-generalizing, and solving unnecessarily hard problems, than I have of not being general enough.

I've seen that often enough that I'm personally now starting to make the mistake of not being general enough regularly. :) But it seems like a good thing if I'm 50/50 between not general enough and too general.

Running a startup was good practice, you just can't afford to work on problems you don't really have. If multiple customers aren't screaming for it right now, it can probably wait.

Ironically, I think I learned this lesson more thoroughly with writing than with computer science. Using abstract language, using categorical and abstract words rather than specific examples, makes for super boring reading. Communicating a pattern is even sometimes even more accurate when you list three specific examples than when you choose the most accurate abstract word; the reader will see the pattern.

Just googling for examples, this one seems decent. Lots of writing advice online leans in favor of being specific. https://www.unl.edu/gradstudies/current/news/use-definite-sp...

"In proportion as the manners, customs, and amusements of a nation are cruel and barbarous, the regulations of its penal code will be severe."

vs "In proportion as men delight in battles, bullfights, and combats of gladiators, will they punish by hanging, burning, and the rack."

alexeiz• 8 years ago

I've seen the other side of the coin, where the desire to solve a concrete problem forces a solution to be unnecessarily brittle and actually harder to implement than a more generic solution.

I'll give an example: "we're never gonna have more than three devices, so this array should have a hardcoded size of 3." It sounds pretty extreme, but believe me, some variation of this theme comes up more often than I would like, especially from people who are not software engineers. It seems like some people think that every abstraction (and potential genericity) is expensive, so they tend to put constraints in places where none need to exist. Very soon you'd need to make a small modification to your code, or extend it slightly, and those unnecessary constraints are going to make things difficult.

Personally, I'd like to approach a problem from the bottom-up. Start with solving a concrete case, and see if any patterns emerge in the process. And if they do, then I generalize those patterns. Frequent refactoring is very helpful during this process. Sure, mistakes and over-generalizations can still occur. But they are usually not very expensive if noticed quickly.

muxator• 8 years ago

In this specific case I would say that the abstraction rises as soon as the number of devices gets >= 2.

Isn't the "0, 1, n" rule a thing?

moring• 8 years ago

I think this conflates requirements and implementation choices. "we're never gonna have more than three devices" is a statement about requirements, and implies the absence of a requirement to support more than 3 devices. It does not imply a requirement to fail to support more than 3 devices though, and leaves an implementation choice how many devices to support, as long as it's at least 3.

On the implementation side, the "0, 1, n" rules already mentioned by muxator dictates that this specific aspect of the implementation should not treat 3 different from 5, and implement support for n devices. (By "This specific aspect" I mean that other factors will surely prevent supporting n = 1 trillion, but we don't have to care about that since it doesn't contradict the requirements).

beaconstudios• 8 years ago

this is why I think one-way specifications are a massive anti-pattern in the software industry. Often times the developer, who has the front-line view of the existing abstractions, could turn a 4 week project into a 1 week one just by adjusting the spec, while still solving the actual problem.

swsieber• 8 years ago

Hmm... it's not quite addressed here (they address generality as the size of the scope of the solution), but I have a slightly different take:

I love generalizing things. If I need to remove a couple of prefixes from a list of strings, I don't hard code those strings in a function. I make a general string prefix removal and pass those prefixes in. I find that I write more correct and stable code by switching from special cased code to generalized code.

One can take this to extremes of course. I think the article is against generalizing solutions that shoehorn things together that normally don't go together. E.g. hal for handling disparate hardware abstractions. Its a forced abstraction to gain certain things.

I guess it's abstract vs generalize. Generalize is good, if it doesn't introduce abstractions... that's the thought process I just had anyway.

adrianratnapala2• 8 years ago

Suppose I already had two function functions like:

   def strip_prefix(prefix, str):
     ...


   def name_from_fooid(fooid):
      return strip_prefix("foo", id)

Now is it better to call `name_from_fooid` or to call `strip_prefix` directly? It depends on the high-level semantics we are trying to express. But assuming we really are extracting a Name from a FooId, then the former is probably better.

But then if you want a name_from_fooid function, it's just an implementation detail that you happen to implement it on top of a strip_prefix utility.

hyperbole• 8 years ago

that's a whole lot of blathering to say pre-optimization is the root of all evil... @example: the Intel meltdown &Spectre bug.

pankajdoharey• 8 years ago

The true meaning of a General solution does not mean providing support for every kind of scenario your data may encounter, the true meaning from a functional perspective is to build composable structures/mechanisms to produce the correct shape of data the problem requires. So for a near sighted programmer YAGNI would work, but for a software or machine to be really durable, it needs to be composable over a period of time. An example of this is emacs, it has kept with the passage of time because of its composable nature. Vim on the other hand is undergoing a massive rewrite project known as Neovim. This is the difference between a true general solution and a specific YAGNI based solution. The only differentiator between the two is Durability, and this durability comes at no extra cost, just the wise decision of writing software in a much more composable language solves half the durability problems.

iamleppert• 8 years ago

A good abstraction leverages the shared knowledge of other previously unconnected pieces of information. It has more to do honestly with psychology than computer science; I don’t need to learn anything about tree or graph data structures when I can use HTML in the same way as I am used to organizing information in bulleteted lists.

gcb0• 8 years ago

playing devil's advocate: if the author's manager had communicated the tech improvements correctcly ("we can now support any hardware"), maybe others teams knowing about that would have moved the company in a different direction instead of "cancelling everything after firewire support". maybe they even got to that decision because one new input source took so much time in the first place.

Solve Less General Problems

Related Stories