Modern Software Over-Engineering Mistakes

https://medium.com/@rdsubhas/10-modern-software-engineering-mistakes-bc67fbef4fc8#.swzmypry1

by grey-area • 9 years ago

198 39 9 years ago

medium.com

aschampion• 9 years ago

See one of many previous discussions here:

https://news.ycombinator.com/item?id=12718270

JamesBarney2• 9 years ago

I really think most of this comes down to the number one principle of good software development is "Keep it simple stupid"

Simple, concrete, un-abstracted code is easy to get to right, easy to read, and easy to change.

olavk2• 9 years ago

Simple is good, but it is not necessary simple to write simple code.

oebs• 9 years ago

There's also "Don't repeat yourself" and often you cannot have both DRY and KISS and have to choose one over the other.

donut• 9 years ago

I think of the combination as DRY KISS - do repeat yourself to keep it simple, stupid!

JamesBarney2• 9 years ago

If it's domain logic, like calculating the total for a shopping cart then you gotta follow DRY. The risk of application inconsistencies are too high. But if it's anything else then I go with KISS.

I really think DRY is far more important for business logic than it is for infrastructure logic. If you calculate what taxes someone owes in two different ways that's terrible. If you serialize two different things in your application in two different ways, probably doesn't matter.

geoelectric• 9 years ago

The thing about DRY is it's not just "don't repeat this code," it's "don't repeat this code that has the same execution assumptions and maintenance expectations." You can make those assumptions broader with parameterization, injection, etc., but there's a limit before you either make it too complex to reason through or end up pulling in diverse maintenance expectations.

In my own niche, software test automation, this can be a huge problem because the meat of every test stands alone from all other tests, otherwise it wouldn't be a unique and interesting test. Applying DRY naively in that kind of situation often leads to a bunch of non-cohesive "do everything but this, and based on this flag do this one different thing" type code.

You really have to jump back a level and look at things more in context of their usage. This is one of the reasons that there's debate about even sharing things like setup methods, never mind test bodies.

Sounds like the evolution of different flows described has a lot in common with this. You have to look past the lexical.

solipsism• 9 years ago

In my many years I've never seen an architecture like what the author describes as Sandwich layers in 2016 with SOLID. One layer injecting implementations into deeper layers, many layers deep? When does this supposedly happen?

In fact, dependency injection and a layered software architecture are mostly orthogonal concepts.

austincheney• 9 years ago

Reading that list one thing kept popping into my head.... design patterns.

olavk2• 9 years ago

Design patterns are tools. Don't use a hammer if you don't need a hammer, you will just destroy something. And vice versa: Don't eschew the use of a hammer when you actually need it, just because you once saw some guy use a hammer in an inappropriate context.

mceachen• 9 years ago

Certainly. But I think people read the TL;DR: as "oh, design patterns are bad" or "don't use the wrong tool for a job." A better TL;DR is a bit more nuanced:

Broad familiarity with your tools and their applicability (design patterns or otherwise), is a prerequisite to picking a reasonable tool for a given task.

segmondy• 9 years ago

If you think there is something wrong with design patterns, then you still have a lot to learn. Improper use of design patterns is a thing and this I can agree with, sadly, from what I've seen, design patterns are rarely used. People tend to stumble unto them and implement them poorly at best.

kedean• 9 years ago

Honestly, I think overuse has become it's own problem.

Design patterns are not a suggestion, you shouldn't strive to use them. Devs shouldn't start a project and think 'what design patterns will I use here'. They exist so that you can identify what you are already doing and avoid past mistakes that others have already made (hell, the GoF book states this in the intro). Nobody should go into a project thinking 'I will use the singleton pattern for this', but if halfway in you realize that your system fits that pattern, it's there for you.

65827• 9 years ago

Trying to make everything generic enough so that will work everywhere everytime is one of my pet peeves. You really don't need to engineer a date picker to work in microgravity and on quantum CPUs and in alternate dimensions. I get that it's fun sometimes, but my god stop.

otaviokz2• 9 years ago

Nothing can express my sympathies better than an all capitals "INDEED". Have a great week.

kyriakos• 9 years ago

Agreed 100%

HalfwayToDice• 9 years ago

Regarding database abstraction, the author writes: "In 10 years, I’ve seen only one business make serious effort to completely swap a fully vested database. And when it happened, the “Magic file” did not help."

Agreed. I almost embarked on a huge PHP mysqli>PDO conversion, and was advised the opportunity costs were huge and the benefits minimal. Glad I took that advice.

silviogutierrez• 9 years ago

Conversely, at one point I wanted to start using Postgres' JSONB features, but was running my production app on a MySQL backend.

Switching was as simple as:

1. Export all data to fixtures. 2. Change settings.py to use the Postgres drive. 3. Load all data into fixtures.

Granted, I was sticking to core Django ORM calls, so I wasn't hard coding any SQL anywhere. And the null ordering does change a little, but the tests caught that.

So I was definitely glad that worked out. And have been super happy with postgres.

bayonetz• 9 years ago

sticking to core Django ORM calls

I think that's what made it easy. That's the trade off when you try to build a generic DB access to support the trivial DBMS swap like the article take about -- you end up writing a lowest common denominator of functionality common to all your anticipated DBMSs. I assume Django's ORM let's one do DBMS specific stuff but you happened to not make use of that.

I agree with the article, 9/10, don't try writing wrappers just to support the "easy future swap out". Write them if they make the specific technology more palatable to work with. An example I'm my life: Rails' ActiveRecord is just a joy to work with but we would be hurting plenty if we tried to swap out to a different DBMS.

mianos• 9 years ago

Swapped Sybase to Oracle and others many times in the olden days. Clients bought our in memory trading system but mandated the database. More recently have gone from Mysql to Postgres with Sqlalchemy. Quite easy even with substantial existing code (but not trivial).

timClicks• 9 years ago

I think points 1 (management vs dev split) & 10 (poor time estimation) are symptoms of another phenomenon: developers are--on the whole--poor communicators. Creating software products requires communicating with other humans. And because those humans have a different view of the product and its users, they have a different perspective of priorities.

eternalban4• 9 years ago

Contrary view:

Abstraction is possibly an art and few in the IT industry are good at it. Poor attempts at abstraction underly the common anectodal experience of 'failure' to consolidate software in context of business.

jjawssd• 9 years ago

Maybe in some cases the high cost of hiring artists capable of abstracting code to several platforms effectively is higher than the cost of writing and maintaining one codebase per platform. See iOS vs Android.

otaviokz2• 9 years ago

I'm not sure your example applies perfectly to the question at hand.

But as a native mobile dev I feel a deep affection for it :P

eternalban4• 9 years ago

At the end of the day software is there to support the business, so if that is the more cost effective approach, why not? I've even argued for disposable software to a few clients.

However, I bristle a bit at a blog that misdiagnoses the problem as "over engineering". The problem is bad engineering, not over engineering.

ChuckMcM• 9 years ago

I think you're not far off, and using art as a metaphor, you don't need to paint your house in shapes and colors that follow the golden rule, or write your to-do list in iambic pentameter. You can of course, and it might give you tremendous personal satisfaction, but it doesn't help the output be better.

eternalban4• 9 years ago

I don't consider good abstraction a mere matter of aesthetics. But the economic realities of this field support your analogy regardless of that clarification.

onion2k• 9 years ago

Abstraction is possibly an art and few in the IT industry are good at it.

Sounds like a good reason to avoid it.

Chris_Newton3• 9 years ago

In my experience, the abstractions that are helpful are the ones that significantly reduce complexity in the code using them by hiding details that we rarely need to see. Put another way, they separate interface from implementation and the resulting interface is significantly simpler. That lets us reason about our code at a higher level without getting bogged down in unnecessary details.

Most of the problematic abstractions I see in the real world don’t hide much complexity but still introduce extra complexity of their own (with the latter even becoming greater than the former in the worst cases) or often leak so we still have to drill down through the abstraction to understand what’s happening anyway. Arguably, these are just two sides of the same fundamental flaw.

plaguuuuuu• 9 years ago

A lot of the most annoying and time-consuming abstractions at my company are, ironically, geared towards maintainability/extendability concerns.

We have wrappers around generic web service callers that rely on factories that create classes that convert between web service objects and domain objects, all wired together using interfaces for each service class and an IoC container, with its own XML configuration file.

The irony is that if the web service were to change its interface, you'd have to change like 10 files instead of like.. a single conversion function somewhere. And to begin with you had to create those 10 files in the first place. It's absolute fucking madness.

eternalban4• 9 years ago

the abstractions that are helpful are the ones that significantly reduce complexity

Accepted! (Did I mention anything about complexity?)

ensiferum• 9 years ago

Hasn't this been posted before? Anyway, when you're solving problems that nobody is having you're overengineering it.

vonseel• 9 years ago

Like with all things, there's a healthy balance between simplicity and engineering for scale. Software Engineering != "Software Over-Engineering".

cpeterso• 9 years ago

Netflix's website is a good example of business processes driving engineering. Their streaming and DVD rental functionality are totally siloed. They have different movie queues, UI designs, and subdomain names. The user experience could be so much better if the UI was more seamless. I assume this is fallout from the streaming and DVD businesses being run by different organizations. It's as if Qwikster still lives, but is just called dvd.netflix.com.

jwatte• 9 years ago

You will never suffer from that if you install nom so you can install gulp so you can install ivy so you can get my AbstractModernSoftwareOverEngineeringMistakePreventionStrategyHelperFactorySingletonTemplate!

teknologist• 9 years ago

OK. Why would you install gulp to get ivy?

awinter-py• 9 years ago

This is a really good point. I think it gets to the heart of why libraries are more powerful than language features (though missing language features can hold back libraries).

It takes a lot of skills and deep understanding of the problem domain. A “Service runner” library needs expertise of how daemons work, process management, I/O redirection, PID files and so on. A CMS is not just about rendering fields with a datatype — it has inter-field dependencies, validations, wizards, generic renderers and so on. Even a simple “retry” library is not so simple.

Modern Software Over-Engineering Mistakes

Related Stories