It's Not Sabotage, They're Drowning

https://shermanonsoftware.com/2019/11/15/its-not-sabotage-theyre-drowning/

144 101 7 years ago

koz_• 7 years ago

This allergy to reality is surely the biggest trap not just for startups but people in general. It's surprising but I feel like the norm is for people to care far more about feeling that everything's okay, rather than everything actually being okay.

That said, one problem I see with the anecdotes provided is that the problem solver is doing two things: discovering a problem and solving it. Conflating these two things is a great way to encourage pushback: every change comes with an associated risk, so the general rule is to say no (using whatever flimsy reason) if it's not well understood and agreed that the problem being addressed is a real problem worth solving.

conradp• 7 years ago

... the problem solver is doing two things: discovering a problem and solving it. Conflating these two things is a great way to encourage pushback

This squares with my experience strongly. At a prior job, I remember two or three times where a set of changes was almost rejected for production because despite solving real bugs. The bugs hadn't affected anything in production yet, but the fixes carried the risk of maybe breaking something.

This resulted in some fun times, like a particular client running something like three major versions behind on our software, precisely because management didn't want to accept any disruption risks there, and the old version grumbled along well enough.

Ensorceled• 7 years ago

Reminds me of this scene from Frank Herbert's Dune:

“Because of an observation made by my father at the time. He said the drowning man who climbs on your shoulders to save himself is understandable– except when you see it happen in the drawing room.” Paul hesitated just long enough for the banker to see the point coming, then: “And, I should add, except when you see it at the dinner table.”

ltbarcly3• 7 years ago

I think saying 'they're drowning' is still giving them too much credit. People who built terrible broken systems don't know they are bad at building things, and they probably don't have a clear idea of what would be better or worse to begin with. They don't know it's better to have a system that can be debugged, they don't even know what the difference is. If you give them examples of good designs and bad designs, they can't tell you which is which. When they say 'this is good' or 'this is bad' they will astound you with the reasoning.

Software is an industry with no standards for employment. All it takes to be a Software Engineer is for an incompetent manager to hire you. More than half the people I interview barely have a basic familiarity with writing software. These are people who have been employed as software engineers for years and will certainly find another job. They struggle to write a nested for loop. They write functions with thousands of lines. They just don't know what they are doing.

The author here claims they don't want the visibility of the situation to increase, but I think they don't know it's a problem to begin with. Their stupid excuses for not moving forward are just their stupid thoughts said out loud.

vanniv• 7 years ago

My team has built garbage software at times. It isn't because we don't know how to build good software, and it isn't because we don't care, and it isn't because we don't want to build good software.

It is usually because of a continuous sequence of emergencies to which we are forced to react.

An emergency occurs, and we have requirements to do something in 1/3 the necessary time. We get 80% of the way though that and, before launch, there's a new emergency, and we have to rush to that.

So we end up with crap software that looks like shit and only almost works.

But it almost works enough to keep our very large business humming along. Sure, we hate it, and we're making more technical debt, and over time, we get less effective.

But sometimes we go through cycles of a year or two when the business either can't afford to do it right or isn't willing to pay the price to do it right.

Sometimes, they really are willing to accept paying 10x as much so long as they can pay in a year or two.

(I hate it, but it is true)

AstralStorm3• 7 years ago

Make it 100x as much with 5 times as many people in 10 times as long. Efficiency is never important when there is money to burn and it's not yours, and you won't be blamed when it breaks.

It does not even take any emergency - often it is conscious or subconscious project management strategy.

It's related to observations about Peter principle, Dilbert principle, Gervais principle and office politics and dynamics.

Avoiding this tendency is essentially only possible with small teams with full ownership of the problem they're working on and direct stakes.

JohnBooty8• 7 years ago

    People who built terrible broken systems don't know they
    are bad at building things, and they probably don't have a
    clear idea of what would be better or worse to begin with.

Sometimes this is true, sometimes not.

The early days of a company (or product) can be full of existential threats and rapidly changing requirements. Resources are always limited and dev teams are always stretched beyond their limits. The crappy code you see may be there because it was a literal choice between that duct-tape solution you're looking at, and missing out on some major deal or perhaps even going out of business.

Now, yes, the people who built such a rickety system can become obstacles to improvement. But, they are not necessarily incompetent. They may have been forced into making the least-bad decision out of several terrible options, over and over again.

Sometimes they're just pretty bad developers though.

lowercased• 7 years ago

I refactored the code from untested and untestable, to testable with 40% test coverage. The senior architect is refusing to merge because the test coverage is to low.

what's likely even more frustrating is that other merges went in for years with no tests or testing in place at all.

umvi2• 7 years ago

Seems like unit tests are only really valuable for library code. Application/glue code... not so much, especially if it is something being actively developed and you are constantly throwing away tons of unit tests.

AstralStorm3• 7 years ago

Totally untrue. Unit tests in strict sense, perhaps, but nobody writes true unit tests - there's a high chance these are actually short range functional or integration tests. Such tests are of high value.

larzang• 7 years ago

The weakest link in your software is always you. Automation that can help compensate for human failings always has value. I can't count the number of times writing tests has led me to catch simple mistakes in code that I'd stared at for hours and declared good.

Not to even get into the ways that tests can help guide sensible architecture, especially for junior devs, because "this is hard to test" is the single most straightforward code smell for "this probably needs to be simplified or broken up".

wheelerwj2• 7 years ago

thats the point if that example...

untested and untestable

mirimir• 7 years ago

When I was trained in lifesaving, one strategy for panicked victims, who might drown the rescuer, was just to give them some space to drown enough to lose consciousness. Then they were much easier to rescue.

wheelerwj2• 7 years ago

it doesn’t usually apply to drowning victims, but more broadly its applied when someone is choking in general.

if someone is refusing help, you wait until they pass out, and it becomes implied consent. I assume in this example, that would be firing or otherwise being told by your supervisor to take the help.

kevingadd• 7 years ago

Consent isn't necessarily the issue in drowning individuals, since the instinctive drowning response can make it dangerous to save them even if they're not about to tell you 'let me drown'. Maybe actually a good comparison for software teams that refuse to make incremental improvements out of fear or panic... it's not that they don't want things to be better or refusing help, necessarily - they're just terrified of the consequences if the improvements go wrong.

Aeolun5• 7 years ago

So being the person that’s manning the lifeboat, what do you do? Kick them down?

If you do nothing everyone will surely drown.

stickfigure• 7 years ago

I can perhaps understand where this one is coming from:

I refactored the code from untested and untestable, to testable with 40% test coverage. The senior architect is refusing to merge because the test coverage is to low.

You have a piece of software running in production; even if it's poorly written and bereft of tests, your customers are satisfied. Customers (unwittingly) have already vetted this version! A rewrite, even with brilliant design and fantastic test coverage, is still more of a risk than fixing what isn't broken.

This isn't to say that you shouldn't rewrite code, especially if it makes other improvements possible. But don't discount how high a bar new code has to reach to measure up with "it already works". Maybe 40% is still too risky.

AstralStorm3• 7 years ago

It's always a lie, about running in production. I've personally seen untested code allegedly running but it isn't actually working. Defect numbers through the roof and test harness literally not being merged exactly because of this issue.

You will get visibility, either by customer complaints, slowing down development speed and ultimately projects being shut down. It's not sabotage and it's not exactly drowning. It's willful attempt to ignore reality and cheat on metrics, prolonging the inevitable and defrauding stakeholders.

BigJono• 7 years ago

It's always a lie? You're basically arguing that code with bad test coverage never works.

Production code is copping constant manual end-to-end tests. Those tests are more valuable than most unit testing, and some automated black box testing. I'll take stable production system over the unit tested new code any day of the week.

tarsinge• 7 years ago

And then you have tested code with useless unit tests that cheat on metrics and defects and customer complaints very high too, because nobody is listening to customers either way. I regularly run untested code (when appropriate, it’s a trade off for simpler design with less abstractions sometimes) with happy customers.

arkh3• 7 years ago

You don't want to test code. Coverage is mostly useless.

You want to test functionalities: setup end-to-end tests. It will be hard, it will take time and it often is a solo endeavor for the first months. But when tested functionalities stop breaking for no reasons even when you refactor a lot of code, when "the project for which a dev put tests" starts being "the project we are confident to push to prod without breaking things" you should get some traction.

And once you think all your functionalities are tested you can measure code coverage: what is not covered should be dead code you can remove.

sverhagen• 7 years ago

Pointing people only at end-to-end testing seems poor advice. There is value in a mix of all sorts of testing. I agree that testing functionality blows counting coverage by the line out of the water. But since a lot of code that implements algorithms, string manipulation, etc. can be tested just fine with unit tests, and since end-to-end tests are expensive, easily more brittle, require a lot more dependencies, it would be better to have a healthy mix of different types of tests.

adrianN• 7 years ago

Coverage is very useful in determining whether your tests actually do what you think they're doing. It's not very useful as a metric to be optimized. Writing good end-to-end tests that really cover the functionality as specified is very difficult, doing so blindly without tool support is doomed to fail.

An even better tool than measuring coverage is setting up a mutation testing framework and adding tests until all mutations are caught.

gmueckl• 7 years ago

Tests need to cover small units one at a time. I've been bitten too many times by tests that were too complex and attempted to test too many things at once. They invariably ended up doing nothing of value. Test setup becomes overly complex and prone to oversights. Writing another test for a variation of a problem that takes a slightly different code path becomes a chore. It also gets hard to spot that odd untested corner case hidden deep inside some complex logic.

The proper way to think about this is to have unit tests that cover the nooks and crannies of the business logic and integration tests that make sure that interfaces are correct.

JohnBooty8• 7 years ago

I can't get behind this.

End to end tests are indeed important.

But let's say you have an end to end test that covers A, B, C, D, E, and F.

One day this test stops working. Well, what's the culprit? A? B? C? D? E? F? If you had more granular tests in addition to the end-to-end tests, you'd know.

Now, I agree there's a possible downside here. Maybe there comes a time when C and D are no longer being used. It might not be obvious and C and D could linger in the code base, with corresponding dead weight in the test suite. This is not ideal. However, I would rather have this than a tough-to-debug test failure.

DrScump3• 7 years ago

  a rewrite... is still more of a risk than fixing what isn't broken.

Defining "broken" is the problem. A long-extant program may be "broken" even while it meets customer expectations, and "fixing" it could be viewed as a regression by the customer.

I had a case early in my career where I was finishing up a rewrite of a report writer program. The old version didn't really have test data, so "testing" mainly meant comparing outputs of the legacy system vs. the rewrite. After I had sussed out all the errors, there were still some differences in which the legacy output just didn't make sense. I met with the customer liaison, and he checked with his users.

It turned out that nobody had bothered even to glance at those specific reports in as long as anyone could remember.

mcherm• 7 years ago

So... did you correct the errors and release your updated (useless) report? Or did you slog through 7 levels of management approval to get permission to get rid of it?

hinkley5• 7 years ago

Just because it works doesn’t mean it isn’t broken.

cjfd• 7 years ago

It can be defended when the system is in a rather passive form of maintenance. E.g., every month a small bugfix. If it is more than that it becomes way to easy to fix one small thing while breaking three. Especially if the functionality is a little or a lot complex.

What it does sound like, though, is that everything was refactored/covered in tests in one fell swoop. I find that questionable. I think it is much better to start out small in an untested code base. Just cover the small piece you are working on now in tests and gradually extend.

Aeolun5• 7 years ago

Some code is so mangled together that trying to start small is more of an investment than overhauling the whole thing altogether.

raverbashing• 7 years ago

40% code coverage is meaningless

It could be 40% weak unit tests that just go through the code but doesn't test much

I would be happier if it were 40% coverage running integration tests with actual data and use cases

JohnBooty8• 7 years ago

My thoughts exactly. I'm not merging a bunch of untested code that replaces a bunch of working code, no matter how bad the old code was!

That seems like a recipe for a production nightmare.

The fact that this situation even arose in the first place would be bad news.

1. Did the other dev just refactor the code on a Red Bull bender one night on his own time? Hopefully this refactor work is something we would have discussed ahead of time. Hopefully I wouldn't have to even tell them that the expected coverage for a big refactor like this would be darn close to 100%.

2. I'd phrase this rejection in a positive way. I'd be excited about the refactor and testability and I'd let the other dev know that I appreciated the work and was looking forward to getting it to 100%, testing and merging it

(I realize the example in the story was purely hypothetical and this didn't actually happen. :P)

hinkley5• 7 years ago

We may have diluted the meaning of refactor if we are arguing about pull requests.

Refactoring is an activity of small steps of a particular kind that are generally sound transformations. The biggest risk is transcription errors.

Of course one of the downsides of PRs is that the they muddle a sequence of individual commits and only show the cumulative result.

When dealing with individuals with these sorts of fears, you have to be patient and strategic. Pick the most powerful refactor out of a set of many useful ones and do them one at a time until you achieve the result. Essentially you have to boil the frog.

thfuran2• 7 years ago

Refactoring is an activity of small steps of a particular kind that are generally sound transformations. The biggest risk is transcription errors.

I think that is far from an established sole meaning of the term.

slowhand09• 7 years ago

That security hole that is leaking customer data hasn't been noticed for the 9 months since it was implemented. Except by that one guy who has captured it all. Customers have only vetted what they see.

CoolGuySteve2• 7 years ago

Nobody’s better at letting better be the enemy of good than software engineers. For some reason our culture likes to argue and nitpick over the most mundane stuff, usually missing the forest for the trees.

I’ve had jobs where the manpower spent just arguing outweighed the cost of the change significantly. One dickhead manager I had spent weeks arguing over the acquisition of a $500 SSD. It made me want to slap $500 down on the table just so he’d shut the fuck up already.

The only way I know to counteract it is by saying “don’t let better be the enemy of good”.

brain5ide• 7 years ago

A few points:

1. Computing and engineering in general is such a discipine where one nitpicky detail can make all the difference. One bit flip, one keystone, one badly selected material. However, the conduct of negotiation, could be improved as always, but it's a general listening problem most of the time.

2. From the dickheads point or view, everything can be solved by spending more money on HW. Each unexpected spend like this is a risk to open the floodgates and lose both financial and engineering discipline by setting a bad precedent. Not sure you want easy handed decisions there as well.

biggestdecision• 7 years ago

It makes no sense to cheap out on hardware though. It's such a tiny cost compared to the overall cost of the employee, you are wasting money if inadequate hardware is making your employees less productive.

ivalm• 7 years ago

Spening a few hours of time on the topic is literally more valuable that the ssd itself. So what if you spend a few k per employee more. It is basically a rounding error in overall comp. This is also why I dont understand companies trying to save money on perks...

hinkley5• 7 years ago

We once spent an entire summer trying to add functionality to an app where some of the customers were already running at max capacity on their self-hosted hardware.

I was trying to explain to my boss how stupid it was for us to spend $100k to avoid a handful of customers having to buy $3k in hardware apiece. It would have been cheaper for us to gift them a box. The opportunity costs of that boondoggle ended up being a huge negative for that team.

JohnBooty8• 7 years ago

I had a manager argue over the cost of a training class once. They'd moved me to a new team with unfamiliar tech and I wanted to bone up on some concepts before getting started.

It was an $11 online class on Udemy.

(Well, technically it was a "$199" class, conveniently discounted to $11. You know how Udemy rolls.)

Generally, I'd say nobody's that petty. A manager would typically only be that petty if trying to get somebody to leave. However... this guy was not very good at his job.

I don't think he was literally upset over $11, and clearly our relationship was on the rocks at the point, but I'm not sure what his exact problem was. May have been me on a personal level. May have been the fact that I wanted to spend a working day uh, learning to do my job better. Perhaps he didn't feel anything on Udemy could be worthwhile. Perhaps it was just his fragile yet rampant ego. Could have been any of those things or all of those things. Nonetheless, it's hilarious that an $11 class was where he attempted to draw some sort of line.

robodale• 7 years ago

I had to explain to my supervisor's boss' boss (Division VP of a $800M USD/year company) for an hour on why I needed a $45 piece of software to read/understand binary files.

dredmorbius• 7 years ago

A possible way to sell instrumentation / metering is to build your own scaffolding, find a problem using it, and sell both the scaffolding and the solution it's provided.

As the article notes, metering by itself simply illuminates problems. For the drowning team (or individual), that simply delivers bad news.

If you can show that the effort actually delivers results, you've got a way to sell the instrumentation as well.

In more pathological cases, you don't reveal the scaffolding, but start cranking out code fixes based on it. You may have the opportunity to sell the metering later, when you're asked how you're finding and fixing the problems.

Rescuing projets from antipatterns is painful at best. Enlightenment is a painful process. It peels away comforting lies to reveal unpleasant truths.

deboflo• 7 years ago

This is one of those things you don’t truly understand until you’ve experienced it first hand. I have. Very appropriate metaphors.

peter_retief• 7 years ago

Could it be a competence barrier to protect incompetence? I have experienced so many similar experiences, normally with middle managers who are terrified they will get caught out for not knowing what they are doing. I noticed they always stick with brands like Microsoft or SAP are deeply suspicious of open source. Oh and they dont drown, they get promoted. Could this be an opportunity for a startup to identify this risk "as a service"

jacquesm• 7 years ago

The way to establish change is to do it gradually, with buy-in from all parties prior to introducing the change and to make small steps in the beginning. If you showed up with a huge chunk of code that has been refactored I'd probably not merge your change either. The test cases have been written by the same person that did the refactoring, there is a very good chance that any wrong assumptions you made about the code will be present in both.

inertiatic2• 7 years ago

I refactored the code from untested and untestable, to testable with 40% test coverage.

This tells me you think "coverage" is a good metric, which also tells me you're you're not covering a whole lot of cases (edge or not) where the "untested code" functioned as it should.

You're asking to replace something which has potentially some known issues, with something that has a potentially unlimited amount of unknown issues.

And the more experienced person at your job refused? What a tyrant!

poutrathor• 7 years ago

I strongly disagree.

The said refactoring was probably not a huge task since he is reporting it done. Larger tasks are most often planned and monitored.

Splitting some class and decoupling some resources + adding the testing framework and implementing a first batch of tests. Of course it might have introduced few regressions.

But

has potentially some known issues

Read again that. Exactly, none knows how many bugs there are, only how many were found so far. Bug tracking is often lacking, behind schedule, wont fix, etc.

On the contrary modern code analyse tools are frightening good at spotting untested conditions every where (sonarcube comes to my mind). Much better and much reliable than human developer and human tester. Sonarcube does not care if the case can not happen because user interface does not allow the input. The bug is here, and the bug is spotted.

The more experienced person is plain wrong : more tests is better. It's not panacea yet, but it's better.

inertiatic2• 7 years ago

function int untested_sum(int a, int b) { return a + b; }

function int tested_sum(int a, int b) { return a * b; }

function void test_providing_total_coverage() { assertEqual(tested_sum(2, 2), 4, 'Expected sum to be 4!'); }

steve_gh• 7 years ago

So many years ago I was involved in an issue with a BTS (a cellular radio base station) for a telecoms equipment manufacurer. The software was written in C, and had a message parsing architecture between processes, with fixed length queues, which discarded messages when they overloaded.

There were various problems (the subject of a number of war-stories, which I won't repeat now) - but basically anything that wasn't using a properly designed state machine could die horribly when messages started getting discarded.

So we did some work around the fundamental design of the queue handling mechanism. We came up with a software modification which would improve the efficiency of the queues handling, which would stop the queues overloading (which we saw happening occasionally at peak loads on the biggest BTSs).

We could measure various aspects of the queue handling performance, and use queuing theory to prove that the performance would be better at very high loads.

We had a test rig and we could test the relative performance of the new software vs the old software at normal loads.

What we didn't have, was a test rig which could run at the highest loads, so we couldn't test the behaviour of the new algorithms at very high loads.

So the improvements were binned. The existing software was known to fail (badly) at extreme loads. The new software could be tested at reasonable loads, and mathematically be shown to improve matters at very high loads, but because it couldn't be tested, the process said that it must not be deployed!

Aeolun5• 7 years ago

How was the original version deployed? The definition of ‘very high load’ was different at the time.

edwhitesell• 7 years ago

I've seen this with people I've worked with that came from those environments. Usually it's a balance between knowing how the current implementation fails is better than not knowing when the new one will.

pvaldes• 7 years ago

"I think that is not sabotage, but I don't have any real data that support or dismiss my statement" would be a better title.

Your mileage may vary.

marcus_holmes2• 7 years ago

I think it's kinda hilarious that the article talks about a psychological effect that hurts software teams, and HN gets into a full-on debate about software testing strategies ;)

I've seen this stuff before. Developers making obviously-bad decisions is usually a symptom of a dysfunctional work culture and a code base that's both vital and fragile. It's not fun to work in such places, but it is possible to fix them.

hvidgaard• 7 years ago

The main issue is for management to accept that velocity will be significantly reduced and that many issues will pop up along the way and require a substantial QA effort to keep the quality from dipping too low.

It's still the best way forward as a complete rewrite is rarely a good idea.

marcus_holmes2• 7 years ago

yeah, I had to explain to a CEO that we weren't going to be shipping any new features for a few months while we upgraded core systems and eliminated tech debt. It was a tough conversation. But about 6 months later we were able to start developing new features again, and everything was much quicker and smoother. The lead dev had regained his composure, too, and wasn't being actively obstructive to everyone who tried to do anything to "his" code base.

merrill_g• 7 years ago

In my world (marketing, not dev) the saying is "you cannot optimise something you do not measure".

stakhanov4• 7 years ago

Between sabotage and people wanting to sink the lifeboat, I have a third option: These people have a warped perception of what's happening and priorities around what's happening. Warped by their own irrational beliefs which have come about as a result of cognitive dissonance reduction. Initially, they can see how messy & bad things are but since they, themselves, are responsible for things having gotten to that point, they start bending their own beliefs in a way that will make themselves and their past actions look good. In the final phase of this psychological process, some kind of Stockholm syndrome kicks in in relation to the poor standard of engineering that is making their lives miserable: They start to actively seek out poor engineering because it's the only mode of existence that their psyche will allow them to fathom.

Warstory time.

A tech company I worked at was being encumbered by a codebase with poor code quality. Bugs were being introduced all the time, with no chance of preventing it because nothing is testable/tested. Bugs were introduced at a faster rate than they were fixed. The whole system was a timebomb because it was built in a dying ecosystem. For certain kinds of code changes, their effect couldn't even be made visible at development-time at all. You had to release them into the production system and look at metrics to see their effect, hoping that nothing would blow up, so the system was layers and layers of code that was trying to be minimally-invasive with respect to existing code.

Whenever an engineer would be newly hired, they would, within their first week of working there, suggest "Let's take a month to refactor this/that part of the system". When an oldtimer would hear the new guy say "refactor", their warped perception would hear "accomplish nothing" or "threaten our mode of existence". They would give the following speech: "Hey, get it in your head, noob: We don't do that here. This company is not focused on code quality, it's focused on a mission. Every week we come to work and we accomplish a part of a mission. I KNOW the code is probably the worst piece of shit you've ever seen in your 20 years. [This company only hires extremely experienced people]. The engineers who work here are all on board with our mission-driven culture and pride themselves about their skill at dealing with the shitty codebase. Your suggestion tells me that you can't deal with it, and you are unwilling to absorb our mission-focused culture. You will not get a career out of working here, unless you properly absorb that element of our culture".

So, in their mind, they'd taken "shitty codebase" which is a negative, and turned it around and made it into a religion about a "mission-driven" culture which is a positive. [Cognitive dissonance reduction at work, right there]. The reality was, of course, the poor code quality was a huge drag on this company's ability to accomplish a mission of any kind: Because changes that took their competitors a day, took them a week. Changes that their competitors did in a month, would have been a suicide mission for them and they therefore rationalized their way out of even wanting to do them. ...but I wouldn't recommend to the newly hired engineer in the story to mention that particular elephant that was in the room.

Now, here comes the really sick part, where the picture of the drowning man falls apart: From time to time, due to a rare alignment of circumstances, this company might be forced into a situation of starting an implementation of a new element of the system with a clean slate, i.e. completely unencumbered by the current codebase. They would intentionally implement the new codebase to be just as shitty as the legacy code. They would implement it in the dying ecosystem. They would generously insert unreachable code. They would pursue the fastest path towards making it untestable.

Now, under normal circumstances, if I hear someone telling that story, I would suppose there might be a motive for this. For example, I've seen developers in hedge funds intentionally rig a system to explode if they should ever be forced out of the company and be as hard as possible for anyone else to fix, for obvious reasons having to do with their incentivization structure. But that's not what was happening here. They had a culture of openly sharing knowledge of the system, were hiring new people all the time, and working hard to bring them up to speed about the system's working etc, so it wasn't that.

My guess is that it must be the "Stockholm syndrome" element I mentioned: They started to develop a psychological attachment to a poor standard of engineering. It was making their lives miserable. But it was also the only existence they knew and they felt was available to them, so cognitive-dissonance reduction was maneuvering their psyches into actually favoring that mode of existence.

Some might have said things like: "One day, after we've grown enough, we will do a complete rewrite." Or "Any day now, we might be acquired by another company and will leave all of this behind". But it was more in the vein as when a religious persons speaks of a glorious afterlife.

There is a lot of really sick shit that the human psyche is capable of, and sometimes it surfaces in engineering. I think that's my takeaway from all this.

palisade• 7 years ago

I have so many war stories there's not enough space on HN to contain them. I'll just state my overall experience. I write elegant well designed code, but generally speaking you won't find that anywhere especially if you're working in teams. Unless the code is all written by one person who had a chance to design and implement it well solely by themselves with adequate time, it's going to be a mishmash of they just didn't have time and the pressure was on.

I'm of the mindset that if it works, it's good code regardless if it isn't. This I think is the healthiest mindset to have in any team. I will refactor things slowly to clean up issues and of course my own code is well thought out and integrated into the codebase regardless how reckless anyone else is being. Any other way of interacting with a team would be antisocial and not beneficial ultimately. The way I see it is they wouldn't be paying me if there wasn't more work to do. The most important thing is that it works and the customer is happy. Your disappointment with elegance or coverage comes second to that.

The thing that actually annoys me is when, having exceeded my own delivery deadlines and everyone else's if I provide value added like a new feature that puts us far ahead of our competition I get pushback. Management doesn't want to have to maintain it. Even though my code is really solid compared to the rest of the garbage going in there. They just see me as a plumber fixing a leaky pipe and don't want anything further. I get that there's ROI and whatnot but it still bothers me when I show them some next level shit and I'm told to mothball it. Especially when I'm the reason we're usually way ahead of schedule. </rant>

stakhanov4• 7 years ago

I've had that same thing happen to me numerous times too. I wrote code that demonstrably and undisputedly added to the value of the product in a way that went far beyond anybody's expectation. I took no longer to do it than it would have taken to do it the "normal" way. People get mad at me making the argument "Well if you leave the company, somebody else is going to have to understand your next-level shit, and we just simply don't have people here who can do that. So then what?"

stakhanov4• 7 years ago

On a separate note: I don't think that what I described above should be referred to as a point on "elegance". Because "elegance" suggests that it's about aesthetics. And regarding aesthetic considerations it is usually the case that people can reasonably disagree about them, and that there is no objective argument that can be made either way. But the arguments I made above about how a bad codebase quality can wreak serious havoc with a company's/team's ability to execute on their mission is something that goes far beyond a failure in elegance. It's a liability. And when I say "liability" then I don't mean in any abstract sense, but in the sense as any financial accountant would understand the term.

taneq• 7 years ago

When you add visibility to a system, the numbers are always bad. That’s why you’re putting in the effort to add visibility to an existing system.

This is something everyone needs to internalize. Don't be upset if the numbers are bad when you first instrument. That's WHY you instrumented. Instrumentation is a ratchet to help you improve your product.

It's Not Sabotage, They're Drowning

Related Stories