The Wetware Crisis: The Dead Sea Effect (2008)

https://brucefwebster.com/2008/04/11/the-wetware-crisis-the-dead-sea-effect/

by andromeduck • 2 years ago

60 52 2 years ago

lisper• 2 years ago

TL;DR: more talented people tend to move on to more lucrative gigs, leaving less talented people behind. The title draws an analogy to the Dead Sea, where water evaporates and leaves salt behind.

Really, that's all the article has to say. You're welcome.

sircastor• 2 years ago

I know it’s bad form to comment on articles without reading them, but my guess from the title was that this was allegorical to shepherds burning Dead Sea scrolls because they didn’t know what they were/how valuable they were to the historical research community.

6510• 2 years ago

I see what you did there... I guess one can write 7 seas of text that might one day be useful to a man looking to eat a fish.

causal• 2 years ago

Yeah that about covers it. I wonder if there's a term for metaphors that make something simple seem more complex.

RajT883• 2 years ago

After college, I lived with a friend who worked in the IT department at our college. He got called in on a Sunday once, and came back around lunchtime quite flustered.

"What's wrong?"

"Sam asked me what a traceroute was."

Sam was the longtime Novell admin. Yes this was a while ago...

"The worst part was, I patiently explained it to him, and he just asked me why that was useful?"

Just one example. There was more, of people of low competence who were very entrenched.

spitfire• 2 years ago

Well, it was Novell. So not that entrenched after all.

RajT883• 2 years ago

It was longer ago than you might think. Novell was a big deal back in the 90's and early 2000's.

jiggawatts• 2 years ago

Novell primarily used IPX and gained support for TCP/IP very late, and then that still wasn't widely used even around 2003-5. Many networks used both: IPX for Novell, and TCP/IP for everything else.

Novell NetWare was most commonly used on a single, flat network with no routers or firewalls. The networks of the era were smaller, and the setup and routing of IPX is more automatic and simpler than IP. It didn't scale "up to the Internet", but it also needed less configuration and troubleshooting.

Notably, the "tracert" tool is specifically for IP, not IPX!

That's why Sam's Novell-expert friend hadn't used it and wasn't familiar with it.

It's like being surprised that someone that had used NoSQL for their entire career wasn't familiar with 3rd normal form and relational algebra.

roenxi• 2 years ago

One of the humorous tells of a novice is they haven't yet had to deal with all their tool knowledge being obsoleted and haven't had to understand that competent people might not know said novice's favoured tools.

The amount of random trivia in tech is a vast ocean. Nobody knows all of it. Specific cultures swim in specific waters, but out of habit more than anything else.

trashtensor2• 2 years ago

I've job hopped a lot over the years and now i make about 30x what i did at my first software job. Now I'm sticking around though despite some of the craziness that i'd prefer not to deal with mostly just to prove to myself i can stick around at a job for more than 2 years.

arwhatever• 2 years ago

I’ve done much the same, but I’m curious if you have any other insights to share?

I’ve mostly settled into a place with a good pay rate-to-work expectations ratio, but I’ve also contemplated optimizing for pay alone.

trashtensor2• 2 years ago

i wouldnt say i have any particularly groundbreaking insights. Get things done, don't be afraid to learn new things as you need to. The best people i work with at any given time are the ones who are willing to try stuff and fail (but they probably wont because it seems like half of succeeding is just trying). So many devs out there get stuck because they are comfortable in their lane and are unwilling to get adventurous.

in terms of optimizing for pay vs lifestyle, i spent some years optimizing for lifestyle but im definitely in optimize for pay mode right now. Make a bit of a bag and get out and go back to optimizing for lifestyle. Things are better that way.

forgetfreeman5• 2 years ago

It's thinkpieces like this that make me believe that IT professionals specifically and white collar workers generally would benefit greatly from a mandatory 24 month stint on a construction crew. One of the most important lessons any job foreman can learn is that every crew needs at least one lame duck.

A single talented carpenter with an energetic but otherwise totally unskilled helper can accomplish more in a shift than two talented carpenters together. If you don't have someone to run to the truck for nails, hold shit, move ladders, etc. productivity suffers because your talent is wasting time on menial tasks.

Same thing applies to IT departments. The folks the author is busy sneering at are at least as valuable as the engineers they're actively trying to retain because (among other things) they soak menial tasks that would otherwise cut into their rockstar's productivity.

cannonpr• 2 years ago

Yet I’ve worked in two types of places, the ones that had a lot of menial labourers and thus lots of shuffling of stairs and carrying of nail buckets, and in places that had very few to no menial labourers and just automated away the menial tasks or designed the systems to never need menial labour in the first place. If you want to keep the metaphor some plasterers use ladders, others use stilts. In any case plenty of ways to skin a cat and people adapt to the workforce available to them.

iwontberude• 2 years ago

Fuckin’ A man… fuckin’ A.

I actually have no idea what working construction is like, but I imagine it’s hell. I hear about jackhammering causing permanent damage to people’s nerves and all sorts of random accidents.

forgetfreeman5• 2 years ago

Having worked extensively in construction and IT (including a couple decades spent writing code) I have to say construction is a much more enjoyable and rewarding career. Pay and benefits are horrible though.

DiggyJohnson• 2 years ago

You get used to it much faster than you probably are imagining. I construction job on a decent project with a decent supe can result in a very satisfying workday.

lolinder• 2 years ago

This hasn't been my experience—the truly talented developers that I've worked with do more of the foundation tasks than the "salt" developers do. They're the ones who upgrade libraries when it's time, fix Jenkins when something goes wrong, and respond to a page and solve it, on top of the rest of their work. Meanwhile they're also automating the repetitive tasks in order to free up their own time and make it possible for them to move on to other things.

Automation is so easy in software development that your analogy to construction becomes pretty weak—we don't need someone moving around ladders because when we realize that's a need we just write a script for it.

forgetfreeman5• 2 years ago

I see y'all over there pretending lifers in your average IT department don't routinely make a career bunker out of legacy support among other odious pursuits despite the author explicitly pointing it out. In any case, if the department manager isn't tasking folks at their level of competence to extract maximum utility that's a failure on their part.

quickthrower2• 2 years ago

Hold on! The construction analogy doesn’t apply because in a lot of code bases, probably most there are very few menial tasks. Some code bases need IQ 120 + to do anything non trivial in and maybe even higher.

So there are menial tasks, but menial like “yeah just redraft this architectural drawing for me mate, to take account of the new 2021 fire regs, I got actual brainy stuff to get on with”.

Pairing senior and junior devs works. But not because the junior is an idiot, but because they have stuff to learn and raw talent and can figure stiff out. They probably already know how to code well from their passion or maybe bootcamps or interning. Not from university usually unless there is lot of coursework.

I am not saying construction workers are idiots. But I am saying the OP article is implying the worst employees are left by this dead sea effect. Are you saying you can absorb untalented people on work sites. In dev teams they slow things down it would be better to pay some of them garden leave.

0823498723498722• 2 years ago

To be specific on the garden leave: "[run to the truck] for some nails" is easy to formulate yet takes effort to do. The majority of the effort for most coding tasks is the formulation, and by the time you've explained to the lame duck how to do what needs to be done, and checked that they actually did it, it would've been easier to have done it yourself.

(were one to have suitable lame duck tasks, how would wetware lame ducks compete in an LLM world?)

economicalidea2• 2 years ago

I disagree with your point that this analogy does not apply.

You say there are some codebases that need over 9000 IQ to do anything. While that might be true, it is not the norm and a small fraction, so it’s not relevant to the broader argument.

You say it’s completely different in programming. Sure it doesn’t help if one of your workers is a complete idiot — but that was not the premise - it was unskilled but energetic.

And there are always menial tasks to do — you pointed out some example yourself.

So to sum up my points are: - most codebases do not require gigabrain for every single change

- there are always menial tasks to be done in a team environment

- “untalented” people don’t necessarily slow things down in development

- in fact talent is irrelevant when writing code - I would even argue that being talented is not always an advantage, in any field or profession - but that last point probably would need some more explanation.

SideburnsOfDoom• 2 years ago

Some code bases need IQ 120 + to do anything non trivial in and maybe even higher.

And in the majority of cases, I would say that this is "accidental complexity" - i.e. it can and should be otherwise, and calling out the unnecessary cognitive load, is a benefit.

throwawayqqq11• 2 years ago

Pairing senior and junior devs works. But not because the junior is an idiot, but because they have stuff to learn.

I agree so much. The "dead sea effect" or whatever is just a lack of knowledge transfer. This problem remains over time too, eg look at the use of Cobol in the banking sector. Newer talents dont tend to learn old stuff. So what else is left for management to do in the long term than document really good and train newbies. If you think "migration", consider the timing, when the lack of knowledge already becomes pressing.

forgetfreeman5• 2 years ago

Couple of things.

1. The author said IT departments. This ostensibly includes everything from software development to systems admininstration, vendor management, and end-user support. Plenty of drudge work to be found there.

2. If your codebase requires someone to be on the edge of the bell curve of human intelligence to work on it is almost certain that mistakes have been made by very smart people.

3. The author is absolutely implying that the worst people are left, which is entirely my point. They have most likely never encountered the idea that managing folks to their strengths is more effective than trying to cram the room full of edge-of-the-bell-curve unicorns.

Terr_2• 2 years ago

not because the junior is an idiot, but because they have stuff to learn

There's also value--sometimes hard to capture--in getting an external perspective on things. In a sense, they are a new user/customer for internal systems and processes, and can identify pain points that veterans have largely learned to ignore or bypass.

ildjarn2• 2 years ago

Given that software has reuse and abstractions, we should all be embarrassed that there are any menial tasks at all.

lostlogin• 2 years ago

A short stint on a Helpdesk will remove any illusion that menial tasks will ever be solved by machines.

The users are actually amazing and seeing what happens is awe inspiring - in a bad way.

caseymarquis• 2 years ago

I grew up working construction on the side with my father, became an appprentice lineman, moved to aftermarket electrical installations in CNC machines, then ended up writing software products for over a decade. I've worked on a wide variety of software: Embedded projects requiring 2000 line assembly ISRs for legacy RS232 protocols, reverse engineering binary network protocols, implementing network server protocols from specifications, writing my own actor and dependency injection frameworks in managed languages, web application backend work, frontend work with JavaScript/TypeScript and frameworks, and so on. I also do a lot of product management, support, customer management, and internal management. Much of my customer support involves teaching external developers from large corporations how to use HTTP and SDKs.

I'm usually one of the smarter people in the room when doing software development and spend a lot of time helping others. In construction, I'm the enthusiastic idiot as the skillset is different. Most tech workers underestimate how high skilled tradework and many other positions outside their experience are, especially management work.

Having said all that, I feel uniquely qualified to offer a rebuttal. The article is describing a clear pattern that I see at large companies that are not focused on software development or lack solid engineering leadership. Ironically, the problem gets worse the more a company tries to outsource IT as they face all the same challenges with less control. It is an organizational health issue however, not some universal truth. If tech workers lack anything, it's experience with organizational growth, change, and group dynamics, not construction experience. Organizations are like people, they grow, change, figure out who they are, have identity crises and need different things in different phases. Ask the highest level/most competent manager you have access to for book recommendations if you're interested in this.

Anyway, back on topic, software "gruntwork" typically implies a department lacks the agency to automate away said gruntwork or lacks the skills to do so. As an example, I work with many organizations using the JVM, but none of them use Scala, Kotlin, Clojure, or any other "nice" JVM language. They use Java. In some cases Java 8. If you're writing Java, there is absolutely stupid gruntwork. I've written example applications for some of these organizations and I had to create code generation tools to stay sane in this environment. In software, you eliminate stupid gruntwork with tools, not people.

We do need average enthusiasts in software development, but it's not the same as construction. In construction the less talented person fetches things and does setup work. In software development, the less talented workers spend most of their time using libraries, plugins, frameworks, compilers, interpreters, databases, and languages while the more talented workers write them.

My experience isn't universal, and I'd be interested in hearing some dissenting opinions on this.

quickthrowman2• 2 years ago

Most tech workers underestimate how high skilled tradework and many other positions outside their experience are, especially management work.

Absolutely, (in particular) MEP (sheetmetal workers, pipefitters, electricians, plumbers) tradespeople and project managers are highly skilled/knowledgeable and the work requires a lot of coordination. You basically have to understand how an entire building/space is constructed from start to finish to properly plan and schedule a project.

For the record, plenty of other trades are highly skilled, but their trade/craft is more specialized/focused and doesn’t require labor from start to finish.

rjsw• 2 years ago

I have spent all week exchanging emails with the "lame duck" in my ISP's accounts department. The task involved should be menial, just copy the bank account name from my email into a form, they are incapable of getting this right.

p_l• 2 years ago

Brooks described the Surgical Team methodology, where one senior tackled the most complex tasks including design, the next senior people were helping with complex tasks and essentially "apprenticing" - with expectation that they will move on to become most senior person on a new team - plus a group of junior programmers took care of the simple "toil", while a group of supports (technical writers for documentation, tool makers, etc) together provided necessary cross-cutting skills.

This was modeled in the essay after "surgical team", with the most senior programmer being the "head surgeon" leading the operation, his direct assistants who could have major tasks delegated, and the juniors who could take on simple but still important tasks. (anaesthesiologist would probably be example of the cross-cutting support task)

BerislavLopac2• 2 years ago

  > A single talented carpenter with an energetic but otherwise totally unskilled
  > helper can accomplish more in a shift than two talented carpenters together.
  > If you don't have someone to run to the truck for nails, hold shit, move
  > ladders, etc. productivity suffers because your talent is wasting time on
  > menial tasks.

But you are mistaken -- each of us (the software engineers) already has numerous helpers who fit this description perfectly. The only difference is that we don't call them "apprentices" -- we call them compilers, interpreters, build tools and the like. Probably the only term that is commonly used on both types of helpers is "git".

Joking aside -- if you find yourself and your team spending too much time on menial tasks, you need to focus more on automating them. Of course, how much can be automated depends on how much hardware is involved in your work; as long as we're in software world, pretty much anything can be automated.

moritzwarhier3• 2 years ago

Maybe the "untalented" colleague is the solution to the marginal cost problem of automation in IT!

E.g. they won't do anything productive until CI and dev setup are "idiot-proof" - which is a feature! Noone wants to need arcane, private knowledge about how to fix that one problem with that one container in dev.

TeMPOraL2• 2 years ago

Alright, so how can I automate away daily standups, timecards, semi-regular random corporate/legal paperwork, and everything about dealing with devops team?

quickthrowman2• 2 years ago

A single talented carpenter with an energetic but otherwise totally unskilled helper can accomplish more in a shift than two talented carpenters together.

I manage electricians, and this is highly dependent on the task. In most cases I’d rather send two JWs vs a JW and apprentice.

I’d send two JW to pipe a new feeder, but I’d send an apprentice and JW if they’re just pulling wire and the pipe is installed already. An apprentice can easily feed wire while the JW pulls, but a JW is going to be much better at piping than an apprentice.

moritzwarhier3• 2 years ago

This sounds intriguing, if it is true, there should be a name for this kind of observation / org law

Terr_2• 2 years ago

as valuable as

I'd like to preemptively emphasize the distinction between "important to have" versus "marginal price to acquire", before folks start talking past one-another with different interpretations of "valuable."

arijun• 2 years ago

[2008]

dang• 2 years ago

The Wetware Crisis: the Dead Sea effect (2008) - https://news.ycombinator.com/item?id=23166786 - May 2020 (127 comments)

The Wetware Crisis: The Dead Sea Effect - https://news.ycombinator.com/item?id=22376468 - Feb 2020 (2 comments)

boffinAudio• 2 years ago

I'm a 'technician' in a company full of 'scientists', and I can say without question, the scientists need help, the technicians need knowledge, and neither of the two realms can really survive without the other.

Sometimes you need the guy who has burned his fingers repairing things, over and over, to review your design - he's going to tell you why you are going to burn your fingers. Other times, the guy doing things in a semi-glib fashion, needs to be pulled off the production line and enlightened as to why things have to be done a certain way.

To consider this relationship hierarchical is to weaponise the subject - instead, these are key relationships. A single great scientist with 2 or 3 good technicians can build amazing products.

3 scientists with no technician, rarely ever do .. but there is of course the odd exception where a technician, by making something really great, becomes a scientist, too - and scientists, without technicians and therefore having to get their fingers toasty, often make amazing technicians. (This is why the hierarchical subjugation of the subject must be resisted.)

2d8a875f-39a2-4• 2 years ago

Internet Blogger A: "I'm super talented. Who are all these no-talent losers in every $BigCorp department I end up in? They take steps to entrench themselves and have no interest in innovating. The high talent people have more options and move on. Why can't I find a high-functioning place to work at?"

Internet Blogger B: "What is wrong with hiring process? We keep getting these clueless noobs showing up. They mess up every time we ask them to maintain prod. All they want to do is some random resume-driven development and then leave us with all the problems. Why can't we hire some real software engineers for once?"

hef19898• 2 years ago

You point a finger at someone, at least three are pointing back to you. And one to the ground, that's a different story so.

moritzwarhier3• 2 years ago

The IT hiring process is broken. Amen. Not only is the IT hiring process broken in many organizations, the entire approach to IT is often broken. (...), Ruby Raley and I wrote an article for Cutter IT Journal that stated that an approach modeled after professional sport teams could well be far more effective, but no one has yet hired me to try it out.

Does this approach also apply to age? E.g., older programmers either take some higher-ranking position (management, product owner etc) or they get replaced?

The Wetware Crisis: The Dead Sea Effect (2008)

Related Stories