CTOs Agree: Cognitive Debt Is the New Technical Debt

https://shiftmag.dev/ctos-agree-cognitive-debt-is-the-new-technical-debt-10229/

by sxx0 • 1 month ago

75 72 1 month ago

ggm• 1 month ago

Decoded: it's next to useless, it's damaged pace, and it isn't value for money. It's boiling the ocean.

Implicitly they've woken up to the value proposition which was latent in their tech hires: detailed knowledge of their code and systems. They've just tossed that away, and even worse they've smooged it into unfathomable information systems which probably share aspects of it with their competitors.

AI destroyed their value.

badgersnake2• 1 month ago

CTOs / CEOs have demonstrated how completely useless they are pretty much across the board over the last few years. Groupthink and bandwagons, zero innovation or use of brain.

gmerc2• 1 month ago

CTOs were told that the companies shares will be sold off if they can't produce AI results, and that the CEO will be deposed for "not having an AI strategy", so they should kindly shut up and go along with the flow.

prymitive• 1 month ago

What’s the point of paying people a ton of money and giving them lots of power if they simply follow a script given to them by others?

Forgeties792• 1 month ago

Sounds like they’re followers and not leaders. So what are they getting paid for?

PLenz• 1 month ago

Between companies we have competition and capitalism. Within companies we have command economies at best and absolute monarchies at worst

lemagedurage• 1 month ago

AI makes it cheap to write code. That is not the same as it being cheap to ship it, or to maintain it. One participant put it cleanly: cognitive debt is the new technical debt.

It being expensive to ship or maintain software still sounds like technical debt, no?

For cognitive debt, I'd expect something like context switching and reviewing large amounts of code being exhausting.

sam_lowry_• 1 month ago

What CTOs did not notice yet is that cheap code exposes inefficiencies elsewhere.

Migrating Spring Boot apps from 3.x t 4.x is now easy given all the tooling available.

But the administrative load can't be reduced by faster code delivery and that's the new bottleneck.

helge92103• 1 month ago

CTOs noticed it. When product pipeline is empty, because engineers finished all the outstanding tasks, the engineers are awarded with more work: "The new software engineer is a product leader. Someone thinking about what the product is, not just how it works", or, in other words, engineers are going to be tasked with putting more content "the what" into the product pipeline.

bryanrasmussen2• 1 month ago

It has been my experience that nobody wants to hear the software engineer's opinion as to what will improve the product however.

olsondv4• 1 month ago

Has that not been what a senior SWE is? You’re making it sound like engineers need to be asked to implement features rather than contribute to design. At my company, if you are not coming up with new features or applications, your days are numbered.

cucumber3732842• 1 month ago

Your average mid to senior SWE is too far from the actual use of the product to actually do that well though.

There are some smaller businesses and selectively structured ones where that's not the case but for the most part organizations keep engineers from interacting with the customers enough to really know what the customers need and want.

There are some cases where a savvy engineer can say "well if I unify X then it'll be trivial to make Y self service and that'll make redundant an entire category of support tickets" or "we've been getting a lot of requests for X, maybe Y should be extensible so that those can be trivially accommodated" but by and large engineering teams are not positioned to have the knowledge to identify these things. And this is mostly an intentional organizational architecture decision.

jrs235• 1 month ago

Agreed. Business prioritization and decision making is even more important now. Hopefully the powers that be realize forcing/making busy work is worse than allowing devs to tinker, learn, and address tech itches and debt until the business figures their stuff out and makes a decision.

foobarchu• 1 month ago

Migrating Spring Boot apps from 3.x t 4.x is now easy given all the tooling available.

Is this true now? I used Claude-backed Cline in March to make that migration, and it got SO much wrong, mainly due to the lack of solid documentation and example code for it, I imagine. Probably the project where I've had to fix the most AI mistakes so far (it was incapable of fixing the mistakes itself, it just kept getting it wrong in different ways).

helge92103• 1 month ago

Considering all the possible levels of abstraction software can represent, I'm imagining it as a fractal. The worst case - a mistake can be introduced by generative AI at any of the abstraction levels at any moment. Meaning, at worst case the whole thing has to be in the head of at least one person to validate the result against. The moment a project is growing beyond a single person capacity to hold it in one head possibility of plausibly looking error introduced at any abstraction level is added to the usual multi-engineer coordination costs.

breve• 1 month ago

If an LLM is trained on GPL code then that code has become an intrinsic part of the model (because if it hasn't then what was the value of training on it). So shouldn't that model now also be licensed GPL?

And how do I know the LLM output is not reproducing substantial chunks of GPL'd code, making my code GPL?

imglorp• 1 month ago

Maybe this, but multiply by N licenses. Any given output may have ideas from all of them.

Law is probably going to take a while to catch up here.

Ekaros• 1 month ago

Or alternatively. LLM is not human. Non human generated content has no copy right protection. Meaning all generative model output is automatically public domain.

lefra• 1 month ago

I don't understand this argument. When someone uses a tool to create something, they are the copyright holder. Why would it be different when the tool is an LLM?

olsondv4• 1 month ago

Github copilot has filters for enterprise that remove the GPL code before it gets returned. At least that’s how my company has been covering itself.

altmanaltman3• 1 month ago

Two years ago, the mandate was simple: spend on AI, no questions asked. That era is over.

2 years is an era now?

Sharlin2• 1 month ago

When it comes to AI, it really does seem so.

altmanaltman3• 1 month ago

No it doesn't. Time still flows linearly. It's just bad writing.

gbxyz• 1 month ago

As a former CTO, nothing is less likely to make me believe a claim than prefixing it with "CTOs agree..."

bsenftner2• 1 month ago

Exactly, this is an advertisement for their CTO dinner clubs.

tgv• 1 month ago

Also when you add: "at other companies. We at ACME Corp. have got everything under control."?

lioeters• 1 month ago

Nine of out of ten CTOs recommend Agentic™. Use it or your company will be left behind! Side effects include cognitive atrophy, uncontrollable logorrhea, loss of context and purpose in life, and occasional torment nexus.

Jgrubb• 1 month ago

On the other hand, basically every beat lines up with our experience, so :man-shrugging:

mathgeek2• 1 month ago

I hear the dentist that doesn’t agree with the other nine is more interesting at parties.

Y-bar• 1 month ago

Most of the time that dentist is more alike JFK Jr than Alfred Wegener, unfortunately.

bregma• 1 month ago

I'm willing to wager most CTOs would agree with you.

sxx0• 1 month ago

My worry is that AI will make these companies product-obsessed, the public will mistake the output for AGI, and the real engineering underneath will keep getting overlooked.

coffeebeqn• 1 month ago

Isn’t engineering always overlooked? It’s like a means to and end to non-engineers.

The best case for engineering is always: nothing went horribly wrong in public.

jongjong3• 1 month ago

There were some good insights in there. I like the idea of changing the hiring interview process to focus on testing code review ability. I feel like this would have been useful even before AI.

A candidate who can identify tradeoffs present in some code and make insightful comments is likely good at systems thinking. It's a highly effective way to test someone's knowledge, intelligence and taste.

It's actually brilliant because it provides the company with a way to actually improve their engineering posture since the company could land on a candidate who is more skilled than the engineers doing the interviewing. Many times in my career, I met engineers who seemed mediocre at first because they didn't know a fraction of what I knew, but I later found out they also knew a fair amount that I didn't know.

I've even had an extreme experience in the past year where a colleague seemed to struggle with basic code constructs and for the first month I was thinking to myself "this guy doesn't know how to code". He asked really basic questions and said he didn't know because his background was Python and not Typescript... But I know both of these languages and I just couldn't make sense of this explanation given the kinds of questions he asked and mistakes he made. If I had been asked to review him at the time, I would probably have given him a pretty bad review but after about 1 month and a half, this guy accelerated and literally became the most productive engineer on the team. No leetcode coding test could have predicted this.

Most leetcode tech interviews are a series of puzzles which most company insiders can solve but they never include problems that the candidate could solve but which the interviewer could not.

Leetcode interviews are horrible because they test a tiny subset of moderately difficult questions under time constraints and ignore a much larger set of problems that are much more complex. There is an incorrect assumption that someone who can solve extremely complex problems can also solve moderately complex problems under time constraints. This is absolutely not the case. It's almost mutually exclusive in fact since people who work on complex problems don't have the time or interest to practice solving simpler problems so they can never solve those fast enough to compete with fresh university grads who have been practicing those for years and don't know anything else.

On a different topic, I was sceptical of this comment:

"I honestly think you can have a fifteen thousand line PR and say, I need a human to review these three lines."

15k lines is a lot of code. I could destroy any software project, irreparably with 15k lines of code and not one engineer out of hundreds would recognize it unless they read carefully. You can absolutely destroy a codebase with 15k lines of code, without any obvious backdoors or malicious code. How would I do it? I would invent counter-productive abstractions and write a lot of unit tests for them to lock down the design... Then I would watch other engineers build on top to further lock it down... Let the flawed design accumulate debt for a few years until the entire codebase becomes slow, insecure and totally unmaintable. Nobody would ever remember that I'm the one who set the project on a bad course. Nobody ever suspects the person who invents the complex abstractions and who everyone comes to with questions... Also most engineers are afraid to ask why we need a SocketContextManager or a TaskContextSwitcherMediator or the TaskOrchestrator and TaskOrchestratorFactory that comes with it... Nobody will ask why we need 10 different Helm charts to split things up into microservices... Nobody will ask why we need 1000 top level dependencies with a total of 10k nested dependencies. Nobody will question those decisions because they're afraid they will look dumb for questioning complexity.

You cannot question system complexity without acknowledging that you find something to be complex... And this can be turned against you; "Oh so this is too complex for you?"

So my view is that every single one of these 15k lines needs thorough analysis. Each of those 15k lines represent the branches from which the next generation of twigs will sprout. If that branch isn't pointing in the right direction, better just cut it off as soon as possible before it becomes a central part of the codebase.

CuriouslyC• 1 month ago

Counterpoint: you're doing a 15k line concrete implementation of a spec, or a module with fixed integration points.

There could still be a lot subtly wrong, but there are going to be stress points and high leverage validation methods you can use to avoid reading the whole 15k.

jongjong3• 1 month ago

Sure that can happen. Could be a long-running feature branch whose functionality is relatively isolated from other parts of the code. Could represent a lot of actual functionality.

But in most situations when I see a PR over a few thousand lines to an existing codebase that's highly depended upon, my stress levels tend to spike. On particularly complex projects and critical component, I'd read every single line.

Some modules I worked on, I would barely trust myself adding 100 lines to it because of how many other developers' code is depending on it. Sometimes they may be using it in ways which were not officially supported but I don't want to break backwards compatibility anyway.

So on such module, I would panic if I saw even a 1k line commit from somebody else.

15k lines is a lot. Each line comes with a probability that it introduces a bug. For a junior dev, the probability may be that 1% of lines introduce a bug... For a senior, it might be 0.1% per line and for the world's best coder, it might be 0.01%... But with 15k lines, and whatever the exact bug probabilities are, it's highly probable that the code contains at least one bug no matter who or what wrote the code.

olsondv4• 1 month ago

Yeah, that didn’t make any sense. If it’s functional, it impacts the system. 15k lines sure sounds like a lot of bloat.

jongjong3• 1 month ago

I already feel like most established big company software projects are unmaintainable. With AI, it's going to be even more so.

Currently, even though a lot of software is unmaintainable in a general sense, there's usually some engineer somewhere you can go to who can fix any given part; there's somebody out there who understands what any given part does; or there's someone who could figure it out if given some time if a minor bug needs to be fixed. With AI, it's going to create a situation where there will be parts of the code which absolutely nobody in the world understands and AI agents won't know where to pull the context from to make sense of it either and there aren't enough attention heads to make sense of it. It won't be just unmaintainable, but absolutely unmaintainable; it will be physically impossible to fix a bug without introducing new ones.

abroszka333• 1 month ago

They should think about what happens when expectations are so high that a single dev must deliver and maintain multiple products. What stops that single dev from leaving and offering the same product on his own.

uberex• 1 month ago

Yep there is the grind em to be so efficient they may as well start on their own. And now the sales and marketing side of things can ne assisted with AI, albeit not as good as pros but better than a dev and no AI to help.

adrianN2• 1 month ago

The engineer might not be good at sales or at raising money or at doing taxes or doesn’t have the appetite for risk.

abroszka333• 1 month ago

That was true before. Who wants to live on their saving for three years to build a product that might flop and do all the extra accounting and legal stuff.

But now it takes 3 months with AI, making a business and charging money has never been easier. Legal is a problem, but AI is a big help there as well.

luckylion• 1 month ago

Moats for software/web companies that are entirely "we've built it, it would be too expensive for you to replicate it" are getting much weaker. It's now quick to produce something that at least looks close to an existing product. You can't clone the backend, the know-how to handle some weird interactions etc, but still, you can get fairly far mimicking the frontend, and LLMs can write you the fancy marketing buzzwords too.

I think the part you can't easily clone will turn out to be the institutional processes that allow you to run at scale, onboard new people, deal with common requests that AI cannot on its own (e.g. legal compliance), the relationships you have with partners and vendors, the legal setups you have in place etc.

I'd assume that plenty of developers feel capable to building a better jira, and some try, and few/none succeed despite atlassian doing everything in their power to drive clients away, because cloning the project isn't the hard part with that type of product.

olsondv4• 1 month ago

Allowing non-technical PMs to ship code is fine if they’re the one getting called up in the evenings and weekends when it breaks. Maybe it’s a good exercise to show how much has effort must be applied to each commit.

simianwords• 1 month ago

Interesting comment because there are some companies that have a different model where SREs take on call for code that SWEs write. How do they work?

bravetraveler• 1 month ago

The SWEs get called too, particularly if SRE isn't able to untangle timely.

Don't want to be called? Don't make incidents. Healthy orgs, as a peer comment points out, avoid too much separation between the positions. Things like change review are supposed to avoid this, but often gets rubber-stamped for 'velocity' (into a wall).

hilariously• 1 month ago

When they "work" instead of "make a giant fucking mess" its because SWEs take shifts as SREs and the distinction is more about time in the mines and less about distinct job functions.

Balinares• 1 month ago

In the original implementation, SREs have the authority to block further releases until the SLOs are green again -- and, if it comes to that, to hand the pager back to SWE until such a time the product can pass a production readiness review again. That requires a lot of trust on both sides, though, the building of which takes both time and deliberate effort, and I don't know how other shops do it.

iLoveOncall• 1 month ago

This makes it seem like one is replacing the other. It's not. It's multiplying it.

shepik• 1 month ago

The counterargument wasn’t a dismissal. It was a reframe: this is a management problem, not a technology problem.

oh wow, what a trustworthy source of information this human written article is.

beng-nl• 1 month ago

It’s not ai penned, it’s human authored.

throwawayoaky• 1 month ago

AI written articles about AI adoption hurt my brain

Twey• 1 month ago

Regular reminder: the ‘lines of code’ metric is a liability not an asset. The best engineer is not the one who can produce the most lines of code in a day but the one who can ship the feature with the fewest lines (and prepare the system to receive the next feature with few lines, too).

Language doesn’t matter anymore: Python, Go, Rust, Node. But system design hasn’t changed.

I keep seeing this claim, and it's incoherent. Programming languages are the languages we use to express system design, and just like Sapir–Whorf in natural languages they admit, emphasize, or force consideration of different concerns in the design process. If we did get to the point where those languages ‘don't matter any more’ it would just be because modern tooling allows us to express ourselves at a higher level than these languages were designed for — but then whatever higher-level language you're using to express yourself matters instead (and matters more than ever: the higher-level you go the more inaccuracies compound).

Programming languages are designed for humans, not for computers.

rramadass• 1 month ago

Cognitive challenges are handled the way it always has been viz; modularization with well-defined interfaces and contracts, bundled as a software component. See https://en.wikipedia.org/wiki/Software_component and https://en.wikipedia.org/wiki/Component-based_software_engin...

Classic Software Engineering stages of Specification and Verification (both Formal and Informal) take center stage with the software unit being treated as a blackbox. Developers are now "Systems Engineers" responsible for end-to-end systems delivery.

See also my previous comment chain here - https://news.ycombinator.com/item?id=48431591

queenkjuul3• 1 month ago

an average AI-assisted non-engineer is probably better than an average unassisted one.

No no no no no no no. Not even vaguely remotely close to being true. Why are managers so delusional

sleight42• 1 month ago

If your team can generate code faster than it can review it, you have a bottleneck. The constraint is human judgment, not output.

Given almost any group of coders, who are proficient in their technologies of choice, coding speed almost always exceeds reviewing speed. Same as it ever was.

Given any arbitrarily worthwhile problem to solve, the actual cost is in solving the problem or the cost to serve or both. Cost to serve if the solution is inefficient or the problem space barely tractable.

CTOs Agree: Cognitive Debt Is the New Technical Debt

Related Stories