northfield27

Excellent article! Aptly describes what I have been feeling and thinking about the claims many AI optimists make.

---

He produced a great deal of code, [...] He could not, when asked, explain how any of it actually worked. [...] When opinions were voiced even as high as a V.P., he fought back.

AI has democratized coding, but people have yet to understand that it takes expertise to actually design a system that can handle scale. Of course, you can build a PoC in a few hours with Claude code, but that wouldn't generate value.

The reason why we see such examples in the workplace is because of the false marketing done by CEOs and wrapper companies. It just gives people a false hope that "they can just build things" when they can only build demos.

Another reason is that the incentives in almost every company have shifted to favour a person using AI. It's like the companies are purposefully forcing us to use AI, to show demand for AI, so that they can get a green signal to build more data centers.

---

So you have overconfident, novices able to improve their individual productivity in an area of expertise they are unable to review for correctness. What could go wrong?

This is one much-needed point to raise.

I have many people around me saying that people my age are using AI to get 10x or 100x better at doing stuff. How are you evaluating them to check if the person actually improved that much?

I have experienced this excessively on twitter since last few months. It is like a cult. Someone with a good following builds something with AI, and people go mad and perceive that person as some kind of god. I clearly don't understand that.

Just as an example, after Karpathy open-sourced autoresearch, you might have seen a variety of different flavors that employ the same idea across various domains, but I think a Meta researcher pointed out that it is a type of search method, just like Optuna does with hyperparameter searching.

Basically, people should think from first principles. But the current state of tech Twitter is pathetic; any lame idea + genAI gets viral, without even the slightest thought of whether genAI actually helps solve the problem or improve the existing solution.

(Side note: I saw a blog from someone from a top USA uni writing about OpenClaw x AutoResearch, I was like WTF?! - because as we all know, OpenClaw was just a hype that aged like milk)

---

The slowness was not a tax on the real work; the slowness was the real work.

Well Said! People should understand that learning things takes time, building things takes time, and understanding things deeply takes time.

Someone building a web app using AI in 10 mins is not ahead but behind the person who is actually going one or two levels of abstractions deeper to understand how HTML/JS/Next.js works.

I strongly believe that the tech industry will realise this sooner or later that AI doesn't make people learn faster, it just speeds up the repetitive manual tasks. And people should use the AI in that regard only.

The (real) cognitive task to actually learn is still in the hands of humans, and it is slow, which is not a bottleneck, but that's just how we humans are, and it should be respected.

You mean you coding is fully coding agent driven?

I think I was right about AI coding tools, but I was probably just early: https://news.ycombinator.com/item?id=46769188#46782672

I’ve been using and experimenting with LLM-based coding tools for ~2 years, mostly in research systems, and I expected both the upside and the limitations to show up eventually.

There was a period mid last year to early this year where social media hype around AI coding felt very strong. A lot of the discussion was about agents and multi-agent setups replacing large parts of engineering work. My experience didn’t really match that level of capability in practice, especially when it came to larger codebases and long-term maintainability.

The gap I kept seeing was between small, impressive demos and actual production constraints like correctness, debugging, security, and system understanding.

I also remember seeing “vibe coding” discussed around that time. My initial reaction was skepticism because it felt like it was abstracting away too much of the engineering process. I might be misremembering details, but even now my view is that these systems are still not very reliable without strong human structure around them.

I don’t really blame individuals for the hype cycle. The incentives on social platforms and in the industry were clearly aligned toward showcasing success cases and productivity gains, so that naturally shaped the narrative.

My current view is not that these tools are useless. They are clearly helpful for many tasks. But I think the more interesting problem is how to integrate them into real engineering workflows without increasing maintenance burden or accumulating hidden complexity over time.

We probably need better patterns for using them in production settings, and better expectations around what they can and cannot safely automate today.

Haha, i was taking N4 and N6, but didn’t figure the steps after that.

To win we need to let knight die because rook can move multiple steps to kill the king.

From a third person perspective R2 is a deceptive move that takes advantage algorithm to make the black king back off to kill its knight.

I have also felt something similar.

A few days back, I tried to implement a PDF reader by pure vibe coding. I used all my free Antigravity, Cursor, and Co-pilot tokens to create a half-baked, but working Next.js PDF-reader that (to be honest) I wouldn't have glued together without 2 weeks of work. As an MLE, I have done negligible web development using JavaScript and have mostly worked with Python and C.

But the struggle actually started after the free tokens were exhausted. I was feeling anxious to even look into those Next.js files. I am not able to describe, but it was probably some kind of fear - fear of either not being able to debug/implement a new feature, or not willing to put in precious hours (precious because of FOMO that I could do something cool with AI-paired vibe coding) to understand and build the feature myself.

I abandoned that project since that day. Haven't opened it yet - partly because I am waiting for the renewal of free tokens.

If everything passes, it ships, automatically. If something fails, the agent fixes it. A human only gets involved when the system genuinely doesn’t know what to do.

This right here, he conveniently skipped elaborating how will a human fix a system he hasn’t genuinely built but just prompted. Writing code by hand helps you build a mind-map that is helpful while debugging. Without it, a human needs to go through all of the complex code. This is not efficiency, but delaying the heavy lifting to a later date, because system fail and they fail often (nowadays).

And based on how its phrased, it looks like AI has written this article.

Agreed but sadly, many people are too optimistic with AI and are completely forgetting that they can be the part of next layoffs.

And others are not able to believe the (not extreme) but visible speed boost from pragmatic use of AI.

And sadly, whenever the discussion about the collective financial disadvantage of AI to software engineers will start and wherever it goes…

The owners and employers will always make the profits.

You can write 10x the code - good code. You can review and edit it before committing it. Nothing changes from a code quality perspective. Only speed.

I agree, but this is an oversimplification - we don't always get the speed boosts, specifically when we don't stay pragmatic about the process.

I have a small set of steps that I follow to really boost my productivity and get the speed advantage.

(Note: I am talking about AI-coding and not Vibe-coding) - You give all the specs, and there are "some" chances that LLM will generate code exactly required. - In most cases, you will need to do >2 design iterations and many small iterations, like instructing LLMs to properly handle error gracefully recover from errors. - This will definitely increase speed 2x-3x, but we still need to review everything. - Also, this doesn't take into account the edge cases our design missed. I don't know about big tech, but when I have to do the following to solve a problem

1. Figure out a potential solution

2. Make a hacky POC script to verify the proposed solution actually solves the problem

3. Design a decently robust system as a first iteration (that can have bugs)

4. Implement using AI

5. Verify each generated line

6. Find out edge cases and failure modes missed during design and repeat from step3 to tweak the design, or repeat from step4 to fix bug.

WHENEVER I jump directly from 1 -> 3 (vague design) -> 5, Speed advantages become obsolete.

Agreed

Isn't that too much work?

Instead, just learning concepts with AI and then using HI (Human Intelligence) & AI to solve the problem at hand—by going through code line by line and writing tests - is a better approach productivity-, correctness-, efficiency-, and skill-wise.

I can only think of LLMs as fast typists with some domain knowledge.

Like typists of government/legal documents who know how to format documents but cannot practice law. Likewise, LLMs are code typists who can write good/decent/bad code but cannot practice software engineering - we need, and will need, a human for that.

++Hard Agree.

I partially agree, but I don’t think “design ways to test the code without inspecting it manually line by line” is a good strategy.

Tests only cover cases you already know to look for. In my experience, many important edge cases are discovered by reading the implementation and noticing hidden assumptions or unintended interactions.

When something goes wrong, understanding why almost always requires looking at the code, and that understanding is what informs better tests.

Instead of "vending machine", I see many people calling generative AI "slot machine", which more aptly describes current genAI tools.

Yes, we can use it 10,000 times to refine our recipes, but "did we learn from it"? I am doubtful about that, given that even after running with the same prompt 10 times, it will give different answers in 8/10 responses.

But I am very confident that I can learn by iterating and printing designs on a 3D printer.

Edit: Changed title

Previous title: "Anthropic: AI Coding shows no productivity gains; impairs skill development"

The previous title oversimplified the claim to "all" developers. I found the previous title meaningful while submitting this post because most of the false AI claims of "software engineer is finished" has mostly affected junior `inexperienced` engineers. But I think `junior inexperienced` was implicit which many people didn't pick.

The paper makes a more nuanced claim that AI Coding speeds up work for inexperienced developers, leading to some productivity gains at the cost of actual skill development.

For AI-coding, I use Copilot with whatever default model under free plan it gives.

Agreed. AI-generated code is "mid" by nature. You won’t feel amazed seeing AI-generated code because its "reasoning" competes with that of a potato.

I have recently started exploring AI-coding -- note that I said AI coding and not vibe-coding because that is for the brain-dead.

By AI coding, I mean I know the inputs, outputs, structures the code should have and the necessary context to write the code. Then, articulating the requirements in English as best as I can and feeding it to agents.

Needless to say, the code is pathetic; it chooses to implement meaningless abstractions even after explicitly providing the design and plan to follow.

I don’t understand how we, as a collective species, agreed to believe the criminally wrong lies of tech CEOs that, instead of implementing a “reliable system by hand,” we choose to convey our ideas and instructions in an “ambiguous,” “inconsistent,” and “context-dependent” language (English), which is then passed through a probabilistic system to generate the reliable system.

Agreed. I believe this is going to be the trend.

I don’t think LLM context will able to digest large codebases and their algorithms are not going to reason like SREs in the next coming years. And given the current hype and market, investors are gonna pull out with recessions all over the world and we will see another AI Winters.

Code has become a commodity. Corporate engineering hierarchy will be much flat in coming years both horizontally and vertically - one staff will command two senior engineers with two juniors each, orchestrating N agents each.

I think that’s it - this is the end of bootcamp devs. This will act as a great filter and probably decrease the mass influx of bootcamp devs.

Thanks, this sounds interesting!

Agreed - ego does push individuals towards mastery because it provides stability and status. But the ones too tied to ego gets lost in history.

What I don't agree with is the current hype based marketing.

First calculators and computers were sold with the promise of improved productivity and those tools actually delivered in the same time frame. But thats not the case with AI.

Consider this promise: "AI will eat up coding completely". Each CEO claims that 92%-95% percent of coding is done by AI in their company. But if you are active here, you would know that even some senior engineers at big tech don't find it useful in production. I think that these "hype-funded" engineers at CEO's company are just working on toy projects, because I find claims of senior engineers true.

These CEOs make wild claims when many of them haven't actually programmed much. Every interview with these CEOs pushes the year-of-AGI 2-3 years forward.

This is such a time where I really hope the investors actually look into the details, experience the gap between the claims and reality and pull their funding.

I have tried using coding agents and was left doing more with myself, so I agree with the broader idea.

https://x.com/gdb/status/2013164524606775544?s=61 But who is gonna tell him.

I don’t understand what these CEO are up to.

It depends.

If you have the power (as the post mentions - like a CEO) you can suggest, direct or butcher a project and no one would see you as a negative person.

But you can get butchered when you don't have the authority to poke around your concerns.

I would prefer to see the ship sink instead of shooting myself in the foot and risking my influence and credibility - as another comment on this thread said "Sometimes, you have to let people fail".

One thing this blog probably misses is what happens when we suggest something to fix a project’s trajectory, they ignore it, but eventually they silently adopt the suggestion and improve the project.

This results in a net loss of ROI: we risk being seen as a negative person, while no one acknowledges our goodwill or instincts, because the project lead presents the improvement as their own win through a strategic change of plan.

Is it a promotion?

I agree with your POV. Technology has been improved exponentially and from a human POV having systems smarter than us can be good (if we have control over them but to think about that makes us smarter and powerful, but that's another discussion).

I agree with this but I just don't believe that the "promised AI" would be an LLM but some other technology.

But specialized harnesses paired with frontier models are also better security engineers than I am, dollar for dollar versus my cost. They're better pentesters than me, for the relative costs.

Don't mind but some parts of cybersecurity are more brute-force in nature, where AI excels, but using that to claim frontier models are better security engineers than humans is a bit of an oversimplification. You are BETTER because of your expertise of X years in cyber security, and I can assure you that there is not frontier model out there with the context capacity to challenge you on that. And that makes you a 10x or even 100x more cost effective (dollar-wise). And your (perceived) cost will go on increasing with each adding year to your experience. AI can do things faster than you but your judgement on which subtree/direction to traverse while pentesting makes you a better than an LLM and the cost of this (human) judgement will increase day-by-day because other humans need other humans to take accountability.

This has been my experience as well.

What I use LLMs for

I don't use Claude Code, Codex or any other hyped-up, VC-approved, buzzword-compliant, productivity-theater SF tool for actual day-to-day coding. I just use ChatGPT outside of my editor in browser.

The only tasks that I feel LLMs right now can do somewhat-reliably without human supervision include:

1. Search: To search and compare different things; that is boring to do manually. As an example, I checked how M1 Pro compares to M4. ChatGPT crawls the web to compare both the processors on similar specs and provides a conclusion but I need to check and verify the claims it reached by briefly reading the sources.

2. Documentation: I have to do things on time so I offload reading documentations for syntax or how-to-do-it-with-this-library to ChatGPT.

3. Single task scripts: I let ChatGPT create me boring and one-time-use python scripts for trivial tasks.

I don't spend ChatGPT's limited but free GPT-5-(latest-model) tokens for such trivial tasks - I have a special keybinding on my mac that fires up ChatGPT in incognito for me.

But don't get me wrong, I find LLMs pretty useful for PoC-level stuff and I do like instructing my agents to make me a half-baked PoC that makes me feel like a revolutionary SF-founder for an afternoon before I delete the repo and go back to writing real code like a responsible adult. Because nothing says ‘disruptive genius’ like ctrl+z and moving on.

---

Why I find vibe coding in production not useful

Like sean goedecke, I feel that using LLMs in production is not useful and (in fact) anti-productive because it gives junior engineer's free pass to push code without actually learning/reading it (Humans often tend to chose the easiest path to accomplish given problem). And due to this, its a pain to review junior PR these days.

I have two reasons why LLMs are not good at production coding:

1. Context: There are enough discussions out on the internet that says LLMs are limited by context. Humans are better because they can handle and work with multiple contexts in mind - an LLMs knows how to solve the user problem, but the engineer knows what business problem this piece of code solves, in which context the code will run, with other process on same service and in which context of other services, what company code idioms to follow while writing code and what will make your pair-programmer think that you are smart.

2. Lack of deterministic output: I would not over-simply by saying its just a probabilistic system but you can feel it right? It would sometimes give correct, decent and sometimes wrong answers for the same prompt. Sometimes it would enlighten you with a very interesting insight/perspective and other times it would be as dumb as a toaster trying to file your taxes - consistently inconsistent (when you reached context rot), yet somehow still sold as “intelligent.”

---

A brief rant

My manager fear-mongers me nearly everyday saying LLMs are really good at coding, there are at-most 10 years left to earn as programmer and stuff. The fact he is hasn't touched coding in last 8 years.

He thinks his half-baked trivial ideas translated to code by an overpriced probabilistic system is a breakthrough for his company.

I am non-native english speaker and you would have probably sensed that by now. I often get tasks to write documentations, and every time my manager doesn't understand something, he takes it as a chance to break my confidence saying that you are bad at using LLM for language related tasks because my english is poor.

But (at-least right now) I strongly feel this is just a hype:

- CEOs want to get-rich-quick or they want to increase company profits by "ai-enabling" it and getting salary hikes.

- Companies are hyping this tech to recalibrate engineer salaries that has skyrocketed since covid.

- My manager wants to make me feel replaceable.