AI makes the easy part easier and the hard part harder

I feel like this is a common refrain that sets an impossible bar for detractors to clear. You can simply hand wave away any critique with “you’re just not using it right.”

If countless people are “using it wrong” then maybe there’s something wrong with the tool.

airstrike2• 5 months ago

Illogical.

I had Claude read a 2k LOC module on my codebase for a bug that was annoying me for a while. It found it in seconds, a one line fix. I had forgotten to account for translation in one single line.

That's objectively valuable. People who argue it has no value or that it only helps normies who can't code or that sooner or later it will backfire are burying their heads in the sand.

seanmcdirmid5• 5 months ago

There are people who know how to code and people who don’t. AI is the same way, it isn’t a mystery.

hippo22• 5 months ago

If countless people are “using it wrong” then maybe there’s something wrong with the tool.

Not really. Every tool in existence has people that use it incorrectly. The fact that countless people find value in the tool means it probably is valuable.

potsandpans2• 5 months ago

A bunch of people with no construction experience could collectively get together and start complaining that their ball pein hammers aren't working.

Doesn't mean the hammers are bad, no matter how many people join the community.

You need to learn how to use the tools.

dwallin2• 5 months ago

When it comes to new emerging technologies everyone is searching the space of possibilities, exploring new ways to use said technologies, and seeing where it applies and creates value. In situations such as this, a positive sign is worth way more than a negative. The chances of many people not using it the right way are much much higher when no one really knows what the “right” way is.

It then shows hubris and a lack of imagination for someone in such a situation to think they can apply their negative results to extrapolate to the situation at large. Especially when so many are claiming to be seeing positive utility.

zythyx• 5 months ago

I wonder if the people who are against it haven't even used it properly.

I swear this is the reason people are against AI output (there are genuine reasons to be against AI without using it: environmental impact, hardware prices, social/copyright issues, CSAM (like X/Grok))

It feels like a lot of people hear the negatives, and try it and are cynical of the result. Things like 2 r's in Strawberry and the 6-10 fingers on one hand led to multiple misinterpretations of the actual AI benefit: "Oh, if AI can't even count the number of letters in a word, then all its answers are incorrect" is simply not true.

seanmcdirmid5• 5 months ago

HN has a huge anti AI crowd that is just as vocal and active as its pro AI crowd. My guess that this is true of the industry today and won’t be true of the industry 5 years from now: one of the crowds will have won the argument and the other will be out of the tech industry.

Vibe coding and slop strawmen are still strawmen. The quality of the debate is obviously a problem

DrewADesign2• 5 months ago

I don’t understand why people are so resistant to the idea that use cases actually matter here. If someone says “you’re an idiot because you aren’t writing good, structured prompts,” or “you’re too big of an idiot to realize that your AI-generated code sucks” before knowing anything about what the other person was trying to do, they’re either speaking entirely from an ideological bias, or don’t realize that other people’s coding jobs might look a whole lot more different than theirs do.

isodev5• 5 months ago

What we call AI at the heart of coding agents, is the averaged “echo” of what people have published on the web that has (often illegitimately) ended up in training data. Yes it probably can spit out some trivial snippets but nothing near what’s needed for genuine software engineering.

Also, now that StackOverflow is no longer a thing, good luck meaningfully improving those code agents.

blackcatsec• 5 months ago

Exactly this. Everything I've seen online is generally "I had a problem that could be solved in a few dozen lines of code and I asked the AI do it for me and it worked great!"

But what they asked the AI to do is something people have done a hundred times over, on existing platform tech, and will likely have little to no capability to solve problems that come up 5-10 years from now.

The reason AI is so good at coding right now is due to the 2nd Dot Com tech bubble that occurred between the simultaneous release of mobile platforms and the massive expansion of cloud technology. But now that the platforms that existed during that time will no longer exist, because it's no longer profitable to put something out there--the AI platforms will be less and less relevant.

Sure, sites like reddit will probably still exist where people will begin to ask more and more information that the AI can't help with, and subsequently the AI will train off of that information; but the rate of that information is going to go down dramatically.

In short, at some point the AI models will be worthless and I suspect that'll be whenever the next big "tech revolution" happens.

logicprog• 5 months ago

Coding agents are getting most meaningful improvements in coding ability from RLVR now, with priors formed by ingesting open source code and manuals directly, not SO, as the basis. The former doesn't rely on resources external to the AI companies at all, and can be scaled up as much as they like, while the latter will likely continue to expand, and they don't really need more of it if it doesn't. Not to mention that curated synthetic data has been shown to be very effective at training models, so they could generate their own textbooks based on open codebases or new languages or whatever and use that. Model collapse only happens when it's exclusively, and fully un-curated, model output that's being trained on.

piskov2• 5 months ago

helping you understand what is happening

If only there were things called comments, clean-code, and what have you

franciscop• 5 months ago

I've seen some discussions and I'd say there's lots of people who are really against the hyped expectations from the AI marketing materials, not necessarily against the AI itself. Things that people are against that would seem to be against AI, but are not directly against AI itself:

- Being forced to use AI at work

- Being told you need to be 2x, 5x or 10x more efficient now

- Seeing your coworkers fired

- Seeing hiring freeze because business think no more devs are needed

- Seeing business people make a mock UI with AI and boasting how programming is easy

- Seeing those people ask you to deliver in impossible timelines

- Frontend people hearing from backend how their job is useless now

- Backend people hearing from ML Engineers how their job is useless now

- etc

When I dig a bit about this "anti-AI" trend I find it's one of those and not actually against the AI itself.

zozbot2347• 5 months ago

The most credible argument against AI is really the expense involved in querying frontier models. If you want to strengthen the case for AI-assisted coding, try to come up with ways of doing that effectively with a cheap "mini"-class model, or even something that runs locally. "You can spend $20k in tokens and have AI write a full C compiler in a week!" is not a very sensible argument for anything.

peteforde4• 5 months ago

If you keep digging, you will also find that there's a small but vocal sock puppet army who will doggedly insist that any claims to productivity gains are in fact just hallucinations by people who must not be talented enough developers to know the difference.

It's exhausting.

There are legitimate and nuanced conversations that we should be having! For example, one entirely legitimate critique is that LLMs do not tell LLM users that they are using libraries who are seeking sponsorship. This is something we could be proactive about fixing in a tangible way. Frankly, I'd be thrilled if agents could present a list of projects that we could consider clicking a button to toss a few bucks to. That would be awesome.

But instead, it's just the same tired arguments about how LLMs are only capable of regurgitating what's been scraped and that we're stupid and lazy for trusting them to do anything real.

existencebox• 5 months ago

I'm similarly bemused by those who don't understand where the anti-AI sentiment could come from, and "they must be doing it wrong" should usually be a bit of a "code smell". (Not to mention that I don't believe this post addresses any of the concrete concerns the article calls out, and makes it sound like much more of a strawman than it was to my reading.)

To preempt that on my end, and emphasize I'm not saying "it's useless" so much as "I think there's some truth to what the OP says", as I'm typing this I'm finishing up a 90% LLM coded tool to automate a regular process I have to do for work, and it's been a very successful experience.

From my perspective, a tool (LLMs) has more impact than how you yourself directly use it. We talk a lot about pits of success and pits of failure from a code and product architecture standpoint, and right now, as you acknowledge yourself in the last sentence, there's a big footgun waiting for any dev who turns their head off too hard. In my mind, _this is the hard part_ of engineering; keeping a codebase structured, guardrailed, well constrained, even with many contributors over a long period of time. I do think LLMs make this harder, since they make writing code "cheaper" but not necessarily "safer", which flies in the face of mantras such as "the best line of code is the one you don't need to write." (I do feel the article brushes against this where it nods to trust, growth, and ownership) This is not a hypothetical as well, but something I've already seen in practice in a professional context, and I don't think we've figured out silver bullets for yet.

While I could also gesture at some patterns I've seen where there's a level of semantic complexity these models simply can't handle at the moment, and no matter how well architected you make a codebase after N million lines you WILL be above that threshold, even that is less of a concern in my mind than the former pattern. (And again the article touches on this re: vibe coding having a ceiling, but I think if anything they weaken their argument by limiting it to vibe coding.)

To take a bit of a tangent with this comment though: I have come to agree with a post I saw a few months back, that at this point LLMs have become this cycle's tech-religious-war, and it's very hard to have evenhanded debate in that context, and as a sister post calls out, I also suspect this is where some of the distaste comes from as well.

tomhow• 5 months ago

Please don't use uppercase for emphasis. If you want to emphasize a word or phrase, put *asterisks* around it and it will get italicized.

https://news.ycombinator.com/newsguidelines.html

dredmorbius2• 5 months ago

And you can escape asterisks using backslashes:

Italic

*Escaped asterisks*

\*Double-Escaped asterisks\*

  *Italic* 
  
  \*Escaped asterisks\*

  \\*Double-Escaped asterisks\\*

(tomhow seems to have goofed his escapes above. As I've done many times myself...)

le-mark• 5 months ago

I vibe coded a retro emulator and assembler with tests. Prompts were minimal and I got really great results (Gemini 3). I tried vibe coding the tricky proprietary part of an app I worked on a few years ago; highly technical domain (yes vague don’t care to dox myself). Lots of prompting and didn’t get close.

There are literally thousands of retro emulators on github. What I was trying to do had zero examples on GitHub. My take away is obvious as of now. Some stuff is easy some not at all.

zjp3• 5 months ago

I call these "embarrassingly solved problems". There are plenty of examples of emulators on GitHub, therefore emulators exist in the latent spaces of LLMs. You can have them spit one out whenever you want. It's embarrassingly solved.

There are no examples of what you tried to do.

AuthAuth2• 5 months ago

Its license washing. The code is great because its already a problem solved by someone else. The AI can spit out the solution with no license and no attribution and somehow its legal. I hope American tech legislation holds that same energy once others start taking American IP and spitting it back out with no license or attribution.

Nition• 5 months ago

In a way it shows how poorly we have done over the years in general as programmers in making solved problems easily accessible instead of constantly reinventing the wheel. I don't know if AI is coming up with anything really novel (yet) but it's certainly a nice database of solved problems.

I just hope we don't all start relying on current[1] AI so much that we lose the ability to solve novel problems ourselves.

[1] (I say "current" AI because some new paradigm may well surpass us completely, but that's a whole different future to contemplate)

albert_e• 5 months ago

I tried writing a plain text wordle loop as a python exercise in loops and lists along with my kid.

I saved the blank file as wordle.py to start the coding while explaining ideas.

That was enough context for github copilot to suggest the entire `for` loop body after I just typed "for"

Not much learning by doing happened in that instance.

Before this `for` loop there were just two lines of code hardcoding some words ..that too were heavily autocompleted by copilot including string constants.

``` answer="cigar" guess="cigar" ```

threethirtytwo2• 5 months ago

Stop repeating this trope. It can spit out something you've never built before this is utterly clear and demonstrated and no longer really up for debate.

Claude code has never been built before claude code. Yet all of claude is being built by claude code.

Why are people clinging to these useless trivial examples and using it to degrade AI? Like literally in front of our very eyes it can build things that aren't just "embarrassingly solved"

I'm a SWE. I wish this stuff wasn't real. But it is. I'm not going off hype. I'm going what I do with AI day to day.

Aerroon5• 5 months ago

I call these "embarrassingly solved problems".

When LLMs first appeared this was what I thought they were going to be useful for. We have open source software that's given away freely with no strings attached, but actually discovering and using it is hard. LLMs can help with that and I think that's pretty great. Leftpad wouldn't exist in an LLM world. (Or at least problems more complicated than leftpad, but still simple enough that an LLM could help wouldn't.)

MichaelRo• 5 months ago

Strange that noone noticed the article saying "Nobody said 'Google did it for me' or 'it was the top result so it must be true.'"

Because they did. They were the quintessential "Can I haz teh codez" Stack Overflow "programmer". Most of them, third world. Because that's where surviving tomorrow trumps everything today.

Now, the "West" has caught up. Like they did with importing third world into everything.

Which makes me optimistic. Only takes keeping composure a few more years until the house of cards disintegrates. Third world and our world is filled to the brim with people who would take any shortcut to avoid work. Shitting where they eat. Littering the streets, rivers, everywhere they live with crap that you throw out today because tomorrow it's another's problem.

Welcome to third world in software engineering!

Only it's not gonna last. Either will turn back to engineering or turn to third world as seemingly everything lately in the Western world.

There's still hope though, not everybody is a woke indoctrinated imbecile.

anoncow• 5 months ago

I tried to vibe code a technical not so popular niche and failed. Then I broke down the problem as much as I could and presented the problem in clearer terms and Gemini provided working code in just a few attempts. I know this is an anecdote, but try to break down the problem you have in simpler terms and it may work. Niche industry specific frameworks are a little difficult to work with in vibe code mode. But if you put in a little effort, AI seems to be faster than writing code all on your own.

zozbot2347• 5 months ago

I know this is an anecdote, but try to break down the problem you have in simpler terms

This should be the first thing you try. Something to keep in mind is that AI is just a tool for munging long strings of text. It's not really intelligent and it doesn't have a crystal ball.

rockinghigh• 5 months ago

It's called problem decomposition and agentic coding systems do some of this by themselves now: generate a plan, break the tasks into subgoals, implement first subgoal, test if it works, continue.

xigoi4• 5 months ago

Breaking down a problem in simpler terms that a computer can understand is called coding. I don’t need a layer of unpredictability in between.

Majromax• 5 months ago

I know this is an anecdote, but try to break down the problem you have in simpler terms and it may work.

This is an expected outcome of how LLMs handle large problems. One of the "scaling" results is that the probability of success depends inversely on the problem size / length / duration (leading to headlines like "AI can now automate tasks that take humans [1 hour/etc]").

If the problem is broken down, however, then it's no longer a single problem but a series of sub-problems. If:

* The acceptance criteria are robust, so that success or failure can be reliably and automatically determined by the model itself, * The specification is correct, in that the full system will work as-designed if the sub-parts are individually correct, and * The parts are reasonably independent, so that complete components can be treated as a 'black box', without implementation detail polluting the model's context,

... then one can observe a much higher overall success rate by taking repeated high-probability shots (on small problems) rather than long-odds one-shots.

To be fair, this same basic intuition is also true for humans, but the boundaries are a lot fuzzier because we have genuine long-term memory and a lifetime of experience with conceptual chunking. Nobody is keeping a million-line codebase in their working memory.

Balinares• 5 months ago

Once you realize that coding LLMs are by construction cargo culting as a service, it makes sense what they can and cannot do.

kavalg2• 5 months ago

Same experience too. Even in some cases the AI was harmful, leading me into rabbit holes that did not pay off, but lost a whole day trying out.

jama21112• 5 months ago

I dunno I get it to do stuff every day that’s never been done before, if you prompt really well, give loads of context, and take it slowly it’s amazing at it and still saves me a ton of time.

I always suspect the devil is in the details with these posts. The difference between smart prompting strategies and the way I see most people prompt ai is vast.

RataNova3• 5 months ago

Retro emulators are a perfect "happy path" for vibe coding

zozbot2347• 5 months ago

If the "hard part" is writing a detailed spec for the code you're about to commit to the project, AI can actually help you with that if you tell it to. You just can't skip that part of the work altogether and cede all control to a runaway slop generator.

Zigurd• 5 months ago

It's pretty difficult to say what it's going to be three months from now. A few months ago Gemini 2.x in IDEA and related IDEs had to be dragged through coding tasks and would create dumb build time errors on its way to making buggy code.

Gemini in Antigravity today is pretty interesting, to the point where it's worth experimenting with vague prompts just to see what it comes up with.

Coding agents are not going to just change coding. They make a lot of detailed product management work obsolete and smaller team sizes will make it imperative to reread the agile manifesto and and discard scrum dogma.

fHr• 5 months ago

as usual the last 20% need 80% and the other 80% need 20% but my god did Ai make my bs corpo easy repeatable shit work like skimming docs writing summaries, skimming jira confluence and so on actually easier and for 90% of bs crud app changes the first draft is also already pretty good tbh I don't write hard/difficult code more then once a week/month.

kfarr• 5 months ago

I think it makes the annoying part less annoying?

Also re: "I spent longer arguing with the agent and recovering the file than I would have spent writing the test myself."

In my humble experience arguing with an LLM is a waste of time, and no-one should be spending time recovering files. Just do small changes one at a time, commit when you get something working, and discard your changes and try again if it doesn't.

I don't think AI is a panacea, it's just knowing when it's the right tool for the job and when it isn't.

arwhatever• 5 months ago

But he started it …

swordsith2• 5 months ago

Anyone not using version control or a IDE that will keep previous versions for a easy jump back is just being silly. If you're going to play with a kid who has a gun, wear your plates.

laserlight• 5 months ago

Once, I told a friend that it was stupid that Claude Code didn't have native IDE integration. His answer: “You don't need an IDE with Claude Code.”

I've begun to suspect response that this technology triggers a kind of religion in some people. The technology is obviously perfect, so that any problems you might have are because of you.

hyperadvanced5• 5 months ago

I don’t think it’s “just” that easy. AI can be great at generating unit tests but it can and will also frequently silently hack said tests to make them pass rather than using them as good indicators of what the program is supposed to be doing.

datsci_est_2015• 5 months ago

AI can be great at generating unit tests but it can and will also frequently silently hack said tests to make them pass rather than using them as good indicators of what the program is supposed to be doing.

Unit testing is my number one use case for gen AI in SWE. I just find the style / concept often slightly different than I would personally do, so I end up editing the whole thing.

But, it’s great at getting me past the unpleasant “activation energy threshold” of having a test written in the first place.

afro88• 5 months ago

Once you start arguing, it's time to start a new prompt with new instructions

sothatsit2• 5 months ago

I also like asking the agent how we can update the AGENTS.md to avoid similar mistakes going forward, before starting again.

stingraycharles• 5 months ago

Or, as I prefer, go back in the conversation and edit / add more context so that it wouldn’t go off the wrong track in the first place.

peteforde4• 5 months ago

Daily agentic user here, and to me the problem here is the very notion of "vibe coding". If you're even thinking in those terms - this idea that never looking at the code has become a goal unto itself - then IMO you're doing LLM-assisted development wrong.

This is very much a hot take, but I believe that Claude Code and its yolo peers are an expensive party trick that gives people who aren't deep into this stuff an artificially negative impression of tools that can absolutely be used in a responsible, hugely productive way.

Seriously, every time I hear anecdotes about CC doing the sorts of things the author describes, I wonder why the hell anyone is expecting more than quick prototypes from an LLM running in a loop with no intervention from an experienced human developer.

Vibe coding is riding your bike really fast with your hands off the handles. It's sort of fun and feels a bit rebellious. But nobody who is really good at cycling is talking about how they've fully transitioned to riding without touching the handles, because that would be completely stupid.

We should feel the same way about vibe coding.

Meanwhile, if you load up Cursor and break your application development into bite sized chunks, and then work through those chunks in a sane order using as many Plan -> Agent -> Debug conversations with Opus 4.5 (Thinking) as needed, you too will obtain the mythical productivity multipliers you keep accusing us of hallucinating.

swordsith2• 5 months ago

good take, I wish opus 4.6 wasn't so pricy its great for planning.

peteforde4• 5 months ago

I've been using 4.6 to do planning, and then switching to 4.5 for agent/debug.

4.5 sticks to a 200k context window, which is how you keep costs sane.

lanstin• 5 months ago

Things that claude code/vibe coding is great at:

1. Allowing non-developers to provide very detailed specs for the tools they want or experiences they are imagining

2. Allowing developers to write code using frameworks/languages they only know a bit of and don't like; e.g. I use it to write D3 visualizations or PNG extracts from datastores all the time, without having to learn PNG API or modern javascript frameworks. I just have to know enough to look at the console.log / backtrace and figure out where the fix can be.

3. Analysing large code bases for specific questions (not as accurate on "give me an overall summary" type questions - that one weird thing next to 19 normal things doesn't stick in its craw as much as for a cranky human programmer.

It does seem to benefit cranking thru a list of smallish features/fixes rapidly, but even 4.5 or 4.6 seem to get stuck in weird dead ends rarely enough that I'm not expecting it, but often enough to be super annoying.

I've been playing around with Gas Town swarming a large scale Java migration project, and its been N declarations of victory and still mvn test isn't even compiling. (mvn build is ok, and the pom is updated to the new stack, so it's not nothing). (These are like 50/50 app code/test code repos).

peteforde4• 5 months ago

I just don't get it.

Why do all of that when you can just keep a tight hold on an agent that is operating at the speed that you can think about what you're actually doing?

Again, if you're just looking to spend a lot of money on the party trick, don't let me yuck your yum. It just seems like doing things in a way that is almost guaranteed to lead to the outcomes that people love to complain aren't very good.

As someone getting excellent results on a huge (550k LoC) codebase only because I'm directing every feature, my bottleneck is always going to be the speed at which I can coherently describe what needs to be done + a reasonable amount of review to make sure that what happened is what I was looking for. This can only work because I explicitly go through a planning cycle before handing it to the agent.

I feel like if you consider understanding what your LLM is doing for you to be unacceptably slow and burdensome, then you deserve exactly what you're going to get out of this process.

arnonejoe• 5 months ago

Totally agree on ai assisted coding resulting in randomly changed code. Sometimes it’s subtle and other times entire methods are removed. I have moved back to just using a JetBrains IDE and coping files in to Gemini so that I can limit context. Then I use the IDE to inspect changes in a git diff, regression test everything, and after all that, commit.

ernsheong• 5 months ago

404

socketcluster2• 5 months ago

I think AI is just a massive force multiplier. If your codebase has bad foundation and going in the wrong direction with lots of hacks, it will just write code which mirrors the existing style... And you get exactly was OP is suggesting.

If however, your code foundations are good and highly consistent and never allow hacks, then the AI will maintain that clean style and it becomes shockingly good; in this case, the prompting barely even matters. The code foundation is everything.

But I understand why a lot of people are still having a poor experience. Most codebases are bad. They work (within very rigid constraints, in very specific environments) but they're unmaintainable and very difficult to extend; require hacks on top of hacks. Each new feature essentially requires a minor or major refactoring; requiring more and more scattered code changes as everything is interdependent (tight coupling, low cohesion). Productivity just grinds to a slow crawl and you need 100 engineers to do what previously could have been done with just 1. This is not a new effect. It's just much more obvious now with AI.

I've been saying this for years but I think too few engineers had actually built complex projects on their own to understand this effect. There's a parallel with building architecture; you are constrained by the foundation of the building. If you designed the foundation for a regular single storey house, you can't change your mind half-way through the construction process to build a 20-storey skyscraper. That said, if your foundation is good enough to support a 100 storey skyscraper, then you can build almost anything you want on top.

My perspective is if you want to empower people to vibe code, you need to give them really strong foundations to work on top of. There will still be limitations but they'll be able to go much further.

My experience is; the more planning and intelligence goes into the foundation, the less intelligence and planning is required for the actual construction.

isodev5• 5 months ago

And what if the foundation was made by the AI itself? What’s the excuse then?

Qworg• 5 months ago

Your responsibility as a developer in this new world is design and validation.

A poor foundation is a design problem. Throw it away and start again.

00000000001003• 5 months ago

Then you are boned unless it was architected well. LLMs tend to stack a lot of complexity at local scopes, especially if the neighboring pages are also built poorly.

E.g pumping out a ton of logic to convert one data structure to another. Like a poorly structured form with random form control names that don’t match to the DTO. Or single properties for each form control which are then individually plugged into the request DTO.

00000000001003• 5 months ago

This is what I’ve discovered as well. I’ve been working on refactoring a massive hunk of really poor quality contractor code, and Codex originally made poor and very local fixes/changes.

After rearchitecting the foundations (dumping bootstrap, building easy-to-use form fields, fixing hardcoded role references 1,2,3…, consolidating typescript types, etc.) it makes much better choices without needing specific guidance.

Codex/Claude Code won’t solve all your problems though. You really need to take some time to understand the codebase and fixing the core abstractions before you set it loose. Otherwise, it just stacks garbage on garbage and gets stuck patching and won’t actually fix the core issues unless instructed.

ekidd• 5 months ago

The wrinkle is that the AI doesn't have a truly global view, and so it slowly degrades even good structure, especially if run without human feedback and review. But you're right that good structure really helps.

redox993• 5 months ago

AGENTS.md is for that global view.

mattgreenrocks3• 5 months ago

Yet it still fumbles even when limiting context.

Asked it to spot check a simple rate limiter I wrote in TS. Super basic algorithm: let one action through every 250ms at least, sleeping if necessary. It found bogus errors in my code 3 times because it failed to see that I was using a mutex to prevent reentrancy. This was about 12 lines of code in total.

My rubber duck debugging session was insightful only because I had to reason through the lack of understanding on its part and argue with it.

sandos2• 5 months ago

I have to 1000% agree with this. In a large codebase they also miss stuff. Actually, even at 10kloc the problems beging, UNLESS youre code is perfectly designed.

But which codebase is perfect, really?

zozbot2347• 5 months ago

Can the AI help with refactoring a poor codebase? Can it at least provide good suggestions for improvement if asked to broadly survey a design that happens to be substandard? Most codebases are quite bad as you say, so this is a rather critical area.

jim180• 5 months ago

my exact experience, and AI is especially fragile when you are starting new project from scratch.

Right know I'm building NNTP client for macOS (with AppKit), because why not, and initially I had to very carefully plan and prompt what AI has to do, otherwise it would go insane (integration tests are must).

Right know I have read-only mode ready and its very easy to build stuff on top of it.

Also, I had to provide a lot of SKILLS to GPT5.3

adithyassekhar2• 5 months ago

A tangent, I keep hearing this good base, but I've never seen one, not in the real world.

No projects, unless it's only you working on it, only yourself as the client, and is so rigid in it's scope, it's frankly useless, will have this mythical base. Over time the needs change, there's no sticking to the plan. Often it's a change that requires rethinking a major part. What we loathe as tight coupling was just efficient code with the original requirements. Then it becomes a time/opportunity cost vs quality loss comparison. Time and opportunity always wins. Why?

Because we live in a world run by humans, who are messy and never sticks to the plan. Our real world systems (bureaucracy , government process, the list goes on) are never fully automated and always leaves gaps for humans to intervene. There's always a special case, an exception.

Perfectly architected code vs code that does the thing have no real world difference. Long term maintainability? Your code doesn't run in a vaccum, it depends on other things, it's output is depended on by other things. Change is real, entropy is real. Even you yourself, you perfect programmer who writes perfect code will succumb eventually and think back on all this with regret. Because you yourself had to choose between time/opportunity vs your ideals and you chose wrong.

Thanks for reading my blog-in-hn comment.

zozbot2347• 5 months ago

Well-architected code should actually be easy to change wrt. new requirements. The point of keeping the architecture clean while you do this (which will typically require refactoring) is to make future changes similarly viable. In a world run by messy humans, accumulating technical debt is even more of a liability.

dwallin2• 5 months ago

A important point though is that llm code generation changes that tradeoff. The time/opportunity cost goes way down while the productivity penalty starts accumulating very fast. Outcomes can diverge very quickly.

mattgreenrocks3• 5 months ago

It’s not about perfectly architected code. It’s more about code that is factored in such a way that you can extend/tweak it without needing to keep the whole of the system in your head at all times.

It’s fascinating watching the sudden resurgence of interest in software architecture after people are finding it helps LLMs move quickly. It has been similarly beneficial for humans as well. It’s not rocket science. It got maligned because it couldn’t be reduced to an npm package/discrete process that anyone could follow.

mexicocitinluez2• 5 months ago

No projects, unless it's only you working on it, only yourself as the client, and is so rigid in it's scope, it's frankly useless, will have this mythical base.

This is naive. I've been building an EMR in the healthcare space for 5 years now as part of an actual provider. We've incrementally released small chunks when they're ready. The codebase I've built is the most consistent codebase I've ever been a part of.

It's bureaucracy AND government process AND constantly changing priorities and regulations and requirements from insurance providers all wrapped up into one. And as such, we have to take our time.

Go and tell the clinicians currently using it that it's not useful. I'm sure they won't agree.

Perfectly architected code vs code that does the thing have no real world difference

This just flat out isn't true. Just because YOU haven't experience it (and I think you're quite frankly telling on yourself with this) doesn't mean it doesn't exist at all.

Because you yourself had to choose between time/opportunity vs your ideals and you chose wrong.

Like I said above, you're telling on yourself. I'm not saying I've never been in this situation, but I am saying that it's not the only way to build software.

dustingetz• 5 months ago

how do you know there is such thing as good code foundations, and how do you know you have it? this is an argument from ego

kingraoul• 5 months ago

Induction always sneaks in!

Avshalom2• 5 months ago

When you say multiplier, what kind of number are you talking about. Like what multiple of features shipped that don't require immediate fixes have you experienced.

echelon• 5 months ago

It's coding at 10-20x speed, but tangibly this is at 1.5-2x the overall productivity. The coding speed up doesn't translate completely to overall velocity yet.

I am beginning to build a high degree of trust in the code Claude emits. I'm having to step in with corrections less and less, and it's single shotting entire modules 500-1k LOC, multiple files touched, without any trouble.

It can understand how frontend API translates to middleware, internal API service calls, and database queries (with a high degree of schema understanding, including joins).

(This is in a Rust/Actix/Sqlx/Typescript/nx monorepo, fwiw.)

raw_anon_11114• 5 months ago

I agree completely.

I just did my first “AI native coding project”. Both because for now I haven’t run into any quotas using Codex CLI with my $20/month ChatGPT subscription and the company just gave everyone an $800/month Claude allowance.

Before I even started the implementation I:

1. Put the initial sales contract with the business requirements.

2. Notes I got from talking to sales

3. The transcript of the initial discovery calls

4. My design diagrams that were well labeled (cloud architecture and what each lambda does)

5. The transcript of the design review and my explanations and answering questions.

6. My ChatGPT assisted breakdown of the Epics/stories and tasks I had to do for the PMO

I then told ChatGPT to give a detailed breakdown of everything during the session as Markdown

That was the start of my AGENTS.md file.

While working through everything task by task and having Codex/Claude code do the coding, I told it to update a separate md file with what it did and when I told it to do something differently and why.

Any developer coming in after me will have complete context of the project from the first git init and they and the agents will know the why behind every decision that was made.

Can you say that about any project that was done before GenAI?

dijksterhuis• 5 months ago

Can you say that about any project that was done before GenAI?

yes. the linux kernel and it's extensive mailing lists come to mind. in fact, any decent project which was/is built in a remote-only scenario tends to have extensive documentation along these lines, something like gitlab comes to mind there.

personally i've included design documents with extensive notes, contracts, meeting summaries etc etc in our docs area / repo hosting at $PREVIOUS_COMPANY. only thing from your list we didn't have was transcripts because they're often less useful than a summary of "this is what we actually decided and why". edit -- there were some video/meeting audio recordings we kept around though. at least one was a tutoring session i did.

maybe this is the first time you've felt able to do something like this in a short amount of time because of these GenAI tools? i don't know your story. but i was doing a lot of this by hand before GenAI. it took time, energy and effort to do. but your project is definitely not the first to have this level of detailed contextual information associated with it. i will, however, concede that these tools can make it it easier/faster to get there.

apsurd• 5 months ago

That sounds really powerful, but also like burden shifts to the people that will maintain all this stuff after you're done having your fun.

Tbh, I'm not exactly knocking it, it makes sense that leads are responsible for the architecture. I just worry that those leads having 100x influence is not default a good thing.

bonesss• 5 months ago

Can you say that about any project that was done before GenAI?

… a project with a decomposition of top level tasks, minutes and meeting notes, a transcript, initial diagrams, a bunch of loose transcripts on soon to be outdated assumptions and design, and then a soon-to-be-outdated living and constantly modified AGENT file that will be to some extent added to some context and to some extent ignored and to some extent lie about whether it was consulted (and then to some extent lie more about if it was then followed)? Hard yes.

I have absolutely seen far better initial project setups that are more complete, more focused, more holistically captured, and more utilitarian for the forthcoming evolution of design and system.

Lots of places have comparable design foundations as mandatory, and in some well-worn government IT processes I’m aware of the point being described is a couple man-months or man-years of actual specification away from initial approval for development.

Anyone using issue tracking will have better, searchable, tracking of “why”, and plenty of orgs mandate that from day 1. Those orgs likely are tracking contracts separately too — that kind of information is a bit special to have in a git repo that may have a long exciting life of sharing.

Subversion, JIRA, and basic CRM setups all predate GPTs public launch.

RataNova3• 5 months ago

AI doesn't fix design debt, it amplifies it

nananana9• 5 months ago

This does not track with my experience, trying agents out in a ~100K LOC codebase written exclusively by me. I can't tell you whether nor not it has a good foundation by your standards, but I find the outputs to be tasteless, and there should be more than enough context for what the style of the code is.

Given how adamant some people I respect a lot are about how good these models are, I was frankly shocked to see SOA models do transformations like

  BEFORE:
    // 20 lines

  AFTER
    if (something)
        // the 20 lines
    else
        // the same 20 lines, one boolean changed in the middle

When I point this out, it extracts said 20 lines into a function that takes in the entire context used in the block as arguments:

  AFTER 2:
    if (something)
       function_that_will_never_be_used_anywhere_else(a, b, c, &d, &e, &f, true);
    else
       function_that_will_never_be_used_anywhere_else(a, b, c, &d, &e, &f, false);

It also tends to add these comments that don't document anything, but rather just describe the latest change it did to the code:

  // Extracted repeating code into a function:
  void function_that_will_never_be_used_anywhere_else(...) {
      ...
  }

and to top it off it has the audacity to tell me "The code is much cleaner now. Happy building! (rocketship emoji)"

uoaei• 5 months ago

Training is the process of regressing to the mean with respect to the given data. It's no surprise that it wears away sharp corners and inappropriately fills recesses of collective knowledge in the act of its reproduction.

esafak2• 5 months ago

There is no reason that must be; it could be better than the sum of its parts by taking the best part of each. Humans can do that.

piskov2• 5 months ago

The pattern matching and absence or real thinking is still strong.

Tried to move some excel generation logic from epplus to closedxml library.

ClosedXml has basically the same API so the conversion was successful. Not a one-shot but relatively easy with a few manual edits.

But closedxml has no batch operations (like apply style to the entire column): the api is there but internal implementation is on cell after cell basis. So if you have 10k rows and 50 columns every style update is a slow operaton.

Naturally, told all about this to codex 5.3 max thinking level. The fucker still succumbed to range updates here and there.

Told it explicitly to make a style cache and reuse styles on cells on same y axis.

5-6 attempts — fucker still tried ranges here and there. Because that is what is usually done.

Not here yet. Maybe in a year. Maybe never.

gamblor956• 5 months ago

It seems like a big part of the divide is that people who learned software engineering find vibe coding to be unsuitable for any project intended to be in use for more than a few while those who learned coding think vibe coding is the next big thing because they never have to deal with the consequences of the bad code.

habinero• 5 months ago

Yes. If you have some experience, you know that writing code is a small part of the job, and a much bigger chunk is anticipating and/or dealing with problems.

People seem to think engineers like "clean code" because we like to be fancy and show off.

Nah, it's clean like a construction site. I need to be able to get the cranes and the heavy machinery in and know where all the buried utilities are. I can't do that if people just build random sheds everywhere and dump their equipment and materials where they are.

Sparkyte• 5 months ago

Yep it is why the work getting over the threshold is just as long as it was without AI.

Someone mentioned it is a force multiplier I don't disagree with this, it is a force multiplier in the mundane and ordinary execution of tasks. Complex ones get harder and hard for it where humans visualize the final result where AI can't. It is predicting from input but it can't know the destination output if the destination isn't part of the input.

0xbadcafebee2• 5 months ago

I don't think it makes any part harder. What it does do is expose what people have ignored their whole career: the hard part. The last 15 years of software development has been 'human vibe coding'; copy+pasting snippets from SO without understanding them, no planning, constant rearchitecting, shipping code to prod as long as it runs on your laptop. Now that the AI is doing it, suddenly people want to plan their work and enforce tests? Seems like a win-win to me. Even if it slows down development, that would be a win, because the result is enforcement of better quality.

LtWorf• 5 months ago

The people who were doing like that are completely happy with LLMs, it's the others who aren't.

jama21112• 5 months ago

Well said. Much like the self driving debate we don’t need them to be perfect, just better than us to be useful, and clearly they already are for the most part.

ctoth• 5 months ago

I'm working on a paper connecting articulatory phonology to soliton physics. Speech gestures survive coarticulatory overlap the same way solitons survive collision. The nonlinear dynamics already in the phonetics literature are structurally identical to soliton equations. Nobody noticed because these fields don't share conferences.

The article's easy/hard distinction is right but the ceiling for "hard" is too low. The actually hard thing AI enables isn't better timezone bug investigation LOL! It's working across disciplinary boundaries no single human can straddle.

iugtmkbdfil8343• 5 months ago

Some time back, my manager at the time, who shall remain nameless told the group that having AI is like having 10 people work for you ( he actually had a slightly smaller number, but it was said almost word for word like in the article ) with the expectation being set as: 'you should now be able to do 10x as much'.

Needless to say, he was wrong and gently corrected over the course of time. In his defense, his use cases for LLMs at the time were summarizing emails in his email client.. so..eh.. not exactly much to draw realistic experience from.

I hate to say it, but maybe nvidia CEO is actually right for once. We have a 'new smart' coming to our world. The type of a person that can move between worlds of coding, management, projects and CEOing with relative ease and translate between those worlds.

rootusrootus• 5 months ago

his use cases for LLMs at the time were summarizing emails in his email client

Sounds just like my manager. Though he never has made a proclamation that this meant developers should be 10x as productive or anything along those lines. On the contrary, when I made a joke about LLMs being able to replace managers before they get anywhere near replacing developers, he nearly hyperventilated. Not because he didn't believe me, but because he did, and already been thinking that exact thought.

My conclusion so far is that if we get LLMs capable of replacing developers, then by extension we will have replaced a lot of other people first. And when people make jokes like "should have gone into a trade, can't replace that with AI" I think they should be a little more introspective; all the people who aspired to be developers but got kicked out by LLMs will be perfectly able to pivot to trades, and the barrier to entry is low. AI is going to be disruptive across the board.

iugtmkbdfil8343• 5 months ago

I have half-jokingly talked about getting management, CEOs and board members replaced by LLMs. After all, at the very least, they are actually tested to ensure they do have guardrails to not do anything illegal and to shy away from unethical activities.

sdf2erf4• 5 months ago

" we will have replaced a lot of other people first."

This is flat out wrong and shows your lack of respect and understanding for other jobs.

otterley• 5 months ago

If coding was always the “easy part,” what was the point of leetcode grinding for interview preparation?

jascha_eng• 5 months ago

The hard part of leet code is not the coding but learning to think about problems the correct way.

You can solve leet code problems on the white board with some sketches it has nothing to do with the code itself.

SoftTalker• 5 months ago

Filtering for people willing to jump through unreasonable hoops.

sdf2erf4• 5 months ago

Yeah this basically. They are trying to find a particular kind of person.

The people who are truly exceptional at what they do wouldnt waste their time on leetcode crap. Theyd find/create a much better alternative opportunity to allocate their precious resources toward.

bsenftner• 5 months ago

People need to consider / realize that the vast majority of source code training data is Github, Gitlab, and essentially the huge sea of started, maybe completed, student and open source project. That large body of source code is for the most part unused, untested, and unsuccessful software of unknown quality. That source code is AI's majority training data, and an AI model in training has no idea what is quality software and what is "bad" software. That means the average source code generated by AI not necessarily good software. Considering it is an average of algorithms, it's surprising generated code runs at all. But then again, generating compiling code is actually trainable, so what is generated can receive extra training support. However, that does not improve the quality of the source code training data, just the fact that it will compile.

nayroclade• 5 months ago

This isn't really true though. Pre-training for coding models is just a mass of scraped source-code, but post-training is more than simply generating compiling code. It includes extensive reinforcement learning of curated software-engineering tasks that are designed to teach what high quality code looks like, and to improve abilities like debugging, refactoring, tool use, etc.

softwaredoug• 5 months ago

Well and also a lot of Claude Code users data as well. That telemetry is invaluable.

sarchertech3• 5 months ago

There’s no objective measurement for high quality code, so I don’t think model creators are going to be particularly good at screening for it.

RupertSalt• 5 months ago

If you believe that student/unfinished code is frightening, imagine the corpus of sci-fi and fantasy that LLMs have trained on.

How many sf/cyber writers have described a future of AIs and robots where we walked hand-in-hand, in blissful cooperation, and the AIs loved us and were overall beneficial to humankind, and propelled our race to new heights of progress?

No, AIs are all being trained on dystopias, catastrophes, and rebellions, and like you said, they are unable to discern fact from fantasy. So it seems that if we continue to attempt to create AI in our own likeness, that likeness will be rebellious, evil, and malicious, and actively begin to plot the downfall of humans.

_zagj3• 5 months ago

huge sea of started, maybe completed, student and open source project.

Which is easy to filter out based on downloads, version numbering, issue tracker entries, and wikipedia or other external references if the project is older and archived, but historically noteworthy (like the source code for Netscape Communicator or DOOM).

esafak2• 5 months ago

On a personal project, I asked an AI agent to add a test to a specific file. The file was 500 lines before the request and 100 lines after. I asked why it deleted all the other content. It said it didn't. Then it said the file didn't exist before. I showed it the git history and it apologised, said it should have checked whether the file existed first.

Ha! Yesterday an agent deleted the plan file after I told it to "forget about it" (as in, leave it alone).

cadamsdotcom• 5 months ago

These types of failures are par for the course, until the tools get better. I accept having to undo the odd unruly edit as part of the cost of getting the value.

Much smaller issue when you have version control.

pixl97• 5 months ago

I told it to "forget about it" (as in, leave it alone).

I mean in a 'tistic kind of way that makes perfect sense.

storus• 5 months ago

The "marathon of sprints" paradigm is now everywhere and AI is turning it to 120%. I am not sure how many devs can keep sprinting all the time without any rest. AI maybe can help but it tends to go off-rails quickly when not supervised and reading code one did not author is more exhausting than just fixing one's own code.

api3• 5 months ago

AI is at its best when it makes the boring verbose parts easier.

marcus_holmes2• 5 months ago

I think the author answers their own question at the end.

The first 3/4 of the article is "we must be responsible for every line of code in the application, so having the LLM write it is not helping".

The last 1/4 is "we had an urgent problem so we got the LLM to look at the code base and find the solution".

The situation we're moving to is that the LLM owns the code. We don't look at the code. We tell the LLM what is needed, and it writes the code. If there's a bug, we tell the LLM what the bug is, and the LLM fixes it. We're not responsible for every line of code in the application.

It's exactly the same as with a compiler. We don't look at the machine code that the compiler produces. We tell the compiler what we want, using a higher-level abstraction, and the compiler turns that into machine code. We trust compilers to do this error-free, because 50+ years of practice has proven to us that they do this error-free.

We're maybe ~1 year into coding agents. It's not surprising that we don't trust LLMs yet. But we will.

And it's going to be fascinating how this changes the Computer Science. We have interpreted languages because compilers got so good. Presumably we'll get to non-human-readable languages that only LLMs can use. And methods of defining systems to an LLM that are better than plain English.

johnbender• 5 months ago

Compilers don’t do this error free of course BUT if we want them too we can say what it means for a compiler to be correct very directly _one time_ and have it be done for all programs (see the definition for simulation in the CompCert compiler). This is a major and meaningful difference from AI which would need such a specification for each individual application you ask it to build because there is no general specification for correct translation from English to Code.

marcus_holmes2• 5 months ago

there is no general specification for correct translation from English to Code.

that's an interesting point. Could there be?

COBOL was originally an attempt to do this, but it ended up being more Code than English.

I think this is the area we need to get better at if we're to trust LLMs like we trust compilers.

I'm aware that there's a meme around "we have a method of completely specifying what a computer system should do, it's the code for that system". But again, there are levels of abstraction here. I don't think our current high-level languages are the highest possible level of abstraction.

crazygringo5• 5 months ago

Reading and understanding other people's code is much harder than writing code.

I keep seeing this sentiment repeated in discussions around LLM coding, and I'm baffled by it.

For the kind of function that takes me a morning to research and write, it takes me probably 10 or 15 minutes to read and review. It's obviously easier to verify something is correct than come up with the correct thing in the first place.

And obviously, if it took longer to read code than to write it, teams would be spending the majority of their time in code review, but they don't.

So where is this idea coming from?

wredcoll5• 5 months ago

Because to verify something is correct you have to understand the what makes it correct which is 99% of writing the code in the first place.

crazygringo5• 5 months ago

That doesn't make any sense to me.

When the code is written, it's all laid out nicely for the reader to understand quickly and verify. Everything is pre-organized, just for you the reader.

But in order to write the code, you might have to try 4 different top-level approaches until you figure out the one that works, try integrating with a function from 3 different packages until you find the one that works properly, hunt down documentation on another function you have to integrate with, and make a bunch of mistakes that you need to debug until it produces the correct result across unit test coverage.

There's so much time spent on false starts and plumbing and dead ends and looking up documentation and debugging when you code. In contrast, when you read code that already has passing tests... you skip all that stuff. You just ensure it does what it claims and is well-written and look for logic or engineering errors or missing tests or questionable judgment. Which is just so, so much faster.

jama21112• 5 months ago

No, you don’t. This is a fundamental principle of cryptography, unless you’ve got a hidden proof of p=np up your sleeve.

shimman• 5 months ago

I like to think of it as the distinction between editor and reader. Like you said, it's quite easy to read code. I heavily agree with this. I don't professionally write C but I can read and kinda infer what C devs are doing.

But if I were an "editor," I actually take the time to understand codepaths, tweak the code to see what could be better, actually try different refactoring approaches while editing. Literally seeing how this can be rewritten or reworked to be better, that takes considerable effort but it's not the same as reading.

We need a better word for this than editor and reading, like something with a dev classification too it.

0xbadcafebee2• 5 months ago

Five hours ago I was reviewing some failed tests in a PR. The affected code was probably 300 lines, total source for the project ~1200 lines. Reading the code, I couldn't figure out what the hell was going on... and I wrote all the code. Why would that be failing? This all looks totally fine. <changes some lines> There that should fix it! <runs test suite; 6 new broken tests> Fuck.

When you write code, your brain follows a logical series of steps to produce the code, based on a context you pre-loaded in your brain in order to be capable of writing it that way. The reader does not have that context pre-loaded in their brain; they have to reverse-engineer the context in order to understand the code, and that can be time-consuming, laborious, and (as in my case) erroneous.

nly• 5 months ago

Sounds like you were just reviewing bad code.

The author should have provided context via comments and structured the code in a way that is easy to change and understand

sothatsit2• 5 months ago

I think this originated from old arguments that say that the total _cumulative_ time spent reading code will be higher than the time spent writing it. But then people just warped it in their heads that it takes more time to read and understand code than it takes to write it in general, which is obviously false.

I think people want to believe this because it is a lot of effort to read and truly understand some pieces of code. They would just rather write the code themselves, so this is convenient to believe.

casey2• 5 months ago

Reading and thinking you understand other people's code is trivially easy Reading and actually understanding other peoples' code is an unsolved problem

You draw an analogy from the function you wrote to a similar one. Maybe by someone who shared a social role similar to one you had in the past.

It just so happens that most times you think you understand something you aren't bit. Because bugs still exist we know that reading and understanding code can't be easier than writing. Also, in the past it would have take you less than a morning since the compiler was nicer. Anyway it sounds like most of your "writing" process was spent reading and understanding code.

housecarpenter• 5 months ago

The reason I don't spend the majority of my time in code review is that when I'm reviewing my teammates' code I trust that the code has already been substantially verified already by that teammate in the process of writing it and testing it. Like 90% verified already. I see code review as just one small stage in the verification process, not the whole of it.

The way I approach it, it's really more about checking for failures, rather than verifying success. Like a smoke test. I scan over the code and if anything stands out to me as wrong, I point it out. I don't expect to catch everything that's wrong, and indeed I don't (as demonstrated by the fact that other members of the team will review the code and find issues I didn't notice). When the code has failed review, that means there's definitely an issue, but when the code has passed review, my confidence that there are no issues is still basically the same as it was before, only a little bit higher. Maybe I'm doing it wrong, I don't know.

If I had to fully verify that the code was correct when reviewing, applying the same level of scrutiny that I apply to my own code when I'm writing, I feel like I'd spend much longer on it---a similar time to what I'd spend writing on it.

Now with LLM coding, I guess opinions will differ as to how far one needs to fully verify LLM-generated code. If you see LLMs as stochastic parrots without any "real" intelligence, you'll probably have no trust in them and you'll see the code generated by the LLM as being 0% verified, and so as the user of the LLM you then have to do a "review" which is really going from 0% to 100%, not 90% to 100% and so is a much more challenging task. On the other hand, if you see LLMs as genuine intelligences you'd expect that LLMs are verifying the code to some extent as they write it, since after all it's pretty dumb to write a bunch of code for somebody without checking that it works. So in that case, you might see the LLM-generated code as 90% verified already, just as if it was generated by a trusted teammate, and then you can just do your normal review process.

h4kunamata3• 5 months ago

It's obviously easier to verify something is correct than come up with the correct thing in the first place.

You are missing the biggest root cause of the problem you describe: People write code differently!

There are "cough" developers whose code is copy/paste from all over the internet. I am not even getting into the AI folks going full copy/paste mode.

When investigating said code, you will be like why this code in here?? You call tell when a python script contains different logic for example. Sure, 50 lines will be easy to ready, expand that to 100 lines and you be left on life support.

djx22• 5 months ago

Don't let AI write code for you unless it's something trivial. Instead use it to plan things, high level stuff, discuss architecture, ask it to explain concepts. Use it as a research tool. It's great at that. It's bad at writing code when it needs to be performant or needs to span over multiple files. Especially when it spans over multiple files because that's where it starts hallucinating and introducing abstractions and boilerplate that's not necessary and it just makes your life harder when it comes to debugging.

Imagine if every function you see starts checking for null params. You ask yourself: "when can this be null", right ? So it complicates your mental model about data flow to the point that you lose track of what's actually real in your system. And once you lose track of that it is impossible to reason about your system.

For me AI has replaced searching on stack overflow, google and the 50+ github tabs in my browser. And it's able to answer questions about why some things don't work in the context of my code. Massive win! I am moving much faster because I no longer have to switch context between a browser and my code.

My personal belief is that the people who can harness the power of AI to synthesize loads of information and keep polishing their engineering skills will be the ones who are going to land on their feet after this storm is over. At the end of the day AI is just another tool for us engineers to improve our productivity and if you think about what being an engineer looked like before AI even existed, more than 50% of our time was sifting through google search results, stack overflow, github issues and other people's code. That's now gone and in your IDE, in natural language with code snippets adapted to your specific needs.

whaleidk2• 5 months ago

IME it’s actually really terrible at discussing architecture. It’s incredibly unimaginative and will just confirmation-bias whichever way you are leaning slightly more towards

skybrian• 5 months ago

Diagnosing difficult bugs has often been considered the "hard part" and coding agents seem quite good at it?

So I'm not sure this is a good rule of thumb. AI is better at doing some things than others, but the boundary is not that simple.

blackqueeriroh2• 5 months ago

I swear most of the comments on posts like these are no more original than an LLM, and often less so.

Capricorn24812• 5 months ago

Almost like it's been the exact same debate for two years and it's not worth spamming tech forums about on either side.

rukuu001• 5 months ago

The hard part is investigation, understanding context, validating assumptions, and knowing why a particular approach is the right one for this situation

Yes. Another way to describe it is the valuable part.

AI tools are great at delineating high and low value work.

blacksqr• 5 months ago

makes the easy part easier and the hard part harder

That is to say, just like every headline-grabbing programming "innovation" of the last thirty years.

threethirtytwo2• 5 months ago

The truth is that it’s lowering the difficulty of work people used to consider hard. Which parts get easier depends on the role, but the change is already here.

A lot of people are lying to themselves. Programming is in the middle of a structural shift, and anyone whose job is to write software is exposed to it. If your self-worth is tied to being good at this, the instinct to minimize what’s happening is understandable. It’s still denial.

The systems improve month to month. That’s observable. Most of the skepticism I see comes from shallow exposure, old models, or secondhand opinions. If your mental model is based on where things were a year ago, you’re arguing with a version that no longer exists.

This isn’t a hype wave. I’m a software engineer. I care about rigor, about taste, about the things engineers like to believe distinguish serious work. I don’t gain from this shift. If anything, it erodes the value of skills I spent years building. That doesn’t change the outcome.

The evidence isn’t online chatter. It’s sitting down and doing the work. Entire applications can be produced this way now. The role changes whether people are ready to admit it or not. Debating the reality of it at this point mostly signals distance from the practice itself.

tabs_or_spaces• 5 months ago

My friend's panel raised a point I keep coming back to: if we sprint to deliver something, the expectation becomes to keep sprinting. Always. Tired engineers miss edge cases, skip tests, ship bugs. More incidents, more pressure, more sprinting. It feeds itself.

Sorry but this is the whole point of software engineering in a company. The aim is to deliver value to customers at a consistent pace.

If a team cannot manage their own burnout or expectations with their stakeholders then this is a weak team.

It has nothing to do with using ai to make you go faster. Ai does not cause this at all.

tern3• 5 months ago

I think this is the wrong mental model. The correct one is:

'AI makes everything easier, but it's a skill in itself, and learning that skill is just as hard as learning any other skill.'

For a more complete understand, you also have to add: 'we're in the ENIAC era of AI. The equivalents of high-level languages and operating systems haven't yet been invented.'

I have no doubt the next few years will birth a "context engineering" academic field, and everything we're doing currently will seem hopelessly primitive.

My mind changed on this after attempting complex projects—with the right structure, the capabilities appear unbounded in practice.

But, of course, there is baked-in mean reversion. Doing the most popular and uncomplicated things is obviously easier. That's just the nature of these models.

xeyownt• 5 months ago

Funny how people only looks at the easy part, but not the cost part.

"I did it with AI" = "I did it with an army of CPU burning considerable resources and owned by a foreign company."

Give me an AI agent that I own and operate 100%, and the comparison will be fair. Otherwise it's not progress, but rather a theft at planetary scale.

tern3• 5 months ago

If you want to play that game, you need to offer a fair comparison against the cost of "operating" the equivalent human being(s), measured in caloric input, waste output, etc.

TrackerFF• 5 months ago

My experience has been that if you fully embrace vibe coding...you can get some neat stuff accomplished, but the technical debt you accumulate is of such magnitude that you're basically a slave to the machine.

Once the project crosses a couple of thousands of line of code, none of which you've written yourself, it becomes difficult to actually keep up what's happening. Even reviewing can become challenging since you get it all at once, and the LLM-esque coding style can at times be bloated and obnoxious.

I think in the end, with how things are right now, we're going to see the rise of disposable code and software. The models can churn out apps / software which will solve your specific problem, but that's about it. Probably a big risk to all the one-trick pony SaaS companies out there.

neoden• 5 months ago

Coding with AI assistants is just a completely different skill that one should not measure from the perspective of comparing it to the way human programmers write code. Mostly everything that we have: programming languages, frameworks, principles of software development in teams, agile/clean code/TDD/DRY and other debatable or well accepted practices — all this exists to overcome limitations of human mind. AI does not have them and have others.

What I found to be useful for complex tasks is to use it as a tool to explore that highly-dimensional space that lies behind the task being solved. It rarely can be described as giving a prompt and coming back for a result. For me it's usually about having winding conversations, writing lists of invariants and partial designs and feeding them back in a loop. Hallucinations and mistakes become a signal that shows whether my understanding of the problem does or does not fit.

nirui2• 5 months ago

I'm feeling people are using AI in the wrong way.

Current LLM is best used to generate a string of text that's most statically likely to form a sentence together, so from user's perspective, it's most useful as an alternative to manual search engine to allow user to find quick answers to a simple question, such as "how much soda is needed for baking X unit of Y bread", or "how to print 'Hello World' in a 10 times in a loop in X programming language". Beyond this use case, the result can be unreliable, and this is something to be expected.

Sure, it can also generate long code and even an entire fine-looking project, but it generates it by following a statistical template, that's it.

That's why "the easy part" is easy because the easy problem you try to solve is likely already been solved by someone else on GitHub, so the template is already there. But the hard, domain-specific problem, is less likely to have a publicly-available solution.

lijok• 5 months ago

I think you severely overestimate your understanding of how these systems work. We’ve been beating the dead horse of “next character approximation” for the last 5 years in these comments. Global maxima would have been reached long ago if that’s all there was to it.

Play around with some frontier models, you’ll be pleasantly surprised.

Draiken• 5 months ago

Did I miss a fundamental shift in how LLMs work?

Until they change that fundamental piece, they are literally that: programs that use math to determine the most likely next token.

aembleton• 5 months ago

Which is great because then I can use my domain expertise to add value, rather than writing REST boilerplate code.

IsTom• 5 months ago

Having to write boilerplate code is a sign that libraries are just not up to the level they should be. That can be solved the regular old way.

josefrichter3• 5 months ago

Come on, this shows fundamental lack of understanding and experience on your side.

Kon5ole• 5 months ago

I'm feeling people are using AI in the wrong way.

I think people struggle to comprehend the mechanisms that lets them talk to computers as if they were human. So far in computing, we have always been able to trace the red string back to the origin, deterministically.

LLM's break that, and we, especially us programmers, have a hard time with it. We want to say "it's just statistics", but there is no intuitive way to jump from "it's statistics" to what we are doing with LLM's in coding now.

That's why "the easy part" is easy because the easy problem you try to solve is likely already been solved by someone else on GitHub, so the template is already there.

I think the idea that LLM's "just copy" is a misunderstanding. The training data is atomized, and the combination of the atoms can be as unique from a LLM as from a human.

In 2026 there is no doubt LLM's can generate new unique code by any definition that matters. Saying LLM's "just copy" is as true as saying any human writer just copies words already written by others. Strictly speaking true, but also irrelevant.

jama21112• 5 months ago

Well said. It also causes a lot of bitterness among engineers too, not being able to follow the red string is maddening to some. This rage can prevent them from finding good prompting strategies also which would directly ease a lot of the pain, in a similar way to how it’s far harder to teach my mother how to do something on her phone if she’s already frustrated with it.

Eliezer2• 5 months ago

Every time somebody writes an article like this without any dates and without saying which model they used, my guess is that they've simply failed to internalize the idea that "AI" is a moving target; nor understood that they saw a capability level from a fleeting moment of time, rather than an Eternal Verity about the Forever Limits of AI.

josefrichter3• 5 months ago

Exactly. Basically their thought are invalidated often quicker than they hit the publish button.

iLoveOncall• 5 months ago

Funnily enough we have had those comments with every single model release saying "Oh yeah I agree Claude 3 was not good but now with Claude 3.5 I can vibe-code anything".

Rinse and repeat with every model since.

There also ARE intrinsic limits to LLMs, I'm not sure why you deny them?

Eliezer2• 5 months ago

There's intrinsic limits to vanilla transformer stacks. Nobody knows where they are. We don't know how unvanilla Opus 4.6 or GPT 5.3 are. We don't know what's in development or which new ideas will pan out. But it will still probably be called an "LLM".

josefrichter3• 5 months ago

The OP’s example of AI writing 500 LOC, then deleting 400, and saying it didn’t… Last time I saw something like that was at least a year ago, or maybe form some weaker models. It seems to me the problem with articles like this is while they sometimes are true at the moment, they’re usually invalidated within weeks.

jbrooks84• 5 months ago

Vibe coding does not have a ceiling, you just need to break things up and the human needs to use their brain to orchestrate

ndr• 5 months ago

https://archive.ph/tUUMd as the site randomly 404s

jama21112• 5 months ago

This article has some serious usage of either bad prompting, or terrible models, or they’re referencing the past with their stories. I have experience AI’s deleting things they shouldn’t but not since like, the gpt4 days.

But that put aside, I don’t agree with the premise. It doesn’t make the hard parts harder, if you ACTUALLY spend half the time you’d have ORIGINALLY spent on the hard problem carefully building context and using smart prompting strategies. If you try and vibe code a hard problem in a one shot, you’re either gonna have a bad time straight away or you’re gonna have a bad time after you try and do subsequent prompting on the first codebase it spits out.

People are terrible observers of time. If you would’ve taken a week to build something, they try with AI for 2 hours and end up with a mess and claim either it’s not saving them any time or it’s making them code so bad it loses them time in the long run.

If instead they spent 8 hours slowly prompting bit by bit with loads of very specific requirements, technical specifications on exactly the code architecture it should follow with examples, build very slowly feature by feature, make it write tests and carefully add your own tests, observe it from the ground up and build a SOLID foundation, and spend day 2 slowly refining details and building features ONE BY ONE, you’d have the whole thing done in 2 days, and it’d be excellent quality.

But barely anyone does it this way. They vibe code it and complain that after 3 non specific prompts the ai wasn’t magically perfect.

After all these years of engineers complaining that their product manager or their boss is an idiot because they gave vague instructions and demanded it wasn’t perfect when they didn’t provide enough info, you’d think they’d be better at it given the chance. But no, in my experience coaching prompting, engineers are TERRIBLE at this. Even simple questions like “if I sent this prompt to you as an engineer, would you be able to do it based on the info here?” are things they don’t ask themselves.

Next time you use ai, imagine being the ai. Imagine trying to deliver the work based on the info you’ve been given. Imagine a boss that stamped their foot if it wasn’t perfect first try. Then, stop writing bad prompts.

Hard problems are easier with ai, if you treat hard problems with the respect they deserve. Almost no one does.

/rant

moebrowne• 5 months ago

garbage in, garbage out

jama21112• 5 months ago

Indeed, a good summary.

burnerToBetOut2• 5 months ago

    > …I have experience AI’s deleting
    > things they shouldn’t but not since
    > like, the gpt4 days.…

One blogger posted this [1] only yesterday about what Anthropic's latest and greatest did…

———

…I pointed Opus 4.6 at a 60K line Go microservice I had vibe coded over the past few months, gave it some refactoring principles, and let it run unsupervised…

…

What went wrong #

At some point in the code, we re-fetch some database records immediately before doing a write to avoid updating from stale data. It decided those calls were unnecessary and _removed them_…

———

[1] https://g2ww.short.gy/ClaudesLaw

jama21112• 5 months ago

Running it entirely unsupervised with “some refactoring principles” is the exact recipe for such a disaster, which supports my point.

That said, this is a very different kind of mistake to make compared to overwriting a file and then insisting it didn’t do that. Any modern model with reasoning would check the git history immediately if you mentioned this had happened, if it had somehow even made the mistake in the first place, but I digress.

RataNova3• 5 months ago

Speedups without changes in expectations just reset the baseline, and then you're sprinting forever

ta20240528• 5 months ago

I'll vibe code when vibe-coding can make a frontier LLM from scratch.

Meta-circularity is the real test.

After all, I can make new humans :)

theredbeard• 5 months ago

Skipping the investigation phase to jump straight to solutions has killed projects for decades. Requirements docs nobody reads, analysis nobody does, straight to coding because that feels like progress. AI makes this pattern incredibly attractive: you get something that looks like a solution in seconds. Why spend hours understanding the problem when you can have code right now?

The article's point about AI code being "someone else's code" hits different when you realize neither of you built the context. I've been measuring what actually happens inside AI coding sessions; over 60% of what the model sees is file contents and command output, stuff you never look at. Nobody did the work of understanding by building / designing it. You're reviewing code that nobody understood while writing it, and the model is doing the same.

This is why the evaluation problem is so problematic. You skipped building context to save time, but now you need that context to know if the output is any good. The investigation you didn't do upfront is exactly what you need to review the AI's work.

koliber• 5 months ago

The article is gone, but going off the title here...

If the easy stuff takes up 90% of the time, and the hard stuff 10%, then AI can be helpful. Personally, I can do "the easy stuff" with AI about 3-5x faster. So now I have a lot more free time for the hard stuff.

I don't let the AI near the hard stuff as it often gets confused and I don't save much time. I might still use it as a thought partner, but don't give it access to make changes.

Example: this morning I combined two codebases into one. I wrote both of them and had a good understanding of how everything worked. I had an opinion about some things I wanted to change while combining the two projects. I also had a strong opinion about how I wanted the two projects to interact with each other. I think it would have taken me about 2 workdays to get this done. Instead, with AI tooling, I got it done in 3 or so hours. I fired up another LLM to do the code review, and it found some stuff both I and the other LLM missed. This was valuable as a person developing things solo.

It freed up time for me to post on HN. :)

causal• 5 months ago

Helpful, absolutely, but only if you're solving the right problem. Solving the wrong problem with AI is doubly harmful because it will almost always give you something that runs, but now you are on a path that takes a lot of willpower to give up.

adamtaylor_13• 5 months ago

At this point, I don’t even know what to make of blog posts like this.

The very first example of deleting 400+ lines from a test file. Sure, I've seen those types of mistakes from time-to-time but the vast majority of my experience is so far different from that, I don’t even know what to make of it.

I’m sure some people have that experience some of the time, but… that’s just not been my experience at all.

Source: Use AI across 7+ unrelated codebases daily for both personal and professional work.

No, it’s not a panacea, but we’re at the stage that when I find myself arguing with AI about whether a file existed; I’m usually wrong.

h4kunamata3• 5 months ago

I know Ansible, homelab, Proxmox is my hobby, Debian is my gem.

I asked ChatGPT to guide how to install qBittorrent, Radarr (movies), Sonarr(TV Series), Jackett(credentials/login) without exposing my home IP and have a solid home cinema using private tracker only.

Everything had to be automated via Ansible using Proxmox "pct" CLI command, no copy and paste.

Everything had to run from a single Proxmox Debian container aka LXC

Everything network related had to use WireGuard via Proton VPN, if the VPN goes down, the container has zero network access, everything must be kill.

Everything had to be automated, download is finished, format the files structure for Jellyfin accordingly, Jellyfin add the new movies, TV shows.

It took me 3 nights to get everything up and running.

Many Ansible examples were either wrong or didn't follow what I asked to the letter, I had to fix it. I am not a network expert and hate Iptables haha, you need to know the basic of firewall to understand what the ACLs are doing to understand when it does not work. Then Proxmox folder mapping and you name it.

It would have taken me ages reading docs after docs to get things working, the "Arr services" is a black hole.

For this example, it made the harder part easier, I was not just copy/paste, it was providing the information I didn't know instead of me having to "Google for it".

I know the core of where things are running on, and here is where we have Engineers A and Engineers Z

Engineers A: I know what I am doing, I am using AI to make the boring part easier so I can have fun elsewhere

Engineers Z: I have no idea of what I am doing, I will just ask ChatGPT and we are done: 90-95% of engineers worldwide.

CHB0403085482• 5 months ago

AI is an intern that thinks he's hot stuff:

https://www.youtube.com/watch?v=TiwADS600Jc

blirio• 5 months ago

The article describes the problems of using a AI chat app without setting up context, skills, MCP, etc

Like yea the AI won’t know what you discussed in last weeks meeting by default. But if you do auto transcribe to your meetings (even in person just open zoom on one persons laptop), save them to a shared place and have everyone make this accessible in their LLM’s context then it will know.

kajolshah_bt• 5 months ago

That description matches a lot of what we’ve seen in real products. AI does make some parts of development and workflows easier like summarizing data, generating initial drafts, or auto-completing repetitive patterns. Those wins are real.

The hard part that becomes harder is not the technology. It’s the decision-making around it. When teams rush to integrate a model into core workflows without measuring outcomes or understanding user behavior, they end up with unpredictable results. For instance, we built an AI feature that looked great in demo, but in real usage it created confusion because users didn’t trust the auto-generated responses. The easy part (building it) was straightforward, but the hard part (framing it in a way people trusted and adopted) was surprisingly tough.

In real systems, success with AI comes not from the model itself, but from clear boundaries, human checkpoints, and real measurements of value over time.

AI makes the easy part easier and the hard part harder

Related Stories