A guide to Gen AI / LLM vibecoding for expert programmers

https://www.stochasticlifestyle.com/a-guide-to-gen-ai-llm-vibecoding-for-expert-programmers/

128 117 11 months ago

iLoveOncall4• 11 months ago

This illusion that senior engineers are too good to fall in all the traps laid by vibe coding really needs to stop.

Just yesterday I was reading the comment of a Principal Engineer saying "of course vibe coding carries a huge risk for more junior engineers that don't understand when the LLM does something that looks right but is actually wrong", as if just because they have more experience THEY CAN.

No, you can't either. Not a single soul on this planet can review the thousands of lines of vibe coded bullshit that LLMs spit out.

Here's my guide to Gen AI / LLM Vibecoding for Expert Programmers: don't.

jandrese• 11 months ago

I think that's going a bit too far. The big thing is to have the AI spit out small digestible modules, check those for correctness, and then glue them together. The same way a person normally writes code, you are just having the AI do the grunt work.

This does have the caveat that reading code is usually harder than writing it, so the total time savings is far less than what AI companies claim. You only get in real danger when you don't review the code and just YOLO it into production.

shortrounddev22• 11 months ago

If you have to describe the code to the ai and then read through each line of it anyways, why not just write the code yourself?

apwell23• 11 months ago

exactly. using ai is not a "skill" that ppl are trying to make it to be.

ivape2• 11 months ago

You underestimate how much code some people have sifted through in their lifetime. I read through every line the LLM codes, function by function.

I know exactly when it’s going sideways. You’d have to know to even know, it’s one of those things. If you don’t know then you don’t know.

Put it another way, you know how some people fail open book tests? How’s that possible? The book is not enough.

exitb• 11 months ago

I found that once I spend enough time to actually fully understand what LLM wrote, I’ve burned through my efficiency gains. If that’s the case, why bother?

abletonlive• 11 months ago

You sound like a cynical junior engineer. As a tech lead I have to review thousands of lines of code from human engineers and let me tell you it’s no different than reviewing llm code

ajaioslaja• 11 months ago

As a senior engineer, I can say I probably wouldn’t like working with you.

For a dev worth paying for, the crucial details of some code is discussed prior to the PR in a way that the PR review becomes an afterthought. You can follow this process with an LLM but the PR review is still brutal. It doesn’t do what you say and it doesn’t learn (in a deterministic way a good human dev does).

High performing teams do not have many changes nor comments in a PR (on average, obviously).

iLoveOncall4• 11 months ago

As a tech lead I have reviewed code written by junior engineers and written by AI, and there is a very clear difference between the two.

You also seem to be missing the point that if vibe coding lets your engineers write 10x the amount of code they previously could in the same working hours, you now have to review 10x that amount.

It's easy to see how there is an instant bottleneck here...

Or maybe you're saying that the same amount of code is written when vibe-coding than when writing by hand, and if that's the case then obviously there's absolutely no reason to vibe-code.

EagnaIonat2• 11 months ago

Not a single soul on this planet can review the thousands of lines of vibe coded bullshit that LLMs spit out.

Even if you could the code can't be copyrighted.

simonw6• 11 months ago

Citation needed on that one.

If that is true, how come huge companies like Microsoft and Salesforce and Google keep boasting about the increasing percentage of their code that is written by LLMs?

What do you know that they don't?

ebiester3• 11 months ago

So, why would you review thousands of lines at a time? That means you didn't break the problem down appropriately. PRs should be around 100-200 lines of code, including tests. You build your tasks appropriately.

Reviewing thousands of lines at a time is always a failure state.

iLoveOncall4• 11 months ago

The number of lines you have to review at once doesn't matter.

Vibe coding is letting AI write code so you spend less time writing the same amount of code (ideally not more, in practice definitely more).

If more code is written you have to review more code. Doesn't matter if you break it down in 10 lines PRs or if you review a million lines at once, you still end up having to review all the code generated.

alfalfasprout4• 11 months ago

PRs are as long as they need to be and as short as they need to be. This idea that any problem can be decomposed into 100-200 line changes is ridiculous. That's not realistic in many cases (esp. for refactoring work, etc.)

rychco• 11 months ago

as if just because they have more experience THEY CAN

I don't follow. Why wouldn't you believe that a senior engineer, that has been reviewing code for years, be any better at reviewing code?

ChrisRackauckas9• 11 months ago

Notice that the examples I show are very minimal changes. If you prompt it to solve the right problems which involves minimal code, then the diffs are easy to manage. There are lots of problems like this. Leave the hard ones for yourself. The only big diff it should ever make is for a refactor.

throwanem• 11 months ago

Vibe coding is useful only if you have enough problems that you’re happy that some subset [is] being solved, not caring what in that subset is solved.

ppqqrr4• 11 months ago

generic advice about how you should use the same tool and methodology (claude code, scrum) that everyone’s already using, lot of hand waving about being “senior,” This may be the most 2025 blog post ever written.

if anyone takes the art of software programming further using LLMs, it’s going to be young inexperienced people who grow up closely observing and learning the transcendental nature of LLMs and software, not hardened industry titans joyfully cracking their whip over an army of junior devs and LLMs.

rafterydj2• 11 months ago

Not to be rude, but what about understanding the "transcendental nature" of LLMs allows people to build more, faster, or with better maintainability than a "hardened industry titan"?

Retr0id• 11 months ago

New generations are always leapfrogging those that came before them, so I don't find it too hard to believe even under more pessimistic opinions of LLM usefulness.

They are young and inexperienced today, but won't stay that way for long. Learning new paradigms while your brain is still plastic is an advantage, and none of us can go back in time.

ppqqrr4• 11 months ago

we’ve been taught to think of programs as sculptures, shaped by the creator to a fixed purpose. with LLMs, the greatest advance isn’t in being able to make larger and more detailed sculptures more quickly; it’s that you can make the sculptures alive.

alfalfasprout4• 11 months ago

generic advice about how you should use the same tool and methodology (claude code, scrum) that everyone’s already using, lot of hand waving about being “senior,” This may be the most 2025 blog post ever written.

This part tracks. It's honestly rather generic.

if anyone takes the art of software programming further using LLMs, it’s going to be young inexperienced people who grow up closely observing and learning the transcendental nature of LLMs and software, not hardened industry titans joyfully cracking their whip over an army of junior devs and LLMs.

This, I'm not sure applies either. TBH the biggest risk I'm seeing already right now is how quickly we're starting to see juniors trying to enter the job market who don't have the faintest idea how actually code. Let alone build software (but let's be honest, that's usually been the case). What decisions to make when writing something are based on factors outside just implementing the functionality: how maintainable is this? how extensible?

Giving a junior a sycophant that's reasonably competent at spitting out something functional (not necessarily functional in the sense you need it to be but apparently working) is a recipe for disaster IMO.

There will absolutely be some who "get it". But, how does that scale?

More worryingly, something I don't see discussed enough is "review fatigue". It's far more fatiguing reviewing the output of an LLM than writing the code yourself a lot of the times. Early on in your career, this might lead to the tendency to just say "eh, looks alright enough".

ChrisRackauckas9• 11 months ago

More worryingly, something I don't see discussed enough is "review fatigue". It's far more fatiguing reviewing the output of an LLM than writing the code yourself a lot of the times. Early on in your career, this might lead to the tendency to just say "eh, looks alright enough".

I am definitely worried about that. I got into a mode of doing morning code review every morning due to having a bunch of active open source repos, so over time as they grow in activity I learned to adapt my schedule, mind set, and everything around how to do the right amount of code review (when to finish the PR yourself, when to give pointers to train a new contributor, when it's not worth your time yet to say more than "tests fail", etc.). I started for years pulling every piece of code locally, running it, figuring out why someone would make a change like that, etc. and then it took like 5 years before I could actually start to do code review well just by reading a git diff. I needed to know the domain, the language, the problem space, and how people tend to work, all well enough to make that simple enough to have a mental model.

Then, I tend to see some junior programmers put up LLM stuff and I go "oh just handle the extra memory allocations and it's good to go" and they go "wait handle what where?". If you can't eyeball that in under a minute of scrolling through the code... then you shouldn't be doing code review by scrolling through it yet. But if they keep doing it like this, they will never build that skill.

That's why I think the right LLM usage is really for those who have already built that code review skill. It's not to gate the usage of LLMs, but people will never learn if they aren't strongly engaging with the code at some level.

mccoyb• 11 months ago

Try and use it for shit and play around with it — you’ll figure out where it sucks, where it is good by yourself.

Play with it — that’s the only way anyone masters anything.

Separate yourself from the results, be prepared to waste time, but try and have some fun and keep your eyes open.

simonw6• 11 months ago

Couldn't agree more. The only way to really get effective at using this stuff is to spend time with it, building up intuition about what it can and cannot do.

Playing with it is a great way to do that.

If there is something you are certain it won't be able to do, it's worth giving it a go anyway. Most of the time you will be right - and will still learn something new in confirming that and watching the way in which it fails. Very occasionally you'll be wrong and you will discover a new capability that surprises you.

GuB-42• 11 months ago

I think the article could be more accurately titled "A Guide to Gen AI / LLM Vibecoding for Programmers who hate their job"

To me, someone who actually love programming, it makes vibe coding look like hell.

The workflow of vibe coding is the same as managing scrums or helping a student through a research thesis

Which programmer wants that?! Just hearing the word "scrum" makes me want to run away and I know I am not alone here. Helping a student through a research thesis doesn't sound so bad, because as a human, helping people feels good. But here, there is no student, no human feelings, and you are not even helping the AI become better in the long term.

uncircle3• 11 months ago

I think the article could be more accurately titled "A Guide to Gen AI / LLM Vibecoding for Programmers who hate their job"

Given how “vibe coding” is all about explaining clearly the requirements and defining context, it’s for programmers who should have chosen middle management as a career.

To actual programmers that enjoy the craft, using an LLM means ruining the beautiful art of abstraction and mental visualisation of a complex piece of logic, by putting it through the very lossy process of explaining it into words. It would be a bit like a composer or a painter having to use words instead of leveraging their well-honed instinct.

jf22• 11 months ago

I've been middle management for half my career and the role has never been about explaining or requirements or defining context like I do with an LLM to code...

starfallg• 11 months ago

The reality is that hacking code isn't always beautiful. Most of the time, it is mundane grunt work.

You can always leave the core logic for your to work on and have the AI handle all the bits that you don't like to do. This is what we do for modelling for example, AI helps with the interface and data backends, the core modelling logic is hand-crafted.

CuriouslyC• 11 months ago

I choose the tech stack and architect the project.

I choose the language patterns and code organization.

I step in to solve hard problems when agents flounder.

What about that says middle management? It's just getting rid of all the low iq parts of the job.

JohnMakin• 11 months ago

almost every job ive ever had has been to build stuff. frequently programming is used to build stuff. programming is not my job, building stuff is. it’s perfectly normal (and a sign of a more mature engineer, imho) to prefer building stuff to the annoying stuff that gets in the way of building stuff, especially since many languages are obnoxious to work with.

polishdude205• 11 months ago

I agree. I'm hired to build stuff and make it work. I get satisfaction from building stuff people want and use. If I can use an LLM to help me focus more on what the user wants in all for it.

I feel good because real humans are using what I've built and they like it.

Sharlin• 11 months ago

Yes; in particular English is an incredibly obnoxious language to try to build software with. (Replace English with any natural language if you want.)

kiitos• 11 months ago

building stuff is the application of programming, not the platonic form of it

if you see programming as a necessary but ultimately annoying means to an end, that's fine, you do you, but there are many other folks who don't look at it that way, and they're no more or less right or wrong than you are

recursive• 11 months ago

I think I enjoy programming. Vibe coding removes most of the parts that I like. It already looks like hell. I'm probably a minority, but I don't think I'm alone in this.

afro882• 11 months ago

Do you enjoy the work you had to put in for every single PR? I'm not trying to make a "surely there's 1" annoying argument, but a "surely there's 5-10%".

For me, that's:

- working in legacy parts of the codebase

- anything that requires boilerplate that can't be code genned

- actually writing the code of unit tests (the fun part is making code testable, and coming up with what to test)

- fixing lint issues that can't be auto fixed yet

- removing old feature toggles

- building a temporary test harness

The list goes on. That's not hell. That's getting someone else on the team to do all the stuff you don't enjoy, without ruining someones day.

ako• 11 months ago

I really like creating software solutions, vibe coding removes the part that is most tedious. LLMs allow me to experiment with different solutions and different designs faster.

lubujackson2• 11 months ago

I like coding. I've been doing it for a couple decades. I disagree with the "managing scrum" metaphor. Sure, you can use LLMs that way. And there is some truth to the fact that it may feel more like writing detailed Jira tickets than actually programming at times if you are trying to have it make huge changes... BUT coding with LLMs is really just a higher level abstraction. And the good news of that is LLMs are more deterministic than they seem, which is a lot of what people are fearful of losing by giving LLMs "the reins".

One nice thing about programming is that the computer is a dumb funnel for your human brain to encoded actions. If the program doesn't work, you did something wrong, missed a semicolon etc. This still applies to LLMs. If the LLM gave you shit output, it is your fault. You gave it shit input. If the model you used was the wrong one, you can still get good results, you just have to put in more work or break the problem down more first. But it's still programming. If you treat using LLMs that way, and start to apply your normal programming approaches to your prompts, you can find a workflow that satisfies your demands.

But even if you only use LLMs to rubber duck a problem, by literally typing "The program is doing X,Y,Z but it's only supposed to do Z. I already looked at A,B,C and I have a theory that G is causing the issue but I'm not seeing anything wrong there." and just paste that in the chat you might be surprised what it can turn up. And that's a fine use case!

LLMs are broadly useful and there are certainly elements of programming that are the "shit-shoveling" parts for you, from debugging to writing tests to planning or even re-writing Jira tickets, LLMs can help at different levels and in different ways. I think prescriptive calls to "code this way with LLMs" are shortsighted. If you are determined to write each line yourself, go for it. But like refusing to learn IDE shortcuts or use a new tool or language, you are simply cutting yourself off from technological progress for short term comfort.

The best part of programming to me is that it is always changing and growing and you are always learning. Forget the "AI eating the world" nonsense and treat LLMs as just another tool in your toolkit.

cheema332• 11 months ago

If someone enjoys building houses with a hammer and nail, they still can. Other like using tools.

You will always be able to write code by hand. But you will not be able to keep up with other engineers who master using AI tools. I am a software engineer, I like building things. And I don't use binary or assembly, or C or any other lower level languages for good reasons. AI is just another higher level abstraction. A tool that allows me to build things a little faster. I use it every day.

wsintra20222• 11 months ago

IDE’s are tools, languages are tools, LLMs are code generators, not quite the same thing.

roadside_picnic• 11 months ago

Which programmer wants that?!

Agreed.

My LLM usage has quickly become a more efficient way to solve problems that basically require copy/pasting from some documentation I have to look up where me doing it myself is more error prone.

I was recently doing a fairly complex set of data transformations and understanding what the data means remained essential. AI tends to fail spectacularly at really understanding the nuances of data (that often requires understanding the underlying business logic generating the data).

However it's very useful when I have to do a bunch of window functions, where I can describe the operation but don't want to look up the syntax. Or just writing SQL when I can explain exactly what I need it to do.

Similarly working with Pytorch involves a fair bit of what I consider pseudo-boilerplate where the code itself is quite boring, but contains just enough complexity that it can't be simply automated away. Hand rolling this stuff often leads to small errors/bugs, but LLMs can do a spectacular job of quickly generating this, provided you do know exactly what you're looking to build.

What's interesting is that this has still been a major speed boost, because looking up how to do some tedious thing you just forgot how to do can really break flow.

ChrisRackauckas9• 11 months ago

Hey, definitely agree in many respects! I have been programming for over a decade because I love to. And the worst part the job has always been giving feedback to a student/engineer who clearly just doesn't care and makes the same mistake again, like why should I keep reading your code instead of working on the fun hard problem I've been wanting to?

But, what I found over time is that as my role grew, the amount of work that could be done grew exponentially. In grad school I had one project to work on, by postdoc there was 10 papers that could be juggled, now there's so many different projects/collaborations/contracts/etc. that I could be a part of that it's all just a matter of "do I have time for that?". Then the team sizes start to grow: having one student turns into 5, into a lab, into a team at one company, then a growing team at another, and an open source organization. In one decade life changed from "wake up, figure out what to program, make it" to "wake up, see the deluge of issues and PRs, start code review, try to get that done so I can start 'my' day", and I think most people in programming tend to have a trajectory like that. The role changes from "build something" to "find out where others are getting stuck and unblock them", plus 200+ issues files about trivial shit. The goal here is to to have the LLMs handle the trivial shit, I unblock someone, and then finally the glory hour of me getting to build something is back!

But I think this is why there are lots of very smart people who have decided they wanted to be individual contributors in Silicon Valley instead. And they are really well-respected, and you want them in your squad! If that's the direction you want to go, then you probably also are of the kind that avoids LLMs a bit (and are thus a good person to have on a team, balancing velocity people with precision people). But if you want to start driving bigger projects, sooner or later teams grow as that's the only way to put out the bigger grander idea.

But I do think that if Anthropic made a Claude stuffed animal with a very punchable face it would be a very hot selling add-on as while this can accelerate the day, sometimes it makes the stupidest decisions and you really have to think "okay it doesn't hate me, it's not trying to spite me, it doesn't even know me, but damn it tried to hide that it commented out the one most crucial test!"

bobsmooth• 11 months ago

Trying to get better at vibe coding is like trying to get better at sleeping. The whole point is not to put effort into it.

leptons3• 11 months ago

Not a good analogy. You can put effort into getting better sleep. If the sunlight wakes you up too early, block out the windows. If your bed is uncomfortable, get a more comfortable bed. If the room is too hot, adjust the thermostat before you go to bed. A little bit of effort can go a long way towards getting better sleep.

But you can't change the fundamental operation of an LLM, by its very nature it is not suitable for producing consistently correct results. Getting better at "vibe coding" is an exercise in futility.

simonw6• 11 months ago

I completely disagree, for both definitions of vibe coding (the unreviewed-prompt-generated-code definition and the llm-assisted-coding definition.)

In both cases the more experience and mindful experimentation you have the better your results will be.

cheema332• 11 months ago

The whole point is not to put effort into it.

You sound like you have not spent much time using AI tools.

jackblemming• 11 months ago

I always wonder what kind of slop people are creating when they say they have a bajillion Claude instances all churning out code.

alfalfasprout4• 11 months ago

Impressive sounding slop that either rarely amounts to anything, breaks, or becomes someone else's problem. Unfortunately I'm starting to see these people at work...

What's worse is they also tend to be the type of "evangelists" that become wildly defensive or accusatory when you question anything about their work.

kasey_junk• 11 months ago

In my experience you would not be able to identify code I wrote by hand from code I generated with an asynchronous agent, because I wouldn’t let it get to you if something stood out as off.

But I don’t “vibe code” anything. I understand and review all the code that gets generated. Most of the stuff coming from my agents is either boilerplate or extensively uses libraries and static analysis tools that make it easy to verify.

I kill aggressively any code the agent outputs that doesn’t match my standards unless it’s obvious that I can get it up to par quickly. This is one of the big advantages of the llm. I can do that without navigating inter personal conflict.

But the output is mostly indistinguishable from what I create by hand.

chermi• 11 months ago

It's very easy to check out his GitHub, no need to wonder. Hint: it's not slop

ChrisRackauckas9• 11 months ago

I'm sure there's lots of slop being generated by this stuff, but at least for my work it's mostly solving the simple issues. Maybe it's the slop of "hey can you find out why my code isn't working?" and "closed issue 268 because it works on the latest versions, and we made your code into an MWE for the future". If you check any open source project that gets traction, there's tens of thousands of issues to handle. A good thousand of them are just "unit conversion would be good here", "this docstring is missing this option", etc. Those little PRs are the slop of Github that someone has to do, and I'd prefer not telling my interns to do that because otherwise they'd just quit. Good thing Claude doesn't quit, he's a good boy.

redeyedtreefrog2• 11 months ago

It's hardly original, but my take is that vibe coding works brilliantly for personal projects (or potentially for tiny startups that need to rapidly churn out CRUD boilerplate, API integrations, etc), but terribly for most large commercial systems.

I'm having fun with Claude Code and Vibe Kanban on personal projects, and before that I spent a lot of time with both the Windsurf and Cursor agents. It's making me literally 10x more productive on personal projects, maybe even 50x.

On personal projects:

- no-one but me needs to decide on requirements

- no-one but me needs to make decisions

- no-one but me needs to maintain the code going forward

- much of the time I'm intentionally using languages and frameworks that I am somewhat clueless about, and an LLM providing continuous ideas (even if sometimes entirely silly ones) stops me getting stuck

- I don't mind if there are large chunks of useless or half-working code

On commercial projects:

- every line of code is a massive liability. Every line needs to be reviewed by another engineer, and every developer who joins the project needs to be aware of it, take it into consideration when making changes elsewhere, and potentially debug it if something goes wrong

- senior engineers are almost always hired to work with languages and technologies they are already very familiar with, meaning for many tasks it's often quicker to write out the code by hand (or perhaps with Cursor's auto-complete) than guide an LLM to do it

- much of the time is spent in meetings trying to unearth the real product requirements or providing updates to stakeholders

- much of the time is spent reading old code and working out how to implement things in extremely large and complex systems in a minimally disruptive way

- a lot of time is spent reviewing other people's PRs, and getting infuriated when people (often either very junior or very senior) produce 1000 line PRs consisting of unnecessary changes, excessive boilerplate, half-finished experiments, and things that clearly haven't been tested properly. This was the case long before LLMs, AI just makes it ever more tempting for people to act this way.

- trying to avoid or gently negotiate political games over who is in control of the project, or who gets to makes technical decisions

fragmede3• 11 months ago

if we have to put numbers on it, I'm attempting projects I wouldn't have, because the startup cost was too high, so does that make me infinity times more productive on that project because the denominator otherwise would have been zero?

redeyedtreefrog2• 11 months ago

Same here, my numbers come from "how much progress do I make with a single day spent working on a given project".

Previously: make a little bit of progress, realize I'd need a year's worth of weekends to complete it, give up after one day.

With vibe-coding assistance: make 10x+ progress, realize that with a few more weekends I could finish it, keep going.

andai• 11 months ago

For the record, I now have about 32 Claude agents continuously running in tmux windows that I can ssh to, so all day long I can just check via laptop or phone and keep plugging along.

What's the cost of 32Cph?

ChrisRackauckas9• 11 months ago

The same $200/month Max plan as anyone else. It just gets rate limited sometimes but who cares, you just go back to the fun part of doing your own coding until it's back.

seblon• 11 months ago

How the hell he's using so much argents at the same time. Is it for 32 different "issues" like feature 1, 2, 3, ...?

simonw6• 11 months ago

This piece is excellent - it's full of ideas and tips that I have not seen in other similar tutorials. Worth spending some time with, especially if you are still skeptical of the value that can be unlocked by AI-assisted programming.

(A minor disagreement: it's using a definition of "vibe coding" that applies to any form of ai-assisted programming. I prefer to define vibe coding with its original definition from all the way back in February where it only refers to code that is generated through prompting without any review.)

isaacremuant• 11 months ago

I use LLMs, just like i use Google and other things but I don't vibe code as defined like the original meaning and I feel that all these people who are trying to tell me how good they are with it are yet to explain how on earth code is the most complex thing.

Coding is the easy part. Knowing what to code, how to work with people, how to interact with the real world and with chaos, with bad data, with poor integrations, and entropy.

I'm all for making AI and coding better and helping people do that and think.

I'm not impressed by anyone who doesn't show me they can think about the problems we face better and faster, and keep doing it without slowing down to a halt or making it someone's else's problem.

Everyone else is just talking themselves up or selling something, neither of which are useful to me.

renodino• 11 months ago

My recent vibe code experience made me realize it's almost exactly like being a tech lead managing offshore development. I learned early on that the key to leading successful offshore projects is precise, detailed specifications, and very rigorous code review and testing. Now I'm using the exact same discipline to "vibe" code. I really think there needs to be a different term for professional solution engineering using LLMs (like "prompt engineering" but for coding) to differentiate from casual prototyping or simple web UI hacking by non-devs that uses "vibe" coding.

ChrisRackauckas9• 11 months ago

I really think there needs to be a different term for professional solution engineering using LLMs (like "prompt engineering" but for coding) to differentiate from casual prototyping or simple web UI hacking by non-devs that uses "vibe" coding.

Yeah, maybe it needs another name because yes when you say you're vibe coding everyone imagines some 20 year old kid with sunglasses spitting out Javascript and calling it Python. While the tools are the same, the understanding and the discipline is very different.

I think "tech lead managing offshore development" is an excellent analogy.

afro882• 11 months ago

The moment you see it go off the rails, just throw it out. That problem is too hard for Claude, it’s for you now.

Or, any of:

- the problem was too big in scope and needed a stepped plan to refer to and execute step by step

- your instructions weren't clear enough

- the context you provided was missing something crucial it couldn't find agentically, or build knowledge of (in which case, document that part of the codebase first)

- your rules/AGENTS.md/CLAUDE.md needs some additions or tweaking

- you may need a more powerful model to plan implementation first

Just throwing away and moving on is often the wrong choice and you'll get better at using these tools slower. If you're still within the "time it would have taken me to do it myself" window, think about what caused it to go off the rails or fail spectacularly and try giving it another go (not following up, throw away current results and chat and try again with the above in mind)

polishdude205• 11 months ago

I totally agree. I've recently been using Cursor to help create a new react project with a lot of functionality. I realized I need to have it do more smaller steps that I get to have lots of input on rather than say "this is the big picture now go forth".

jpollock3• 11 months ago

I've just spent the better part of two weeks trying to convince a LLM to automate some programming for me.

We use feature flags. However, cleaning them up is something rarely done. It typically takes me ~3minutes to clean one up.

To clean up the flag:

1) delete the test where the flag is off

2) delete all the code setting the flag to on

3) anything getting the value of the flag is set to true

4) resolve all "true" expressions, cleaning up if's and now constant parameters.

5) prep a pull request and send it for review

This is all fully supported by the indexing and refactoring tooling in my IDE.

However, when I prompted the LLM with those steps (and examples), it failed. Over and over again. It would delete tests where the value was true, forget to resolve the expressions, and try to run grep/find across a ginormous codebase.

If this was an intern, I would only have to correct them once. I would correct the LLM, and then it would make a different mistake. It wouldn't follow the instructions, and it would use tools I told it to not use.

It took 5-10 minutes to make the change, and then would require me to spend a couple of minutes fixing things. It was at the point of not saving me any time.

I've got a TONNE of low-hanging fruit that I can't give to an intern, but could easily sick a tool as capable as an intern on. This was not that.

polishdude205• 11 months ago

Which LLM? How are you prompting it?

I've been using Cursor for the last few months and notice that for tasks like this, it helps to give examples of the code you're looking for, tell it more or less how the feature flags are implemented and also have it spit out a list of files it would modify first.

jpollock3• 11 months ago

I gave it explicit ordering, instructions on what tools to _not_ use, and before/after examples from the codebase. A full page of instructions.

After iterating on that for a while, I did a bunch manually (90) and then gave the LLM a list of pull requests as examples, and asked _it_ to write the prompt. It still failed.

Finally, I broke the problem up and started to ask it to generate tools to perform each step. It started to make progress - each execution gave me a new checkpoint so it wouldn't make new mistakes.

theLiminator• 11 months ago

Might make sense getting it to instead create a CST traversal that deletes feature flags by their id. Then you have a re-usable trustworthy tool that you can incrementally improve/verify.

jpollock3• 11 months ago

That was the lesson I was learning. I should use the LLM to generate the tools that I use for consistently repeatable tasks.

Then I can rinse and repeat using the tool, fixing the bugs in the tool myself instead of repeating the expensive (in time) cost of using the LLM.

That was my last attempt, but I ran out of time.

SparkyMcUnicorn• 11 months ago

If you have examples, can you provide git commit hashes that it can diff and use as a reference?

For repeating patterns I'll identify 1-3 commit hashes or PRs, reference them in a slash command, and keep the command up to date if/when edge cases occur.

bgwalter• 11 months ago

The author's 200 GitHub packages not withstanding, the startups he is involved in are both founded on machine learning, so I don't consider him unbiased.

It is always the founder type who is trying to peddle fantasies like this:

Vibe coding turns any individual into the CTO leading a team of 20, 30, 50, 60 interns.

How much does his new hobby cost?

On my $200/month Claude 20x Max subscription I used enough tokens for about $5,200 of compute in the first month. This is obviously not sustainable, but hey, it’s a startup world and VCs are paying for it right now.

wsintra20222• 11 months ago

Let’s just call it like it is, Vibe coding takes away the thinking of the execution of steps to solve the problem and offloads that part of our mind to a text generator with intelligent emergent properties. If you enjoy keeping the intelligence sharp for laying out the steps to solve the problem and executing the steps then don’t vibe code. If you don’t mind the engineering skills atrophy then go ahead and vibe code. But know this. You will lose the skills.

ChrisRackauckas9• 11 months ago

If you don’t mind the engineering skills atrophy then go ahead and vibe code. But know this. You will lose the skills.

I don't see why it's an either or. As explained in the article, I'm still coding all day. Just some trivial PRs are getting done and tidied up while I'm doing so. You still have to do all of the code review, and you still have to solve all of the hard problems, you just do less of the refactoring by hand.

polotics2• 11 months ago

This appears to me to be a surprisingly low-effort article. Nothing about managing context length and starting tasks from a good context, managing .md files and structuring repos, very anthropomorphising which does not help, nothing about tools to use to build the right context, no comparison of different approaches, what kind of MCP and/or RAG to get agents to look at documentation.

Ethee• 11 months ago

The article clearly isn't directed at how expert programmers are 'supposed' to use AI, though we see at least one of those a day on the front page of HN now. The first paragraph makes it pretty obvious that the author is trying to convince people who have completely disregarded AI, for one reason or another, to give it another try in a more meaningful way. Telling the reader how the author configures his AI isn't going to convince those people, as they likely are already very aware of how these tools 'work' the problem is in the 'how to use them' department which has nothing to do with what MCP tool you're using.

ChrisRackauckas9• 11 months ago

That's kind of the whole point. Setting up a few MCPs etc. is really a minor thing in the grand scheme of things. Sure, I have setup context7, Sequential Thinking, etc. and my Claude.md has a lot of details in it, but that only improves the accuracy so much. My whole point is that if the mindset is "I need this to be very accurate for it to work", then you're already putting too much effort into it. I personally have not found any of that time worth it. For any PR I had to put effort into, I could have done it quicker myself. So I learned those things and they just didn't have a payoff for my work. They did really well in some things like front end development, devops, and stuff of that sort, but if it's the normal kind of problem that is my "here's my actual hard problem of the day", i.e. something where I would need to start white boarding a new algorithm and semi-prove some numerical stability result before throwing code, the LLMs just fail to ever come up with anything new enough to get a solution.

But, slamming down commands for it on only easy problems in order to get 90% of a PR done and just finishing it yourself? That can get you 6 PRs in instead of 3, where you did the hard 3 and just looked at the transcripts for another 15, and tossed all but the 3 that looked good. Using Claude like that takes about half an hour out of your day and you get a good benefit, and that is what I am pointing to as a very useful approach.

I personally don't think any amount of MCP servers will make "Claude, find me a novel algorithmic improvement in this space and code it up" work, but hey if that works for you that's great. But reviewing and checking the proofs would likely make it not worth it for what I'm doing.

polotics2• 11 months ago

ok you're right, i got fooled by the title: "A Guide to Gen AI / LLM Vibecoding for Expert Programmers"

esoterae• 11 months ago

They did really well in some things like ... devops

Did they, though? I ask because the vast majority of people touting LLM as a huge win usually seem to speak from a.. DK-heavy perspective. As in, the bulk of their profession experience hasn't actually been doing the subject they're gushing about the LLM crushing.

So I ask you, what is your level of devops experience to make such a dismissive, sweeping statement? I am pointedly inquiring about my own area of expertise, in case that wasn't clear.

theusus• 11 months ago

This looks like rambling than a guide.

nphardon• 11 months ago

I think the idea that it's important to know how to code is going to be seriously challenged. I know I feel like the learning process and insight I gain is important, but I wonder if it is, beyond the subjective.

Like I'm sure the grad students working for Euler learned a ton generating logarithmic tables by hand, but it proved to be useless in the end. Could having a solid grasp on memory management/access in C be the same?

I think this is why obsolescence can be hard to predict.

Like if in 30 years all code is run and managed by ai bots, then all this debate about "it's important to know how to code!" will seem really silly.

mangamadaiyan• 11 months ago

Could having a solid grasp on memory management/access in C be the same?

Perhaps not when the code is getting written (by yourself or by an AI).

When you're dealing with a memory leak in production (yes, they can and do happen), or worse -- if you're dealing with a drop in performance -- good luck with the vibe debugging.

Maybe there will be a time not too far off in the future when the stochastic cockatoos become sentient enough to do all of this by themselves, it's sure as hell not happening tomorrow.

Oh, and logarithmic tables were still a useful thing as late as forty odd years ago :)

percentcer• 11 months ago

I like programming. It's fun, and sometimes it even requires creativity. There are plenty of times when it's not that fun, and doesn't require any creativity (because whatever it is has been done by someone else a thousand times over), and you just want a result. Vibecoding is great for _that_ stuff (think little throw-away scripts, shell one liners, tool plugins, etc)

I generally don't get great results from LLM code because most of my work is in C++ (which I'm guessing is underrepresented in the training data?), but when I point it towards some well-worn javascript thing I've had real successes! Most recent example is this little chrome plugin I had it whip up in one shot (https://chromewebstore.google.com/detail/favicon-tab-grouper...) because I couldn't find the exact functionality I needed in other plugins.

Works perfectly for my needs, took less than five minutes to spin up, and I use it all the time. If you're looking to get started with vibecoding stuff, try making plugins that provide niche functionality for your hyper-specific workflows.

dimtass• 11 months ago

After 25 years in embedded, embedded Linux, devops and cloud security I don’t care about coding anymore. I’ve done almost everything in so many programming languages and assembly that I can’t even count. Every programming task seems like a repetitive task. But architecture and solving a real problem wow, this still kicks in me. So, LLMs are my pals that we spend so much time in planning and go to every single small detail and in such depth that it’s hard to explain the mental erection. Just give it a prompt to try to contradict every step and make it hard for you and try to convince it that it’s wrong. I can spend hours just building the plan and the idea. Then I have the solution and it’s just boring to write the code. I know I can write the code and I’ve already done it before. So, I just create a full plan with specs and a water flow to do list and let it do the job. I check if it does it right and keep it on track not to over engineer things (they tend to do that a lot) and just enjoy the outcome. I would say I do vibe-planning before vibe-coding.

A guide to Gen AI / LLM vibecoding for expert programmers

Related Stories