luckilydiscrete

insert obama medal meme

https://www.faros.ai/blog/ai-software-engineering

The actual study with the data, minus the "I was right all along" commentary

It's a combination of being bad at prompting and different expectations from the tool. You expect it to be one shot, and then rewrite things that don't match up to what you want.

Instead I recommend that you use LLMs to fix the problems that they introduced as well, and over time you'll get better at figuring out the parts that the LLM will get confused by. My hunch is that you'll find your descriptions of what to implement were more vague than you thought, and as you iterate, you'll learn to be a lot more specific. Basically, you'll find that your taste was more subjective than you thought and you'll rid yourself of the expectation that the LLM magically understands your taste.

I'll say "maybe-sometimes works" is a misunderstanding.

It feels like that initially, but that's no different from any new tool you adopt. A jackhammer also "maybe-sometimes works" as a hammer replacement.

AI is indeed just hype in a lot of cases, but also has revolutionary value in a other cases. Trying it is the only way you'll be able to differentiate the latter from the former.

While true in a subset of problems, linters will also miss stupid mistakes because not everything is syntactical.

AI for example can catch the fact that `phone.match(/\d{10}/)` might break because of spaces, while a linter has no concept of a correct "regex" as long as it matches the regex syntax.

I don't think anyone is arguing that replacing linters with AI is the answer, instead a combination of both is useful.