AI coding may not be helping as much as you think

https://garymarcus.substack.com/p/breaking-news-ai-coding-may-not-be

by wyclif • 1 year ago

6 3 1 year ago

garymarcus.substack.com

I have a tool that I vibe coded. Pretty basic, it pulls data from a bunch of different tools, and writes that data to a spreadsheet. I’m too busy to really invest in it(which is why I vibe coded it) and now I’m stuck. Not impenetrably so, I’m sure I could debug it with “relative” ease. But the vibe coding is going in loops, keeps undoing things and one of the integrations just fundamentally doesn’t work.

For all the “AI will be writing 100% of code by the end of this year claims”. Man I’m bought into the dream, I just wish it wasn’t so disappointing.

wyclif• 1 year ago

OP here—I'm posting this mostly to see what others think, but I have some serious problems with this study.

What happens when they break down the tasks? This study measures "development tasks."

Also, the study only mentions "245 tasks in mature projects" but doesn't specify what these tasks were anywhere. Were they debugging, new feature development, code review, architecture design, or simple maintenance? The tasks were in "mature projects" where developers had 4.9 years of experience on average.(Average residence at FAANGs is about 2-3 years before engineers move on...is that still true in 2025?)

That makes me wonder if these don't generalize to most real-world development work:

"experienced open-source developers"

"AI lab research engineer" jobs referenced

Those also make me wonder what kind of roles these are. They don't sound like general roles.

Another thing: Cursor Pro is a very specific tool. What about the time developers spent learning to use the AI tools during the study period (before they reached performance goals)?

techpineapple2• 1 year ago

Are you saying that you think the tools are better than the study suggests or worse?

AI coding may not be helping as much as you think

Related Stories