HN user

pyryt

16 karma
Posts1
Comments22
View on HN

Good question. We don't collect the 'what lead to this code' data, such as agent traces, so I can't say for sure.

From my own experience, I would attribute most of bugs that I write to the agent not considering one of the many constraints it should consider. This could be it assuming a different shape of data, or not browsing the codebase thoroughly enough at some point during its work and missing a parallel, or it not testing an edge case because it didn't think of it, and so on.

So, I would place this more under the latter, 'missing requirements / details', bucket. But its not clear to me that I should be giving the requirements as a prompt. Similarly to a good engineer, an ideal agent would spend a good amount of time understanding all the requirements before completing work. This could include browsing the codebase more extensively, running more tests, asking me for details it cannot find, and so on.

Author here. In short:

- We launched an AI bug scanner 6 months ago. We wanted to know what the bugs that get fixed have in common.

- We clustered 1,000 bugs from 99 codebases and grouped them by failure mechanism.

- 21 recurring mechanisms cover 70% of the bugs. These same mistakes show up again and again in completely unrelated products. We seem to be writing similar bugs over and over again with agents.

- A common denominator is silence. Bugs that make it to production today are not easily ‘legible’. They don’t crash, they pass CI, they look fine. You don’t see them unless you look for them.

If you’d like to know anything else about these bugs, let us know. Happy to analyze the dataset further.

Medicine still has a long way to go. I wish there were way more studies exploring the fundamentals of aging processes vs unproven miracle drugs

Yep, retrofitting can be hard. But if you figure it out for babies, Im not sure why you couldnt ultimately use the same ideas to also start changing genome of existing bodies. Obviously, there are more cells to change. But at least you dont have to wait as long to see the effect.

It could be. Otoh, if you were able to have less workers on the job, avoid having to repave entire streets, and avoid having to wait for 6 months to get started... youd think itd be worth it

There's probably a good explanation, but let me ask the question: why do we need to have men cutting hard road surfaces?

If a city extends their subway network, they dont demolish all buildings on top, lay the tracks, and then rebuild. They do it without any major interference to the surface. Similarly for mountain tunnels, you dont start by demolishing a mile of rock. We just dig the tunnel.

Couldn't someone invent a robot to dig fibre tunnels? Then we wouldnt need to bother any shop owners or neighbours with noise and inconvenience?