What's the worst thing your AI agent did in production without asking first?

https://news.ycombinator.com/item?id=48658607

by rockingraj • 29 days ago

3 3 29 days ago

news.ycombinator.com

cableshaft• 29 days ago

The worst that's happened, that I'm aware of, is I made a change with A.I. that I didn't realize broke something else (that I wasn't even aware our reports supported, it was new to me), and apparently QA didn't think to check for it either (although I did ask them specifically to verify everything in the report still worked), so it made it into Production, clients complained, and I was told about it and got it fixed over the next couple of days.

But that could have happened if I wasn't using A.I. also, because it wasn't something I was aware I had to check for and I could have also broken that same thing while doing the work manually.

We don't have AI agents do anything directly in Production, so I think this is about the worst that can happen with our setup right now.

coreyp_1• 29 days ago

In our company, Claude keeps trying to run terraform commands (another team member's experience). Thankfully, it's in a locked-down environment, but it still keeps trying, even if you tell it not to.

In development, it will sometimes do all sorts of wild and convoluted things because the conversation has been compressed and it suddenly can't remember the structure of the project or other critical instructions.

AI should never have access to a production environment. Ever. Thankfully, we enforce that.

montfort• 29 days ago

Under very specific instructions to obtain only logs and not make any changes to files, Gemini Pro 3.1 applied remediation measures by altering configuration on two occasions. It only stopped because we discontinued its use.

What's the worst thing your AI agent did in production without asking first?

Related Stories