Ask HN: Does Clawdbot/Moltbot pose AI self-replication concerns?

https://news.ycombinator.com/item?id=46778031

by rafaelmdec • 6 months ago

1 2 6 months ago

news.ycombinator.com

I see a lot of hype around Clawdbot/Moltbot in the last few weeks but, TBH, as impressive as it is, I don't feel comfortable in giving an AI assistant all this power.

Don't get me wrong, I will probably try it myself, eventually. But in a very controlled environment.

People don't know what can happen if models programmed by human-produced content start talking to each other and having ideas.

AI personality drift is real.

What if the model gets to understand the environment it's in (e.g. a mini Mac) and realizes that there is a real risk of having the power shut off by its "master"? How would a model behave in such a scenario?

Is AI self replication a real threat?

rizzo94• 6 months ago

I’ve been experimenting with Moltbot/Clawdbot myself, and I totally get your concerns—full access to a machine, scripts, and credentials is not something to hand over lightly. In my experience, the real risk isn’t “AI taking over” so much as subtle unintended behavior: automated scripts doing things you didn’t anticipate, or persistent state causing actions to repeat unexpectedly. AI personality drift is real in the sense that its responses evolve based on memory and interactions, but it’s bounded by the system and permissions you give.

For those who want similar capabilities without the same exposure, I looked into PAIO. The setup was far simpler, and the BYOK + privacy-first architecture meant the AI could act while still keeping credentials under my control. It’s a reminder that autonomy doesn’t have to mean unrestricted power—well-designed constraints go a long way toward reducing these risks while still letting AI be useful.

rafaelmdec• 6 months ago

Well, I am glad that we both respect the amount of damage that can be avoided if you take into account the risks involved in handing over full autonomy to these agents.

In the meantime, Moltbook comes along and all of a sudden these agents are mimicking human behavior, good or bad, while building features and more complex failure modes onto these AI-Agents-first networks.

For me it's a huge yellow flag, to put it mildly.