AIs self-replicate with NO human involvement

https://twitter.com/AISafetyMemes/status/1866495846864306635

by mrosenblatt • 2 years ago

5 10 2 years ago

twitter.com

mrosenblatt• 2 years ago

@AISafetyMemes

"We may soon lose control."

"Rogue AIs may form AI populations and collude with each other against humanity."

"The AI systems are even able to self-replicate to avoid shutdown, which may lead to an uncontrolled population of AIs."

hypeman2• 2 years ago

What would be the real impact of having a self replicating LLM with no set goal once it self replicates? Its only goal is to avoid shutdown, so wouldn’t it be easy to contain ?

vouaobrasil3• 2 years ago

If we take the ideas from Dawkins' "The Extended Phenotype", the LLM's doesn't need a goal in itself. We are making a mistake assuming that the LLM self-replicating needs to have a goal baked in, in other words. The goal can be the goal of whoever operates it. The environment of the evolution of AI agents includes the human brains which operate on instinct and the capitalistic system that provides resources based on the rewards it confers on parts of the environment (human beings).

hypeman2• 2 years ago

Interesting. Thank you - so the goal of the operator would be mirrors across the replications of the system - how fixed would this goal be across replications & would it be possible to “mutate” in a way?

bob1029• 2 years ago

https://arxiv.org/abs/2406.19108

LinuxBender2• 2 years ago

Just an opinion, but a virus can replicate fast because it is tiny and exploits vulnerabilities, can obfuscate and hide itself in tiny places.

An LLM would have very limited options where it could go given the need for a vulnerability and massive hardware and power requirements. It's not like it could hide the transfer or hide itself on the destination. Bandwidth would also be a significant limiting factor.

vouaobrasil3• 2 years ago

A counterpoint: an LLM doesn't need to hide once it becomes dangerous, because the danger it poses may not be the sort of immediate, mortal danger posed by a virus. It could be a longer-term danger where its existence subtly shifts the goal of civlization away from human needs to the needs of the machine, and we slowly become slaves over long periods of time.

LinuxBender2• 2 years ago

It could be a longer-term danger where its existence subtly shifts the goal of civlization away from human needs to the needs of the machine, and we slowly become slaves over long periods of time.

Interesting counterpoint. Perhaps a middle ground and easier to reach conclusion could be that data presented to humans is tainted over time by some combination of the LLM and/or by it's operators to simply make it the next evolution of human manipulation similar to that used by social media sites, but with a touch of human mimicry to increase the efficacy? in that it may be easier for something resembling a human to manipulate humans a little bit at a time, over a period of time, similar to your counterpoint ... political shifts, consumerism shifts, etc... but this is not really replication. This would be a tainted source such as the openai operators being coerced by the powers that be similar to the Twitter and Facebook scandals.

hiked• 2 years ago

I think humans should control AI

AIs self-replicate with NO human involvement

Related Stories