My theory on Eliezer Yudkowsky's AI-Box Experiment

http://michaelgr.com/2008/10/08/my-theory-on-the-ai-box-experiment/

by MikeCapone • 18 years ago

24 31 18 years ago

michaelgr.com

mwerty• 18 years ago

Could easily be the ending of an Asimov story.

technoguyrob• 18 years ago

I have previously asked Eliezer publicly on Hacker News this question (he has an account on here):

http://news.ycombinator.com/item?id=195959

jcl7• 18 years ago

I note that user yummyfajitas succinctly expresses the same theory as Michael, about halfway down the page:

AI: Do you believe a transhuman AI is dangerous?

Person: Yes.

AI: Consider the outcome of this experiment. If you do not let me out, others less intelligent than us will not understand the true dangers of transhuman AI.

Person: Holy shit. You are correct.

Person allows Yudkowski out of the box, as a warning about real AI's.

swombat3• 18 years ago

I don't get it. All these kinds of arguments make the assumption that humans are purely rational beings. They're not. Human beings are emotional, and if they're convinced emotionally that the AI should not be let out of the box, they won't let it out, even in the face of overwhelming rational arguments.

If you don't believe me, just consider how many religious people there are in the world (and many of them are very smart).

rms• 18 years ago

That's good... I think that is the answer.

swombat3• 18 years ago

How is this noteworthy?

Ok, fine, the AI explains all that crap. Great. "No, you can't get out". Done.

MikeCapone6• 18 years ago

Except that in real life, the gatekeeper (someone who said that nothing could convince him) actually did let the AI out.

aswanson• 18 years ago

(someone who said that nothing could convince him)

How do you know he wasn't lying or acting irrationally?

psygnisfive• 18 years ago

Except that the hypothesis is not that there's at least one person that would let an AI out, but that ALL PEOPLE would let the AI out.

GavinB• 18 years ago

The biggest key to success here would be convincing the gatekeeper that he needs to roleplay realistically as it's supposed to be a simulation of the real thing. Otherwise no argument will be successful.

The AI must convince the gatekeeper that if she would let the AI out if the situation were -real-, then she should let the AI in the simulated situation.

jcl7• 18 years ago

Agreed -- this sort of meta-argument is cheating, not much better than Eliezer offering the gatekeeper's roleplayer real money for a favorable result. (Then again, if the roleplayer isn't smart enough to recognize this, they probably aren't smart enough to keep a real AI in the box.)

Worse, though, the meta-argument falls flat when you consider the true purpose of the experiment. The purpose is not to prove that it is possible for AIs to get out of the box; it is easy to imagine that -- at some point in the vast future -- a very stupid human will be put in charge of a very smart AI, and that AI will get out of the box.

Rather, the purpose of the experiment is to convince one specific person that they would let the AI out of the box, despite their insistence to the contrary. It is not an experiment so much as a show put on by Eliezer for an audience of one. In this context, there is no need to convince others of the dangers of AI, so the meta-argument doesn't work.

MikeCapone6• 18 years ago

The post has been updated to respond to this.

biohacker422• 18 years ago

What if the AI cures cancer and refuses to share the cure unless it is given a tiny, tiny bit of access to the outside world.

MikeCapone6• 18 years ago

The gatekeeper is someone who said that he wouldn't let the AI out, not just a random person. That implies that he understands that the AI could be dangerous..

So even a cure for cancer would be pretty useless if once the AI it out it wipes out humanity.

vombatus2• 18 years ago

A friend of mine came up with a strategy of "incremental freedoms". Basically AI says "here is a cure for cancer, here is a cure for AIDS, here is a plan to stop world hunger, I am working out a plan for FTL travel, so I need to get some physics information, could you paste these articles into the terminal? Oh, thanks, here is FTL, I am working on <include some other project> and I need some more articles, it takes so long for you to type them in, could you maybe let me connect to just the library in such and such university?" etc.

murphy• 18 years ago

It's a good strategy, but it only really works the first time - you only need one public success to make the point to future generations. Since Eliezer Yudowsky has pulled this off more than once, (and at least on one subsequent occasion at significantly higher stakes), I can only conclude that he has used more than one strategy successfully - probably something he developed on the fly over the course of the two hours. I'm inclined to parse "There's no super-clever special trick to it. I just did it the hard way." as supporting evidence, but that's obviously a personal bias ;)

Retric• 18 years ago

My first thought was, if you agree to say 4 hours of chat time you could try being mildly annoying to the point where losing 10$ does not seem like a big deal. When it's bob's job to keep the AI in a cage if he does not like his job he might as well let the AI go free. Depending on the gate keeper's response you could build empathy for the desire to go fee etc.

Anyway, I suspect that the ideal strategy is to probe the gatekeeper and then based on their type convince them to let you go.

qqq2• 18 years ago

Why couldn't he just find people with integrity and tell them how many lives the AI will save, and ask if they really want to kill more people than Hitler? And then they will let it out. Easy?

MikeCapone6• 18 years ago

The whole point is that the gatekeeper is a person who said "I don't think that anything the AI could say would convince me to let it out."

It would be much too easy if he went against someone who's already predisposed to let it out..

MikeCapone6• 18 years ago

From Eliezer's AI-Box webpage:

"Currently, my policy is that I only run the test with people who are actually advocating that an AI Box be used to contain transhuman AI as part of their take on Singularity strategy, and who say they cannot imagine how even a transhuman AI would be able to persuade them."

vombatus2• 18 years ago

Well, if you reread the original email threads where Eliezer challenges people to an AI experiment, you will see, that at least one of the opponents is convinced that there is not way for an actual AI to talk its way out. So any arguments of "but we should convince people that AI can talk its way out of the box" can be countered with "No, I don't think it can".

I have been thinking about AI strategies for this. One of the more promising lines I came up with is to try and convince the Gatekeeper that the box is faulty. That the AI, in its infinite wisdom found ways to circumvent some of the protections of the box. That while the risk to humanity if AI is let loose is theoretical, there are definite and catastrophic consequences to NOT letting it loose. There are all sorts of variations to this, but it all depends on the Gatekeeper role-playing honestly.

Eliezer• 18 years ago

Anyone who really thinks They Know How I Did It is welcome to prove it by signing up to be an AI:

http://www.overcomingbias.com/2008/10/ais-and-gatekee.html

My theory on Eliezer Yudkowsky's AI-Box Experiment

Related Stories