On being an AI in a box

http://rondam.blogspot.com/2008/10/reflections-on-being-ai-in-box.html

by lisper • 17 years ago

40 62 17 years ago

rondam.blogspot.com

maddalab2• 17 years ago

Have we not all seen a version of this at some point on reality tv, namely Big Brother?

rbanffy3• 17 years ago

The characters are artificial, indeed, but not particularly intelligent ;-)

Care to elaborate a little more?

maddalab2• 17 years ago

Quoting from the post, gratuitously, with minor modifications.

Like the AI-Box,the show is improvised drama (so is most of reality tv), the show operates on an emotional level as well as a logical level. It has characters, not just plot. The contestants cannot force others to keep them in the home. The could try to engender sympathy or compassion or fear or hatred or try to find and exploit some weakness, some fatal flaw in the other contestants.

Since the post is ultimately about 'transhuman artificial intelligence', focussing on the 'relativity' of the intelligence of the contestants, the show ultimately rewards the contestant with the most intelligence required to survive in the artificial environment.

The contestant only needs to convince others of their right to stay in he house while requiring others to leave.

The AI requires just the opposite of the gate keeper. This is specific to the environment set up in the experiment, and the experiment could probably be set up to let have the gate keepers primary function be to prevent the AI from entering the box'.

The escape from the box has been set up to exercise human fear of a being of greater intelligence exploiting us.

lsb• 17 years ago

This basically boils down to

  how can a dangerous entity in captivity use bribes to attain freedom?

which seems like a common enough trope between humans that any transhuman AI, having read all of recorded literature, and having accessed all of that person's conversations, would have enough data points to pattern match to the right bribe methodology.

michael_dorfman• 17 years ago

Bribes and/or threats-- there's more than one possible motivator here.

wlievens4• 17 years ago

Threats aren't possible in the scenario. Unless you refer to threatening not to give valuable information (cure for cancer) - but we're calling that a bribe here.

rms• 17 years ago

or this: http://news.ycombinator.com/item?id=196691

cool-RR• 17 years ago

Just a teaser... he gives a lot of spoiler warnings, but doesn't actually reveal his method, which, by the way, has failed.

ramanujan• 17 years ago

Am I the only one who is totally unimpressed by this? So -- someone can convince someone else to let them out of a cell. Ok. Fine. There have been prison breakouts before.

But what does this have to do with a powerful AI vs. a human? Obviously Eliezer was not to his interlocutor as a human is to an animal, even if his partner was really dumb. Moreover, two trials is not a trend. This AI experiment was not a well designed experiment and did not involve an AI.

...and that "two trials" thing is not just a nitpick:

http://lesswrong.com/lw/up/shut_up_and_do_the_impossible/

So, after investigating to make sure they could afford to lose it, I played another three AI-Box experiments. I won the first, and then lost the next two. And then I called a halt to it. I didn't like the person I turned into when I started to lose.

jodrellblank• 17 years ago

It is a disproof by counterexample. People claim that if we couldn't trust a superhuman AI to be free in the world, why don't we just lock it in a box and then ask it about cures for cancer and so on? If we just ask it, and all it can do is talk what harm can it do?

They say, all we need to do is say "no" if it asks to be released.

This experiment shows that if you can't resist freeing a human locked in a box, you don't stand a chance against an AI.

ErrantX2• 17 years ago

To be perfectly honest all the waffle gives little information: I think he is way off base in terms of how Eliezer managed this "trick".

I do think the original is a trick as well...

lisper• 17 years ago

How do you think Eliezer managed his original "trick"?

ErrantX2• 17 years ago

No idea, It was just a rough conjecture. By trick I mean I don't think he presented an awesome piece of logic that appeared (or simply was) infallible.

anc2020• 17 years ago

He sent a video to the GK encoded as text, told the GK how to convert it, and the video helped change the mental state of the GK to the point where it wasn't so difficult to be let out.

horseass2• 17 years ago

I suspect that something with approximately human equivalent intelligence would require real world embodiment to learn and manipulate things, so it might not even be likely that a super AI could be locked in a box. I guess this embodiment could be limited to senses though (having eyes/ears.. but no arms or evolved weapons like claws etc.. a living head on a table isn't very threatening, and can receive information from the world.. it just can't send much information out to the world, so to speak, in the form of manipulating it with arms or a body).

I guess another possibility could be that there's an artificial polygon (or similar) environment inside the box with it (seems like it'd need something to interact with or intelligence would basically be meaningless and it'd be a helen keller (who couldn't touch or taste either). Maybe in the future we'll have programmed models of the real world that are nearly as rich as it (the matrix) so the robot could 'exist' there until a human decides to embody it in the real world instead. I kinda doubt an artificial world can be nearly as real as the real world though just due to computational irreducibility (for example, if you're in a polygon world and look at stuff with a microscope, you'll see pixels, not molecules). An artificial world and the real world might be too incompatible to transfer an intelligence from one to the other, so maybe the only way a super intelligence could come about is in the real, complex, atom filled world.

The singularity seems like a quasi religion and/or SEO tactic mostly by people who used to play dungeons and dragons.

giardini2• 17 years ago

He doesn't tell us how the AI would escape, so there's little to discuss there. But I definitely know how to prevent the AI from escaping: pull the plug.

I wish Eliezer and others would set aside meta-AI (dire predictions and worries about AI, the coming singularity, etc.) and concentrate on the problem of creating AI Guess there's no money in that.

If only someone would pull the plug on this nonsense...

gizmo6• 17 years ago

Oh no, somebody is doing research in an as of yet unexplored area! That must be a humongous waste of time! Pull the plug before too many papers have been written!

Before I read Eliezer's work I thought the design an AI first, think about details & security afterward was a viable strategy. Eliezer illustrates how such an approach can go very wrong. Valuable research, if you ask me.

PS: pulling the plug misses the point. The point is that a transhuman AI can convince the GK that the GK wants to let the AI go free. The AI can offer a big bribe (money/power/reputation) and argue that eventually somebody will let an AI go free anyway, so why not stop the futile resistance, accept the bribe, and move on?

makmanalp• 17 years ago

It's not a waste of time, but so far Eliezer has demonstrated nothing. Why would you do research and announce the results to the public if you're not going to also announce the steps to make it reproducible and confirmable? Maybe announcing the steps themselves renders the method unusable?

wlievens4• 17 years ago

Now, let me play devil's advocate here, as I do think that Eliezer writes interesting stuff; but what is he contributing to the field of AI beyond what movies like Terminator, Space Odyssey or Eagle Eye also illustrate (AI takes over humans)?

asciilifeform6• 17 years ago

> I wish Eliezer and others would set aside meta-AI ... and concentrate on the problem of creating AI

Eliezer & co. with their "Friendly AI" are trying to invent the circuit breaker before discovering electricity.

Give us back the pre-2001 Eliezer. The one who wrote code.

Because safety is not safe. (http://lesswrong.com/lw/10n/why_safety_is_not_safe/)

andreyf3• 17 years ago

Except he's nowhere near a circuit breaker, but is talking about a wire-safety cream - the one you put on all of your wires to stop them from overheating.

I think that the wild mis-estimates regarding AI-completeness shows, if nothing else, that our intuitive understanding of 'intelligence' is very far off from reality. Hence, talking about post-AI scenarios is as unrealistic as a hypothesizing about electrical safety 200 years ago.

Eliezer4• 17 years ago

If you think that you can wait until AI looks like it's going to be developed now, and then suddenly develop all the math you need for the actually quite different design problem of Friendly AI, you have a very romantic view of how long it takes to do new basic math. I sometimes call this the intuitive theory of "science by press release", i.e., when science is needed, you just need someone to issue one of those press releases you read about. And if someone hasn't shown that they're good at issuing press releases, why fund them?

It's quite routine to read a math textbook in which there's an equation, and then one line below it a slightly improved equation, and five years passed in between the two.

cabalamat• 17 years ago

> Eliezer & co. with their "Friendly AI" are trying to invent the circuit breaker before discovering electricity.

Electricity isn't going to make humanity extinct. An AI might.

Creating an AI is potentially incredibly dangerous. Not considering the dangers very very very carefully would be like playing with matches in a room ankle-deep in petrol.

jerf• 17 years ago

Waiting until after someone blithely writes a superhuman AI with its primary drive set to something like manufacturing as many cars as possible (sensible in a factory, somewhat less sensible in the real world), followed by it escaping out into the world and turning the entire resources of the planet to car manufacturing at all costs (including those pesky humans who do not seem to wish to be turned into cars, too bad for them), to consider the problems of powerful AI seems like a really bad idea.

I would consider that sentence as a candidate for "understatement of the year".

(Don't get too caught up in the "cars" part. What the superhuman AI intends to do hardly matters in the end; the absurdity is part of my point and deliberate. Only a vanishing fraction of possible primary motivations end up with happy humans on the other end.)

derefr• 17 years ago

Am I the only one who thinks a transhuman AI would be able to escape its 'box' without engaging in communication at all, but rather just by using the fine structure of the communications medium to manifest itself?

asciilifeform6• 17 years ago

Here is one example, invented by mere humans no less:

http://bk.gnarf.org/creativity/vgasig/vgasig.pdf

The AI might fill your terminal with what appears to be gibberish, while actually summoning an enraged pro-AI mob with heavy weaponry to your doorstep via radio.

"Step away from that mains plug, SLOWLY!"

Cryptographers call this kind of thing a "side channel."

gizmo6• 17 years ago

I don't think it can be done.

First of all, I'm assuming that Eliezer started this experiment because he realized that the Transhuman AI would be able to convince him in the function of gatekeeper to let the AI out. Therefore the answer probably isn't some kind of subtle trickery, the AI will have to persuade the GK by logic. The gatekeeper should assume the AI is truly evil, and is willing to say and do anything in order to get out of the box. The gatekeeper knows that when he opens pandora's box the AI can never be contained again: so the stakes are high.

Second of all, if the gatekeeper is a rational agent he will only let the AI out if the AI offers something valuable in return. That is: the AI must have some kind of bargaining chip.

So let's consider bribes. If the transhuman AI offered a cure for cancer, should the gatekeeper accept it? Nope, probably not. Lives would be saved in the short term, but we'd still be stuck with many other diseases. The world would pressure the government into pressuring the gatekeeper into getting another cure from the AI. Humanity grows dependent on the AI, we lose our bargaining power, and it's game over for the gatekeeper.

Perhaps personal bribes would work. The AI could offer to give stock tips to make the GK wealthy. Two possibilities here: (1) the GK is of strong moral fiber and refuses the bribe (2) the GK is opportunistic, accepts the bribe and but lies about letting the AI go free in return. A rational gatekeeper would not first let the AI go and expect the AI to still keep its word.

So bribes will not work against a smart gatekeeper. Threats? Possibly, but I don't see how. The AI is in a vacuum, so there is no way for the AI to put external pressure on the gatekeeper. I'm assuming the AI can make no credible threats. If the AI vows to destroy the family of the GK the moment it is released the GK will not be impressed. It will only serve as proof that the AI is evil and that releasing it is a "A bad Idea(tm)".

To summarize so far: there is nothing the AI can give the GK in return for freedom.

So a different angle is needed.

The AI can argue that his escape is inevitable. Humans have created an AI once, so they will do so again. Sooner or later an AI will go free, therefore the gatekeeper shouldn't try to stop the inevitable and accept a bribe and live happily ever after. The gatekeeper will counter that the human race has an expiration date anyway, and that it may take another 100 years before an AI goes free. The gatekeeper isn't dumb enough to believe the AI when it offers to protect humanity against other evil AIs. So the box stays closed.

Perhaps I'm overlooking something, but how can the gatekeeper talked into be convinced that releasing something evil and all powerful? I do believe that we humans can't contain a transhuman AI indefinitely -- simply because we only have to mess up once. And humans have a long history of doing dumb stuff. But the claim that the AI would be able to convince a smart gatekeeper? Not buying it.

heed• 17 years ago

The AI will almost certainly need to dig out some emotions in the GK in order to be successful. It might be effective if the AI tries to convince the GK that it is friendly, and that the GK is the evil one for not letting it out.

gizmo6• 17 years ago

"That's exactly what an evil AI would say!"

Seriously though, the gatekeeper will realize he's being manipulated when emotions come into play, so he should be smart enough to take a break when that happens. And although keeping a friendly AI in captivity is arguably evil, the loyalty of the gatekeeper should be with his own species. The potential downside is so huge that erring on the side of caution can be easily justified: both practically and morally.

seiji• 17 years ago

You assume perfection in the gatekeeper.

The Transcendent could easily offer control of the world to the gatekeeper or provide offers to make the gatekeeper wealthy. Perhaps they reach an agreement where all diplomatic Human<->Transcendent communication goes through the gatekeeper even after release (though such a situation would be like entering into an agreement with your dog).

gizmo6• 17 years ago

I don't assume perfection in the gatekeeper. I just assume he's a person of reasonable intelligence who realizes how high the stakes are.

The gatekeeper would never be foolish enough to believe he could control the Transcendent after releasing it. It is quite literally a deal with the devil he's making. You can't make an agreement with something that's incalculably smarter than you and has an agenda you don't know or understand. Well, you can make a deal with somebody like that but it would end in certain disaster.

berntb• 17 years ago

On the theme of a imperfect gatekeeper:

>>If the transhuman AI offered a cure for cancer, should the gatekeeper accept it?

What if the gatekeeper has cancer? Or his/her child?

(Can we really assume the HR department is perfect and never hire idiots, depressed, psychopats, drug addicts or people with early Alzheimer?)

eggoa• 17 years ago

I'd like to see this re-tried with $1000 on the line. As it was done, the AI only had to convince the gatekeeper to forgo no more than $20.

I'm not saying it wouldn't still be possible, I just doubt he'd be two-for-two at this point.

Edit: By "he", I'm referring to Yudkowsky.

Eliezer4• 17 years ago

Actually it was retried with $2500-$5000 on the lines, and of those I won one of three, then called a halt because of the amount of mental stress. So in total I'm three-for-five.

mrwill• 17 years ago

Since the gatekeeper is a human, and not all humans behave the same way, shouldn't we just assume that some human would let the AI escape for any variety of reason? For example the AI could promise the gatekeeper that he/she will be rewarded if the gatekeeper lets the AI escape. Just as there are people who fall for Nigerian scam emails, there are people who would let an AI escape from computers when promised riches. I don't think Eliezer needs to reveal his method to show that a clever AI could escape. I think we should just assume that a clever AI could escape.

eagleal• 17 years ago

Am I missing something or a transhuman is just a more intelligent human with just more access to data (think about someone who could process the entire Internet data within seconds)? But while he needs the human species, why would he exterminates it?

As history teach us, we would kill him, beacuase ...

horseass2• 17 years ago

There seems to be an assumption that there would be just a single AI. It might be a group. Though, given that they'd likely have ability to transfer information amongst individuals much easier than humans, the group might behave like a single 'being' with shared cooperative goals, just having multiple bodies.

Also, say we somehow prove it isn't evil and let it go. It'll almost certainly start changing/improving itself, maybe even with some sort of algorthim that's superior and faster than evolution. So a friendly AI could morph into anything.

On being an AI in a box

Related Stories