Why AI is a dangerous dream - interview with Noel Sharkey

http://www.newscientist.com/article/mg20327231.100-why-ai-is-a-dangerous-dream.html?full=true&print=true

by limist • 17 years ago

40 77 17 years ago

Leon• 17 years ago

His sticking point is that the brain or mind could be a physical system that is not computable or equivalent to a Turing machine. If we take the assumption that the mind is a physical system, then it would still only be a matter of time before we found the reason for the difference in computational ability (maybe it would be like Penrose's quantum arguments for the mind), but those systems would still be within reach of implementation within a computer system. The brain is still a combination of proteins, amino acids, and a few other chemicals. We would just have to move to using chemical computers (which are already being researched).

Even following his arguments, it would only be a matter of time before sentient machines are created, equivalent to our own minds.

His arguments against AI are mostly 'just because' and has little proof.

gruseom• 17 years ago

A system that is "not computable or equivalent to a Turing machine" is not "within reach of implementation within a computer system". (Unless, of course, the Church-Turing thesis is false. Do you have any evidence of this?)

jacquesm18• 17 years ago

No, his arguments are not 'just because', this is a pretty smart guy and from what I get out of it his reasoning goes like this:

If the mind is a physical system then we can attempt to recreate it by computer but this is no guarantee that such an attempt will be successful.

For an example of such a situation, imagine that it would take all the resources of a planet in order to create a single AI then it might be theoretically possible but in practice it would fail anyway.

And that is assuming that it is possible, for which there is no proof until it is done. If the mind turns out to depend on mechanisms that we can not model accurately enough to get the desired effect then the whole deal is off.

There can be no 'proof' that AI is impossible, just as much as there can be no 'proof' that green elephants do not exist.

The only proof that would be acceptable is for someone to produce an AI.

Nobody needs to present proof that it is not possible.

Locke16892• 17 years ago

Well, that's not entirely true. What is true is that if the mind is more powerful that a Universal Turing Machine, then we have no idea how to approach building one. However, we know that the resources required to build one are small -- humans do it all the time in a relatively cheap process called "sex." That is, the lower bound for creating AI is very small. Of course, the danger is that our brains have certain limitations for a reason -- these could be the property of this super-Universal Turing Machine. Therefore, a true AI may look nothing like our modern computer. The GP is correct in that we should at least be able to build a simulacrum of our own mind -- even if that is simply an artificially constructed duplicate using biology, genetics, and biochemistry. I would say that this may be fairly far away, because we have yet to truly understand many of the processes and structures in the human brain. This is why I believe AI is much farther off than most people think: to understand AI we must first understand ourselves, which is much more complex than at first take.

Now, this entire hypothesis rests on the idea that our brain is super-Universal. That is, it cannot be phrased as a computable function (by the Church-Turing thesis). This is a fairly unpopular view in the AI community, but valid until we prove otherwise. Up till now there has been no evidence that such a Turing machine exists in theory, much less in practice, which is a minor blow. In addition, many of our psychological structures and chemical activities seem to suggest a Universal Turing machine.

Edit: I just realized that I implied that a super-Universal Turing machine is a Turing machine. This is obviously untrue, by definition. Consider my terminology to be a replacement for whatever name we would give to this noncomputable function solver (since we have no category for that at the moment).

Edit2: Not that we don't have a category for noncomputable functions, only that we don't know which category they would fall into. I guess the catch-all "hypercomputable" is suitable. Thanks, naveesunder

thunk• 17 years ago

If we take the assumption that the mind is a physical system, then it would still only be a matter of time before we found the reason for the difference in computational ability

I don't see how the latter follows from the former. The physicality of the mind says nothing of its computability. You're assuming the computationalist axiom that reality is completely computable, which is quite an assumption.

The brain is still a combination of proteins, amino acids, and a few other chemicals. We would just have to move to using chemical computers (which are already being researched)

We already produce hundreds of thousands of them every day. The process is rather fun. As some point the definition of success has been stretched so far as to be meaningless.

omouse• 17 years ago

This was already posted, wasn't it?

astine• 17 years ago

It's the first time I've seen it, so I appreciate it if it was re-posted.

gizmo5• 17 years ago

Silly. So very silly.

First of all: he doesn't use his definition of intelligence consistently. According to his usage of the word a being can be truly intelligent, and another one can _appear_ perfectly intelligent but be in reality a dumb machine that merely simulates intelligence. That's his chess example. But he _defines_ intelligence as the science of making machines do things that lead us to believe they are intelligent. A chess computer that outsmarts me most definitely fits that definition of intelligence.

He continually makes the claim that "Technological artifacts do not have a will or a desire". There is no reason to assume that you need some primal force to get a will or desire (perhaps any sufficiently complex system will have a will or desire as a side-effect), and there is no reason to assume a will or desire can't be perfectly emulated.

He claims that there are no signs that computer processing speed will eventually overtake that of the human brain. I say that computer processing speed is undeniably improving rapidly, and that the number of tasks that computers can do is rapidly increasing. Unless there is a hard limit somewhere the assumption should be that we will be eventually overtaken.

Computer AIs can play Chess, Checkers and Super Mario. They can create art, and compose music. They can drive some vehicles, and land planes at night. How about science? Some proofs are made and proved correct almost entirely by computer. Some proofs are so complex they can only be verified by computer. In many research fields a single human is almost guaranteed to contribute nothing. A computer, on the other hand, can probably brute-force his way to many new discoveries.

He again makes the claim that computers are innately unable to feel compassion or empathy. Only to finish with the dumbest remark of the article: "I don't think they will be very good at faking fouls", which is clearly a matter of basic game theory. And I think that robots will completely crush humans at soccer, even if they lack strategy. The moment robots are good enough to take the ball away from a pro human player, they will be able to do so consistently. Even if they run slower and shoot only semi-accurately, they will never make big mistakes. And history shows that in any game where computers can compete the humans have to play (almost) perfectly to even stand a chance (see: Chess / Checkers / Poker). We meat bags with our 100ms+ response times will never be in the same league as robots. Either we will be far superior to the robots, or the robots will run circles around us.

jacquesm18• 17 years ago

> A chess computer that outsmarts me most definitely fits that definition of intelligence.

No, it may merely indicate that you are a lousy chess player. What you believe to be intelligent and what is intelligent are two different things.

An intelligent computer would not be programmed with anything specifically related to chess, it would simply learn the game like any human learns board games.

> He claims that there are no signs that computer processing speed will eventually overtake that of the human brain. I say that computer processing speed is undeniably improving rapidly, and that the number of tasks that computers can do is rapidly increasing. Unless there is a hard limit somewhere the assumption should be that we will be eventually overtaken.

That depends on the kind of tasks. Bookkeeping, data storage and retrieval, encoding, decoding, these are all things that a computer is exceptionally good at. But they are all at heart still simply ways to manipulate numbers, basic arithmetic if you want.

Composing a piece of music by computer takes a programmer. No computer ever woke up one fine Sunday morning and wrote a symphony because of the way it felt about the light.

They do not create art, they do not create music, they are excellent tools in the hands of artists and composers.

Tichy5• 17 years ago

There are programs that can listen to a composers music and then compose music in that composers style.

gizmo5• 17 years ago

I beg to differ.

According to that logic reading a physics book is "dumb", I should derive all physics knowledge myself. Reading a book is precomputed knowledge, and that's no different from precomputed chess strategies. If I'm allowed to use books as a foundation for my intelligence, then computers are allowed precomputed knowledge also. They're just better at retaining it. Good for them.

The problem with your definition is that you can have a humanoid who speaks a dozen languages fluently, is great at math and physics, can answer any history question, can correctly understand and act in different social situations and despite all those things, is still classified as dumb.

You can define intelligence as the capacity of learning, taking knowledge completely out of consideration. It does not, however, match the definition of intelligence in common language. It also means that when you have a robot that knows everything it is by definition dumb, because there's nothing left to learn.

> But they are all at heart still simply ways to manipulate numbers, basic arithmetic if you want.

And humans are just a bunch of atoms. Atoms form molecules. Molecules form nucleic acids and other building blocks. You can build things of immense complexity with primitive primitives. That something as complex as a human can consist of a few elements is pretty hard to grasp. But if I accept that, then why can't it be simulated with bits?

Art created by computers is primitive, sure. But human art started out pretty primitive too. I see this as a difference in complexity, not as some magic line computers have to cross for their output to become art-worthy.

alexfarran• 17 years ago

> But he _defines_ intelligence as the science of making machines do things that lead us to believe they are intelligent. A chess computer that outsmarts me most definitely fits that definition of intelligence.

No he doesn't. He defines AI that way, and that's the point. AI is an illusion of intelligence. He sees a danger in mistaking it for the real thing - in effect mistaking machines for people.

Tichy5• 17 years ago

What's "the real thing"? Of course you could always define AI as "not real intelligence", then by definition AI will never be "really intelligent". It would not be a very interesting claim, though.

gizmo5• 17 years ago

You're right, he defines AI that way. He seems afraid that robots will take on human responsibilities even though they're unable to feel or understand human emotions. That, I agree, may not work out so well.

I don't see why we should have a different metric of intelligence for computers and humans, though.

naveensundar5• 17 years ago

> And history shows that in any game where computers can compete the humans have to play (almost) perfectly to even stand a chance (see: Chess / Checkers / Poker).

What about GO? http://en.wikipedia.org/wiki/Computer_Go

rwolf3• 17 years ago

The important part of the quote there is "where computers can compete". Go-bots have a long way to go before they'll be considered competitive.

anigbrowl• 17 years ago

Oh God.

We can automate a process, and with technology we can automate increasingly complex processes. But intelligence is a fundamentally different problem from the things you describe. I am a believer in the possibility of strong AI. I can envision and would like to live a world of sentient machines. But your way of thinking, as expressed here, will not get us there.

Let's take the example of robot soccer players. I agree with you, it's basically just an engineering problem. We have robots that walk around, we have computers that can solve ballistic problems in real time, we have networking and so on. I can easily, easily imagine a winning soccer team made up entirely of robots. If I was rich I would tempt you with money to hasten the creation of such a thing.

What I can not imagine - or rather, what I am unable to conceive a route towards - is a robotic soccer player that announces one day it is sick of putting balls into goals and is retiring to take up painting instead.

Or take chess. I can't beat my computer at it any more, although that's not saying much as I was never all that good. Yes, I can envision a chess program that says 'hmm, you keep losing, how about I start without a pawn until you improve?'. Indeed, we're not so far away from being able to build a good chess tutor, not least because chess play has a fully defined grammar and so winning positions in chess games could be regarded as true statements in a formal system. On the other hand, further development in this area is unlikely to produce software with a secret addiction to backgammon.

We have software that can create pictures or music in a certain style. But although they can produce interesting and unexpected results they're still horrendously limited in other ways. If you request a new composition and don't like the results, will your computer denounce you as an ungrateful philistine, while mocking your bourgeois notions of formalism?

Computers don't want to do things. We want them to do things, and currently all we know how to do is build simulacra of ourselves. Doubtless, as time goes on, we will be able to build more general-purpose simulacra - our robotic soccer players will also compete at rugby and tennis, our chess programs will handle a wider domain of dynamic analytical problems, our artistic software will employ inductive processes that begin with general concepts of proportion, contrast and so forth.

But we are not going to get to real artificial intelligence until we start building systems that are a) self-modifying and b) allowed to engage in perverse behavior whose purpose is not obvious to us.

tybris• 17 years ago

Seems like he went from underestimating to overestimating human intelligence.

mynameishere• 17 years ago

Soccer robots can move quickly, punch the ball hard and get it accurately into the net, but they cannot look at the pattern of the game and guess where the ball is going to end up.

I'm pretty sure that computers already play sports (minus the robots) acceptably well.

coderdude• 17 years ago

If you're talking about sports video games, the game knows of and keeps track of everything going on in the game world.

darien• 17 years ago

In this interview he is suggesting that people stop a self-fufilling prophecy before it is to late. I think his fear is that if people believe superior ai is inevitable that researchers will work off of those assumptions as if they were given facts - eventually creating systems which might pose threats or general harms to humanity. Other than that, only time will tell who is more correct.

hopeless• 17 years ago

Just as an aside, Noel Sharkey taught a neural networks course I took but everything I learnt about them came from implementing a back-prop net in Occam for another class!

rms• 17 years ago

You should probably just read Eliezer Yudkowsky on this topic. http://yudkowsky.net/singularity/ai-risk

jongraehl• 17 years ago

While I acknowledge the possibility that "real" intelligence may be inherently non-computational, it's ridiculous to claim that it "equally might be" so. (which I take to mean roughly 50% probable). I claim 99.9% certainty that a computational process will demonstrate general human intelligence in the next hundred years, given that civilization isn't decimated before then. We don't even have to be clever enough to organize superpowerful computers into rudimentary general intelligence; we just need to understand our own brains at a fine enough level to emulate them computationally.

And, of course, you don't need anything close to general intelligence to do well in sports.

growingconcern• 17 years ago

"But accepting mind as a physical entity does not tell us what kind of physical entity it is. It could be a physical system that cannot be recreated by a computer." Child Please. Why is anybody even listening to this kook?

rortian• 17 years ago

You are displaying very misplaced arrogance. Simulation of all sorts of physical phenomena are intractable. An example I am familiar with is attempting to simulate a large number of sand particles being vibrated. If you think this is trivial, then you seriously underestimate how computationally difficult simulation can be for physics problems with relatively straightforward mechanics. The mechanics of the brain are not yet seen as straightforward.

The person you refer to is not a child and displayed much more sophistication on these issues than any 'rapture of the nerds' advocates I have ever seen.

endtime• 17 years ago

>I'm an empirical kind of guy, and there is just no evidence of an artificial toehold in sentience. It is often forgotten that the idea of mind or brain as computational is merely an assumption, not a truth. When I point this out to "believers" in the computational theory of mind, some of their arguments are almost religious. They say, "What else could there be? Do you think mind is supernatural?" But accepting mind as a physical entity does not tell us what kind of physical entity it is. It could be a physical system that cannot be recreated by a computer.

This sounds like the same kind of argument as "intelligent design" people use. ""What else could there be?" is not a religious argument, it's a perfectly reasonably question, and to suggest that there are physical systems that cannot be recreated by computers shows a lack of fundamental theoretical understanding of computation.

limist2• 17 years ago

Some very interesting points and discussion here; what surprises me is no mention yet of the major thought experiment addressing many of the AI/intelligence arguments made here - the 29-year old Chinese Room paper by John Searle:

http://en.wikipedia.org/wiki/Chinese_room

In sum,

(A1) "Programs are formal (syntactic)." (A2) "Minds have mental contents (semantics)." (A3) "Syntax by itself is neither constitutive of nor sufficient for semantics." (C1) Programs are neither constitutive of nor sufficient for minds.

req2• 17 years ago

http://lesswrong.com/lw/15a/a_note_on_hypotheticals/

limist2• 17 years ago

Thanks for the link. While I don't disagree with the general point of the linked article, the writer PhilGoetz misstates Searle's conclusion. Also, Goetz's counter-argument - that the locus of consciousness need not be the man in the room - is addressed in the Wikipedia article too:

http://en.wikipedia.org/wiki/Chinese_room#System_and_virtual...

billswift• 17 years ago

This was posted 5 days ago; 3 days before this reposting. At that time I left the comment: I thought this might be something like Eliezer's arguments against developing a GAI until it could be made provably Friendly AI, instead I just got an argument exactly like the ones in 1903 that said heavier than air flight by men was impossible - go back and read some of them, some of the arguments were almost identical. Some of the arguments are currently true, but some of them amount to "I can't do it, and no one else has done it, therefore there must be some fundamental reason it can't be done".

Why AI is a dangerous dream - interview with Noel Sharkey

Related Stories