Ask HN: Opinion about Searle's Chinese Room?

https://news.ycombinator.com/item?id=10877500

by wsieroci • 11 years ago

12 28 11 years ago

news.ycombinator.com

Hi,

I would like to know your opinion about John Searle's Chinese Room argument against "Strong AI".

Best, Wiktor

lazyant• 11 years ago

we had a thread just yesterday https://news.ycombinator.com/item?id=10867791 and also last month https://news.ycombinator.com/item?id=10740748

p4wnc64• 11 years ago

I'm incredibly surprised that out of both previous threads, the piece "Why Philosophers Should Care About Computational Complexity" by Scott Aaronson [1] was only mentioned in one comment, and seemed to spark no further discussion.

Aaronson dedicates significant space specifically to the Chinese Room problem, and has a good literature review of different computer science objections (my favorite was probably the one estimating the entropy of English and coming up with space bounds on the size of all 5-minute long conversations in English).

It is one of the most comprehensive takedowns of the Chinese Room problem.

Along similar lines, Aaronson discusses later in the paper the problem "Does a rock implement every Finite-State Automaton?" popularized by Chalmers [2] and many computer science rebuttals.

[1] http://www.scottaaronson.com/papers/philos.pdf

[2] http://consc.net/papers/rock.html

makebelieve10• 11 years ago

Aaronson does not critique the chinese room argument at all, at least not in the paper cited.

dbpokorny3• 11 years ago

In theory we could rig up a quad copter with a machine gun, loudspeaker, and AI, with programming that demands that every human it encounters be interrogated with the question, "do I, a mechanical computer, have the ability to think?" and if the human gives a negative reply, it uses the machine gun on the human until the human is dead or decides to utter a positive response: "yes, this flying death machine can think".

Whether or not we say "machines can think" is a political question, and political power comes out of the barrel of a gun. If a machine can wield political power, then it can get you to say "machines can think" because truth is inherently political.

cousin_it2• 11 years ago

You can also be forced to say that 2+2=5, but that doesn't make it true. "Can machines think?" demands a substantive answer.

dbpokorny3• 11 years ago

The answer to that question depends on whether or not a robot is pointing a gun at your head. Might makes right.

makebelieve10• 11 years ago

Searle is making an argument about awareness. That a computer system is explicitly unaware of any of it's content. That it's programs are functions and the data also performs a purely functional role. In essence, computers cannot engage in acts of meaning. The programmers and users are engaged in acts of meaning.

For instance, saying a program "has a bug", is a completely misleading statement. No programs have bugs. It is impossible for a program to have a bug, just as it is impossible for a physical process to "do something wrong". Programs do what they do, just as molecular processes do what they do. The concept of error and meaning does not exist in a program, just as it does not exist in the physical universe. Meaning (and errors and bugs are a kind of meaning) are things outside programs and outside physics. When a program "has a bug" it means the programmer screwed up, not the program. A program cannot produce errors, because programs, and computer systems in general, do not have the capacity to have meaning. This is what Searle is demonstrating with his argument.

This is true for all the popular computational approaches we have today. However, because the human brain appears to function in a purely physical way, and computers function in a purely physical way, it should be theoretically possible to create a computer system that is conscious and aware of meaning just as we are. You refer to this as "Strong AI". Other refer to it as Artificial General Intelligence. I refer to this as machine consciousness. To solve the machine consciousness problem means understanding how awareness, meaning, and representation in general, works. Then building a computer system that engages in representation and instantiates awareness.

If an actual person were put into Searle's box, the person would learn chinese. Also, the person could 'intentionally' produce incorrect answers annoying the "programmers" who set the box up in the first place. But a modern computer system cannot 'intentionally' produce errors. it's completely non-sensical to talk about computers as having intention at all. programmers have intention, not computers.

Solving the intentionality problem is the other leg of machine consciousness. Elon Musk, Steven Hawking, Nick Bostrom and others make arguments about the dangers of an AI (of any variety) which may acquire intentionality and representational ability, while ignoring the actual deep problems embedded in acquiring those abilities.

Awareness, representation, and intention are so fundamental to experience that we have a very difficult time understanding when they happen and when they do not. We see a representational world all around us, but very explicitly, there are no representations at all in the physical world.

I believe machine consciousness is possible, but none of the existing approaches will get us there. Searle's chinese room is one succinct argument as to why.

The approach I am taking is a kind of metabolic computing. Where single function processes interact in some way similar to molecular interactions and those processes are developed to produce, computational structures like membranes and DNA and eventually "cells". These cells then form multi-cellular structures. These multi-cellular structures and underlying "molecular" interactions instantiate representations and representational processes, like a nervous system. A computational nervous system which embodies representation, intention, sensation, action, imagination, and because it engages in representation making, would be aware.

I would love to hear someone describe how any kind of computational approach can produce meaning inside a computer system. We produce meaning and representations so easily; it's hard to understand the difference of perspective necessary to see how representations must form. If someone has an easier approach than the one I am taking, I would be very interested in seeing how they solve the problems of meaning and intention with code.

tstactplsignore• 11 years ago

I can't see your side of the chinese room. A sufficiently complex digital system can simulate any analog one; the human brain is an analogy system; therefore, a digital system could simulate the human brain; therefore, if a human brain is conscious, or can produce meaning, a digital program can do the same thing. The simulation might be probabilistic, it might be stochastic, but it'd still be a digital simulation. We could simulate it atom by atom with a computer the size of the sun if that's what it'd take, but it could still be done. I don't see how the specific "hardware" that the mind runs on, whether made of membranes or transistors, has anything to do with it, nor the level of abstraction. How is there anything more to the discussion?

makebelieve10• 11 years ago

a system that simulates some process or system is not the process or system it simulates.

Just take writing programs: Can we simulate the process of writing programs? Could we create a system that writes and compiles programs? What about writing programs which contain errors? Could that system recognize the errors in those programs it wrote and correct the errors? could it write programs and then optimize those programs? Or rewrite the programs to make them more efficient or tweak them to do other tasks? Does "simulation" actually help us create a computer that can write programs at all? If so, how?

If you know how to write a program that can write and optimize programs, Google will hire you tomorrow! And it can't be that hard. It's just combining ascii characters together into combinations based on some rules...

Error making is the essence of actual learning, because it is a component of comprehension. Simulation, automata theory, mathematics itself, do not address issues of comprehension or error recognition. How can a computer system make, recognize, and correct errors? Errors do not actually "exist". Errors are things we apprehend but which have no obvious physical counterpart.

We do some simulation of atoms, but it is laughably inefficient. Think about the simulations we do to figure out protein folding. Protein folding is going on in every neuron with each synaptic firing. Protein folding not performed by a rule or an extrinsic function, but is an intrinsic process of the molecule itself. For instance, how do a few molecules of LSD produce such an incredible change in actual experience? How would you go about simulating psychedelic phenomena? How would you go about simulating wave lengths of light as colors? How would you simulate colors (as in dreams or imagination) without the corresponding wavelengths of light? How would you simulate what sound is?

We can certainly produce and record vibrations with speakers and microphones attached to computers. But what is the experience of sound? when you hear someone's voice in your head, what is that? it's not a vibration, it's not a string of ascii characters, it's not a wav file.

What you experience looks easy because experiences occur effortlessly to you. Now try to write a program that can be aware of something, that can think about something, that has experiences. That is Searle's point, our computers have no experiences at all.

If I say: "Don't forget to brush your two teeth with your toothbrush." You will understand what the "twos" mean. A computer has no comprehension of to, too, two, 11, or 2. It's not just too hard for a computer to do, it's a categorically different problem. To get a better understanding of these problems you could read about qualia https://en.wikipedia.org/wiki/Qualia. and then wonder how you could get a computer to see magenta. And then wonder how you could get it to like Pink (the singer).

wsieroci3• 11 years ago

I agree with you totally, if you wanted to talk about this more and your approach then my e-mail is: w.sierocinski-a-gmail.com

cousin_it2• 11 years ago

Some people are uncomfortable with the idea that brains can work on "mere" physics. But what's the alternative? A decree from Cthulhu saying human brains are the only devices in the universe that can communicate with consciousness, and all other objects are doomed to be dumb matter? I think that's really unlikely. So it seems like we must bite the bullet and admit that the computation implemented in the Chinese room exhibits genuine understanding. I'm not completely certain, because we don't really understand consciousness yet and we could always find new evidence, but the evidence so far points in one direction.

fallingfrog• 11 years ago

Well, a Chinese room can simulate a computer, a computer can run a physics simulation and a brain is a physical thing and brains are conscious. So it's pretty open and shut imho; yes, a Chinese room can be conscious. That much is clear. Now what it means to say that something is conscious is another question. I've never seen the word rigorously defined.

chrisdevereux• 11 years ago

Its popularity in undergraduate philosophy courses reflects the fact that it is easy to criticise more than it telling us something interesting about consciousness.

makebelieve10• 11 years ago

you might want to watch this lecture by Searle at Google: https://www.youtube.com/watch?v=rHKwIYsPXLg

wsieroci3• 11 years ago

I saw it and I am constantly pretty amazed just how people can't grasp his argument.

Ask HN: Opinion about Searle's Chinese Room?

Related Stories