Natural Language Processing and the human brain

http://thetokenizer.com/2012/12/25/understand/

by ohadfrankfurt • 14 years ago

29 14 14 years ago

I wish there was a mathematical theory for what 'understanding' is. For instance, I too suspect that whatever we call a function of a system, something that the system does, can not simply occur by random parts coming together.

To be clear, I don't think a layer of sands has the function of filtering water. It does filter water, but it does it in the virtue of static property, in the first of the geometry of the system not the mechanics.

There is something about systems that do something that I can't name, having limited knowledge on the subject.

Could anyone give me a hint on research done on this?

RodgerTheGreat• 14 years ago

Er- you seem to be describing the concept of an "emergent property" of a system, but beyond that I don't really understand what you're asking.

pirateking2• 14 years ago

I believe the phenomenon you are referring to is emergence[1][2].

Most of the research in this area I have come across is related to education and epistemology. I would recommend reading works of Seymour Papert and Jean Piaget.

I think the mathematical theory you are looking for to explain "what understanding is", is actually the philosophical motivation for the field of mathematics itself. At least it is why I am really interested in studying it these days.

[1] http://en.wikipedia.org/wiki/Emergence

[2] http://plato.stanford.edu/entries/properties-emergent/

cristianpascu3• 14 years ago

Emergence sounds right. But I think there's something more. The properties of a car are emergent on the properties of its part. But the function of the car is not necessarily so. There is intent, something that a pile of rocks does not have, although it has emergent properties.

gatlin• 14 years ago

As another commenter said, this sounds like the basis of math (or CS) itself. Understanding is emergent. Inference and learning are, to me, the pillars of it and we can do those. Understanding is the ability to internalize some idea and make correct inferences about it. But then what is an idea? And down the rabbit hole I go ...

pbateman• 14 years ago

Two points:

1) Kahneman's model is probably too high level to apply to solving NLP problems. It's an interesting abstraction, not a road map to human reasoning.

2) System 2 is Effortful Reasoning, not just doing math. For example, if you were engaging system 2 to plan how long a project was going to take system 2 would think about what kind of questions were relevant to generating an answer (how long have similar projects taken, are there any differences between this project and the others, etc.) and then think about how to combine these to produce an answer. Computers do not perform this kind of reasoning well at all.

jonsen• 14 years ago

System 2 is not a stand alone system. System 2 cannot function without an underlying System 1. Comparing a computer with a brain System 2 is a weak analogy.

beering• 14 years ago

I think this is quite similar to Moravec's paradox, where tasks that are "natural" to the brain are the most difficult to simulate on a computer, seemingly difficult tasks are easier on a computer. It makes sense if you consider that the mechanisms of each are fundamentally different, and each exploits its own strengths.

Planning how long a project takes given data about similar projects could readily be done on a computer (e.g. case-based reasoning or other machine learning). Having seen how bad many people are at planning, there's a good chance a computer could do just as well as an average human.

woodchuck642• 14 years ago

It kind of makes intuitive sense that "understanding" is a graph of nodes representing arbitrary (meaningless) concepts with edges doing all the work of imbuing meaning.

So how do we test this idea with a graph large enough and fast enough to approach the complexity of the brain? All computer architectures seem to deeply resist the graph architecture by forcing edges/nodes of large graphs to be sequentially accessed over a set of limited memory bus lines. Large computer networks force edges/nodes to be sequentially accessed over network access points, hubs and switches. Everywhere we look, graph traversal and transformation is unavoidably sequentialized at critical moments on today's hardware. What's the solution?

redwood• 14 years ago

While I certainly can't answer your question, you did make me think of something interesting:

Since the internet as a whole acts more and more like a slow brain, and since the internet influences how we act as people more and more (e.g. everything from groupon deals to the Arab Spring), perhaps humanity as a whole is beginning to act like a decentralized network brain!

I recognize there's nothing particularly novel or useful about what I've just said. It's just an interesting visualization.

woodchuck642• 14 years ago

Reminds me of Eric Schwitzgebel's question: is the United States conscious? http://schwitzsplinters.blogspot.com/2011/10/is-united-state... (and http://www.youtube.com/watch?v=EbQi0tVNsdo)

dschiptsov• 14 years ago

Just two systems is too simple, too naive.

The Society of Mind by Marvin Minsky is a must read for anyone who is trying to speak about AI.

The notion that there are quick and slow systems are old one - reflexes vs. recognition is the simplest example - recognition is too slow.

But there are not two or three systems, there are.. I cannot say how many, but if we believe that we have a distinct "subsystems" to recognize eyes, and mouths and faces by matching some cues, there must be vast hierarchies of such sub-systems (agencies in Minsky terminology).

So, some processes (agencies) are "slow" some are "fast", some works with quick-and-dirty data, such like a sudden motion catches our attention, before we're able to recognize what's going on..

The notion that each word we see does some "priming", pre-fetching in CS terms leads to the vastly complex view of how even seemingly simple tasks are performed.

So, all we can do is to recognize familiar (known in advance) patterns and look at its weights as modern computer translation services do.

Natural Language Processing and the human brain

Related Stories