Why Mastering Language Is So Difficult for AI

https://undark.org/2022/10/07/interview-why-mastering-language-is-so-difficult-for-ai/

by TeacherTortoise • 4 years ago

72 51 4 years ago

undark.org

sbierwagen• 4 years ago

Saved you a click: it's an interview with Gary Marcus.

CharlesW• 4 years ago

Saved you another click if you also have never heard of this person: https://en.wikipedia.org/wiki/Gary_Marcus

wpietri• 4 years ago

It's nice to see some coverage of AI that is realistic about the issues.

kelseyfrog3• 4 years ago

Replace GPT-3 with Bob, language model with person, and deep learning with learning, and you arrive at the conclusion that we're not able to determine whether people actually understand the world around them. Any utterance is indistinguishable from a sufficiently advanced computation model which simply produces the next token of text. Which is to say that the essential discriminating characteristic isn't the fact that GPT-3 runs on a computer, it's that it doesn't have a basis of reality rooted in the present.

The article correctly recognizes this, but then fails to actually make any meaningful contributions. Say the model was in fact conditioned on the basis of a present reality, would that change the interviewee's mind? The fact that it's difficult to say simply hints that the interviewer didn't really do that great of a job drilling down to the interesting questions.

Jensson• 4 years ago

Replace GPT-3 with Bob, language model with person, and deep learning with learning, and you arrive at the conclusion that we're not able to determine whether people actually understand the world around them.

Sure we can, there are diseases which greatly hampers your ability to understand the world but doesn't hamper your verbal abilities at all, Williams syndrome for example. They are great at generating nonsense stories that doesn't make sense. GPT-3 is like that but much worse.

https://en.wikipedia.org/wiki/Williams_syndrome

kelseyfrog3• 4 years ago

Right, so there is a characteristic other than computational substrate that allows us to differentiate between systems that have this ability and those that don't.

Say we apply the diagnostic criteria of William's syndrome to language models and deduce that they meet the criteria for diagnosis, then the diagnostic itself ie: the test results are the grounding for the conclusion not the computational substrate. In that way the discussion surrounding the computation substrate is a red herring or at the very least a unclear and roundabout way of grouping of the characteristic we're actually interesting in.

chongli• 4 years ago

For a particularly striking example of the opposite problem, see people with Wernicke's Aphasia [1]. Here's an example [2] of a man who had a stroke that left him with the disorder. He can speak fluently but his speech is meaningless.

Byron can understand what's happening in the world around him and can communicate it through body language (just see the warmth in his eyes and his smile) but when he speaks everything comes out in a stream of nonsense.

If we tried to apply a language-model-centric view of intelligence we would rate Byron is unintelligent. That would clearly be a mistake, as anyone can plainly see that he is intelligent.

[1] https://en.wikipedia.org/wiki/Receptive_aphasia

[2] https://www.youtube.com/watch?v=3oef68YabD0

beezlebroxxxxxx• 4 years ago

I think implying that GPT-3 and a human learn a language in the same way is an error. AI "learning" is very rigid; while human learning is often a complicated mishmash of behaviour, social feedback, reasoning, and imagination.

conditioned on the basis of a present reality

If we grant this hypothetical, we might find what it "says" fundamentally alien to us --- we might not even know it is saying anything at all.

Tijdreiziger2• 4 years ago

Neural networks are modeled after the neurons in human brains, so by definition, the way NNs learn approximates the way humans learn.

mistermann• 4 years ago

Replace GPT-3 with Bob, language model with person, and deep learning with learning, and you arrive at the conclusion that we're not able to determine whether people actually understand the world around them.

Wouldn't the fact that people's facts do not match up prove that they do not?

hinkley2• 4 years ago

I recall reading some advice to parents not to freak out if they discover their child lying at an early age. The sentiment was that lying is built on a mental model of another person, and therefore requires some substantial development of emotional intelligence to even make the attempt.

There's some romanticism in common between magicians, con artists, and fraudsters about how unclearly 'normal people' perceive the world so perhaps there's some truth to what you say. And the perpetual problem with artists and visionaries is that they see the world before everyone else does. For the one group they see things as they are now while everyone else seems to see things as they were 2, 5 years ago. For the other they see what the world could be, by exercising the tools near at hand.

One wonders then how an AI that actually understand things would be seen. As a trickster? A misunderstood intellectual? A madman?

civilized• 4 years ago

Except - as pointed out near the start of the interview - Bob knows that Biden is the current president. GPT-3 not only doesn't know, it doesn't know that it doesn't know. Despite having been trained on the entire internet, which talks quite a bit about who was president when.

Intelligence has been concisely defined as "the ability to profit from one's experiences." GPT-3 seems unable to profit from its ample experience being told who the president is.

People get so incredibly gullible when an exciting fantasy like sentient AI is in front of them. ELIZA was a laughably simple program and still fooled people.

benlivengood2• 4 years ago

Language is a communication method evolved by intelligent beings, not a (primary) constituent of intelligence. From neurology it's pretty clear that the basic architecture of human minds is functional interconnected neural networks and not symbolic processing. My belief is that world-modeling and prediction is the vast majority of what intelligence is, which is quite close to what the LLMs are doing. World models can be in many representations (symbolic, logic gates, neural networks) but what matters is how accurate they are with respect to reality, and how well the model state is mapped from sensory input and back into real-world outputs. Symbolic human language relies on each person's internal world model and is learned by interacting with other humans who share a common language and similar enough world models, not the other way around (learning the world model as an aspect of the language itself). Children learn which language inputs and outputs are beneficial and enjoyable to them using their native intelligence and can strengthen their world model with questions and answers that inform their model without having to directly experience what they are asking about.

People who don't believe the LLMs have a world model are wrong because they are mistaking a physically weak world model for no world model. GPT-3 doesn't understand physics well enough to embed models of the referents of language into a unified model that has accurate gravity and motion dynamics, so it maintains a much more dreamlike model where objects exist in scenes and have relationships to each other but those relationships are governed by literary relationships instead of physical ones and so contradictions and superpositions and causality violations are allowed in the model. As multimodal transformers like Gato get trained on more combined language and sensory input their world models will become much more physically and causally accurate which will be reflected in their accuracy on NLP tasks.

ripe2• 4 years ago

You might be right that these "multimodal" transformers, by integrating additional data from non-text sources, would be more capable than GPT-3. But I don't think that invalidates Gary Marcus's point.

The word "model" is another of those words that Minsky called "suitcase words"--- they can be used to mean many things. I don't think Marcus is saying that that LLMs have "no model", just that they don't have a model of the type that a symbolic system could have. He gives many examples of deductions that a symbolic AI system can easily do, which GPT-3 is simply incapable of.

benlivengood2• 4 years ago

To hint at what I fundamentally mean by model I'd be interested to see a symbolic model of vision. E.g. take 5125123 numbers and give them names and then follow some rules to arrive at "cat" or "dog". Image recognition, I think, is demonstrably not symbolic. Likewise most transformations from the real world to model state are not symbolic. Within model state, symbolism may have uses but Church-Turing claims that it isn't necessary.

It seems clear to me that if the CLIP-like part of Imagen or Stable Diffusion can take an image made of pixels and yield "cat" and similarly take "dog" and produce a 3D neural radiance field that we can light just like any other 3D model and recognize as a dog then there must be an accurate and useful model of both how vision works and what dogs look like and the English relationship between those two things inside the machine.

I also wish Gary Marcus was replying to Google's Minerva paper instead of GPT-3. The ability to answer multi-step symbolic problems is basically here. https://ai.googleblog.com/2022/06/minerva-solving-quantitati...

I'd also note that decades of attempts at automated theorem proving with symbolic systems haven't yielded similarly impressive results. Now we have deep learning models helping with proof search.

noobermin2• 4 years ago

Reading things like this on hacker news and twitter reminds me a lot like particle physicists unwilling to grapple with lack of evidence for SUSY. Every time someone points it out, these guys merely recite the holy texts (describe the model in gross detail with more magic words as if the criticism is because the critic don't understand it well enough), as if that addresses the critique at all. They then eventually gesture at the energy horizon, suggesting that "with more s[0] you'll eventually see my particles."

It's feels a bit similar here. There are clear limitations to AI by just NNs, namely they like Stable Diffusion must be trained on so much data, far more than any single artist will ever see, and they still fall short sometimes. With NLP it seems a little more obvious because the limitations seem to lag actual speech understood by humans still, even with the massive dataset again beyond what any single human will ever digest. There are clear issues here, merely reciting to critics what your model of intelligence is doesn't address the shortcomings these systems have which any literate person can perceive. And gesturing at the data horizon (you need even more data??) is not convincing either.

Apparently this cat (the interviewed author) has some baggage, but the things he's talking about (symbols) have already been studied and used by philosophers and linguists long before AI or computers were a thing. It helps to build upon what other researchers have already done, I don't know why NN researchers don't seem to map what they are doing to existing research on intelligence in other areas, namely cognitive science. Do NN researchers have any contact with linguists or cognitive scientists generally?

Again, being a little ignorant of his baggage, I do think a hybrid approach sounds like it would be promising. Symbolic AI failed in the 70s, NN is getting a bit of the way there now and is pretty useful but misses a few things, why go all in on just one type of approach?

[0] s is basically the center of mass energy, what particle physicists mean when they say the "energy" of an interaction (collision)

xwolfi• 4 years ago

You say far more than any artist but I m not so sure. When we spend 20 years looking at stuff before doing one passably interesting work of art (bar absolute outlying geniuses), it's not a small time nor a small training cost. It may feel easy for us but our brain never stop ingesting information for decades before being original enough.

317070• 4 years ago

This NN breakthrough in language is only about 2 years old now [0].

People are working on your symbolic logic hybrid approach, but not much interesting has happened yet. This is a bit expected, since the short amount of time, and the fast scaling these language models have done in those last two years. It's been hard to outscale them with any other approach, let alone by the particular one of symbolic logic.

I think the main discripancy with particle physics, is that every iteration of bigger language models are also significantly better, no diminishing returns in sight.

Better for concrete applications as well. There is therefore no need to claim these models are scaled for scientific reasons in the search of AI, the accountants are happy as it is.

[0] https://arxiv.org/abs/2005.14165

Der_Einzige3• 4 years ago

"Every time I fire a linguist, the performance of my speech recognizer goes up"

hackinthebochs2• 4 years ago

Yes, and the history of philosophy is replete with examples where the learned opinion on a subject stymied progress. Respect for one's intellectual forebearers is good, but being constrained by them has proven to be detrimental in areas where the learned opinion is largely speculative. The progress made by deep learning is so unlikely anything that came before, trying to shoehorn old methods and ontologies into the new paradigm would only harm progress.

dwallin• 4 years ago

It’s worth pointing out that you are not making a fair comparison when you compare the amount of data that a NN must parse to that of an individual when learning a specific new concept. A human brain is not just trained on their own lived experiences, but also the experiences of all their ancestors via the process of evolution, in addition to years and decades of general training and fine tuning.

You specifically mention Stable Diffusion, but SD has an analogue to the learning of an individual, textual inversion, which can adapt the NN to express new concepts with as few as a handful of examples.

fspeech• 4 years ago

Well put on what language is. Language encodes the delta of worlds. There are way more unsaid than what is said explicitly.

hinkley2• 4 years ago

Are you saying that GPT-3 doesn't grasp Object Permanence?

pmontra• 4 years ago

TL;DR "A parrot’s not a bad metaphor, because we don’t think parrots actually understand what they’re talking about. And GPT-3 certainly does not understand what it’s talking about."

BTW a parrot knows a big deal about the world. Not much about human language but still more about language than GPT-3. Of course a 737 doesn't know how to fly but it still flies people around the world. Landing on a branch, not much.

Gibbon1• 4 years ago

Grandmother mentioned her aunt had a parrot. One time she and her aunt are hanging out. There was a lineman up on the pole in front of the house and the bird was curiously watching him. After a bit a young lady walks by and the bird chuckles and gives out a loud wolf whistle. And the lady then begins to cuss out the lineman up on the pole. The bird giggled and then giggled off and on for the rest of the afternoon.

GPT-3 is closer to a talking toaster than a Parrot.

lostmsu2• 4 years ago

Talkie's the name, toasting's the game. Anyone like any toast?

lvxferre3• 4 years ago

[taunt]Most things that I hear about natural language processing boil down to a bunch of codemonkeying assumers trying to oversimplify linguistic phenomena that they are completely ignorant about, and yet trying to sell it as if it was the next best thing after sliced bread.[/taunt]

Anyway. There are three things that NLP is notoriously bad at:

1. Using world knowledge to interpret utterances in a given context. The article itself provides a neat example of that with United-Statian presidents.

2. Contextualisation that goes beyond the text that the utterance is found in.

3. Assessing the pragmatic purpose of an utterance.

But of course, "linguistics is useless for NLP!", the codemonkeys say.

Der_Einzige3• 4 years ago

No, it unironically IS useless. You're welcome to bark up the symbolic tree along with Chomsky, Gary Marcus, and others who are about to fade into obscurity.

Or you can embrace the bitter lesson, which is that symbolic techniques are considered harmful, like the future winners of the AI race have: http://www.incompleteideas.net/IncIdeas/BitterLesson.html

lvxferre3• 4 years ago

I could evoke Brandolini's Law on this one, but for the sake of other posters, I won't.

No, it unironically IS useless.

There's an example of why Linguistics isn't useless in _the very comment that you're replying to_: pragmatics. That is, modelling the _purpose_ of an utterance is essential to process it.

Without that pragmatic modelling, I believe that NLP will keep failing to deliver what it should, and it'll continue being just a cute toy to keep codemonkeys busy with their blackboxes. Granted, I may be wrong (I'm not stupid to claim certainty over the future), but I do not think that I am.

You're welcome to bark up the symbolic tree along with Chomsky, Gary Marcus, and others who are about to fade into obscurity.

Damn, could you lend me that crystal ball that allows you to claim certainty over the future?*

It's perfectly possible that they fade into obscurity. It's also perfectly possible that they get hailed as heroes of language processing. Or anything in between. We, in 2022, are in no position to know it. The same applies to any other linguist that is interested on natural language processing.

Specifically on Chomsky. He often spouts some stupid reductionist shit and, paradoxically, unnecessary complexity to keep that reductionism intact. Even then, his approach looks saner than "NEEDZ MOAR NODES X-D".

Or you can embrace the bitter lesson

"If I say so then you should accept it" is not a valid argument, sorry.

which is that symbolic techniques are considered harmful, like the future winners of the AI race have

Another instance of "I KNOW THE FUTURE! CHRUST MEEEE" through your comment...

Checking the link,

The ultimate reason for this is Moore's law

It's already common knowledge that Moore's Law is dead, and brute force only goes so far.

*That specific sentence is a great example highlighting the role of pragmatics. Humans can easily get what I did there; NLP should too, but it doesn't.

avgcorrection• 4 years ago

Chomsky has been saying that the way to do what we call AI is with statistics since about approximately before you were born. People wanted him to use his grammar ideas to research AI in the 50’s but he declined because he was interested in linguistics, not stats.

Not sure why you criticize a man for having different research interests than you in a completely different field.

noobermin2• 4 years ago

Thankfully someone is pushing against the hype, good for him. Thank god.

It really frustrates me when people, particularly people on here specifically respond to criticism of NNs and the like by saying nonsense like "but YOUR brain is a neural net!" as if all your brain is but mimickry after all and there's nothing else. There is this need to reduce all the mind's complexity down to a model that you seem to understand, just because you understand it, which seems like such a lazy, reductionist way to think. Reductionism in general has value but it tends to narrow the mind.

I feel like Gary Marcus is right, you don't just need the mimickry even if it plays a role, you do need "symbolic" understanding somewhere, and that makes it easier to determine whether Trump is president vs. Biden at the moment, as they talk about in the article. You do need concepts (symbols) deep down somewhere, it seems so bizarre to think you don't because no one even in their own lives lives that way or thinks that way. I mean literally right now, me reading and writing comments on hacker news is all about ideas and concepts, symbols in the philosophical sense. I feel like even if it started as mimickry when I was an infant, eventually the mimickry takes a life of its own, I figure out concepts themselves, and I no longer even work in mere mimickry, I work with symbols.

It reminds me of Baudrillard's concept (unintended pun, I think) of simulacra, it's not that simulacra are just representations of the real and are thus fake, like money represents some amount of food or gold or something and so money is fake. Money, as we all know, is in fact very real! Our daily lives are shaped around it, they shape our reality, things like loans, credit, bank accounts, stock markets eventually become things we care about. Money is therefore hardly fake. Eventually the simulacra develop a reality of their own, in fact, they become hyperreal.

Ideas are the same, and any real AI will need them.

everly2• 4 years ago

Oh, he loves to push against the hype alright. To his credit, he’s very smart - I think he sold an AI startup to Uber several years ago. And I agree with a lot of his criticisms and general AI philosophy.

Still, it’s impossible for me to shake the feeling that he just likes to hear himself talk and doesn’t want to see progress in AI.

I think I trust him more than Sam Altman though, not that anyone’s being asked to make that judgement.

ripe2• 4 years ago

Still, it’s impossible for me to shake the feeling that [Gary Marcus] just likes to hear himself talk and doesn’t want to see progress in AI.

He says he wants to see progress in AI. He wants people to get their heads out of the neural network cul-de-sac and use other tools, too. His quote:

"Imagine a world in which iron makers shouted “iron,” and carbon lovers shouted “carbon,” and nobody ever thought to combine the two [to make steel]; that’s much of what the history of modern artificial intelligence is like." [1]

An adjustment will happen when the limitations of these purely statistical methods play out and burn a few self-driving car investors. Unfortunately, it will also probably taint the entire field of AI and start another AI winter for a decade or so.

[1] https://nautil.us/deep-learning-is-hitting-a-wall-238440/

gojomo• 4 years ago

While language seemed difficult for AI for a while, it now seems to be falling quite rqpidly to recent techniques.

(It reminds me a little of the early years of aviation. In 1901 the US Navy's chief engineer called manned flight a "vain fantasy", & in 1903 the New York Times suggested flying machines would be possible in "one million to 10 million years". But it was accomplished later in 1903, with rapid improvement after that. Things that seem impossible now can become humdrum in a few decades.)

What are the current best results from Marcus, or others using his preferred approaches?

freeopinion• 4 years ago

Mastering language is difficult for humans. I don't think most people realize how often they just guess at meanings, and how often they are wrong. Especially in spoken language.

Constantly asking for clarification is exhausting and gets you labeled as pedantic and "difficult". So most people don't do it. I'd guess that a fairly high percentage of human communication results in a misunderstanding. This is largely masked by human inaction or ability to disguise their lack of understanding. The AIs have no ego and no incentive to hide their confusion.

swframe2• 4 years ago

DreamFusion can learn 3d shapes without a 3d dataset. Next step is to build a model to learn 3d motion from videos. This should get us closer to mastering language.

benjismith• 4 years ago

The central thesis of his argument, that large language models are capable only of large-scale mimicry, is flat-out wrong.

I think that people are led to believe that this system actually understands human language, which it certainly does not. What it really is, is an autocomplete system that predicts next words and sentences. Just like with your phone, where you type in something and it continues. It doesn’t really understand the world around it. And a lot of people are confused by that.
They’re confused by that because what these systems are ultimately doing is mimicry. They’re mimicking vast databases of text. And I think the average person doesn’t understand the difference between mimicking 100 words, 1,000 words, a billion words, a trillion words — when you start approaching a trillion words, almost anything you can think of is already talked about there. And so when you’re mimicking something, you can do that to a high degree, but it’s still kind of like being a parrot, or a plagiarist, or something like that. A parrot’s not a bad metaphor, because we don’t think parrots actually understand what they’re talking about. And GPT-3 certainly does not understand what it’s talking about.

I think this is an easy accusation to disprove, and we can do so with a battery of simple tests!

To test whether a language-model is only constrained by mimicry, we can just give it a task that it has never seen before. One of my favorite tasks is the construction of novel analogies.

Here's a kind of "template" that I use often for prompting novel analogies...

PROMPT: You might not realize at first, but “________” and “________” actually have a lot in common. For example,

You can fill-in-the-blanks with various different phrases and see how well the LLM is able to construct a coherent analogy. I have an LLM running in a beta environment (trained on the literary corpus at prosecraft.io), so I'll ask it to run a few of these tests for me.

For example, here's what we get when we ask for an analogy between “writing poetry” and “driving a race car”...

PROMPT + COMPLETION: You might not realize at first, but “writing poetry” and “driving a race car” actually have a lot in common. For example, your first time in a race car, you’re scared. But after a few races, you’re confident. Or in poetry, the first time you write a poem, you’re scared and unsure. But after you’ve read and improved the poem many times, you’re confident. And you are a better writer because of it.

I'm pretty sure that the training corpus doesn't include any metaphors about writing poetry and driving race cars, so the model is relying on a hierarchy of connected abstractions in order to navigate a novel pathway between two disparate concepts. In this case, the central connective-tissue of the analogy is the fact that writing poetry and driving race cars can be scary for beginners, but with more practice, the practitioner increases their skill and confidence.

It's truly astonishing that a system comprised primarily of matrix multiplications on word vectors is capable of achieving this level of abstraction on such a complex cognitive task, but there ya go.

That's what the "deep" in "deep learning" refers to... The LLM training process decomposes language understanding into a hierarchy of layered composite concepts, and the inferencing process can use those different layers of meaning to compose and manipulate novel concepts.

With the "transformer" architecture, and the "self-attention" mechanisms, an LLM is capable of learning a vocabulary of symbols, a hierarchy of relations between those symbols, and a set of operators for transforming those symbols and relations.

We can try it again with “playing chess” and “rock climbing”:

PROMPT + COMPLETION: You might not realize at first, but “playing chess” and “rock climbing” actually have a lot in common. For example, at first glance, chess is a fairly abstract and rigid game. But as you play and gain skill, you realize that certain situations require a certain decision from you. For example, you realize that certain sequences of moves are either bad or good depending on your opponent’s next move. And as your skill increases, you find the moves you’re used to thinking are bad now can be good, and vice versa. The same is true with rock climbing. And with each successive attempt, you realize how much you’re actually improvising in terms of where you place your feet on the wall, how you orient your body, what holds you, where you lean.

In this example, the AI clearly shows that it has an underlying real-world model for the meaning of both chess-playing and rock-climbing, and it was able to find a semantic parallel between those concepts: that they both involve improvising a sequence of strategic moves toward a goal.

Pretty cool, huh?!

It's true that there are still a lot of missing capabilities in these models. For example, a temporal model would certainly help LLMs answer questions about history and current events more accurately (Who is the current president of the US? Who was the previous president?)

Likewise, since most LLMs operate at the "token" level, they actually don't understand how most words are spelled or pronounced, and they're not good at answering questions about rhythm, rhyme, alliteration, etc. For example...

PROMPT: The word “whale” rhymes with the word

COMPLETION: “blue”

Huh? Obviously, the model doesn't have enough information about the pronunciation of words to solve this problem. But if we trained a model jointly on language-as-text and language-as-speech, then there's no reason the model couldn't integrate those sensory data into a unified model. And again, it would be a multi-layered hierarchical model. Not just a lookup-table to be used for mimicry.

If you've been following along with CLIP (VQGAN, CLIP-guided diffusion, Stable Diffusion, etc), which is a joint-sensory model trained on text + imagery, it's easy to imagine future projects that might train joint-sensory models on other modalities (voice, music, locomotion, time, physics, mathematics, etc).

There's no reason those models can't be trained too!

But it makes sense to start with text and imagery, and then bootstrap the other models from there.

uup• 4 years ago

Perhaps it's just me, but if I ask myself whether or not the comments on this post, or the article itself, was written by GPT-3, I'm unable to decide one way or the other.

mrjin• 4 years ago

Ask yourself that can you master something without understanding it?

Cryptonic• 4 years ago

Oh yes and it can be annoying. Especially bad it is for any other Language than English. Every evening me and my children have problems like this: "Alexa spiele Paw Patrol Folge 67 auf Spotify" "OK Paw patrol Folge 122 auf Spotify wird abgespielt..." "ALEXA STOP!!1"

I have no accent or such, plain standard German,same for my wife and Kids - Alexa drives us crazy with wrong understood episodes and sometimes titles.

Why Mastering Language Is So Difficult for AI

Related Stories