Recent Advances in Natural Language Processing

https://deponysum.com/2020/01/16/recent-advances-in-natural-language-processing-some-woolly-speculations/

by saadalem • 6 years ago

278 176 6 years ago

p1esk6• 6 years ago

Note this is pre-GPT-3. In fact I expect GPT-4 will be where interesting things start happening in NLP.

curiousgal• 6 years ago

I honestly don't get where the big deal is with NLP. So far the most useful application has been customer support chatbots and those still don't rise to the level of having an actual human that can understand the intricacies of your special request.

p1esk6• 6 years ago

those still don't rise to the level of having an actual human

What if they did? Do you see where the big deal is now?

thekyle• 6 years ago

If a chatbot was as good as a human then would you notice it was a chatbot?

feral• 6 years ago

I work on such a bot @Intercom.

A lot of support requests aren't actually intricate and special: there's almost always loads of simple requests that come in again and again.

When you ask a request like that and get the answer instantly, that really is ML delivering value.

You mightn't think it, but a lot of people work in customer support, and spend a lot of time answering rote questions again and again.

People talk about how much hype there is, or the chance of another AI winter. Yes there's a ton of hype - but I think they aren't considering the real value already being delivered this time around.

Everyone is excited about GPT3, but there's already been amazing progress in practical NLP already, over the last few years.

ben_w• 6 years ago

Current NLP is bad. Still useful (Google search increasingly feels like it is doing NLP to change what I asked for into what it thinks I meant) but bad. A hypothetical future “perfect” NLP can demonstrate any skill that a human could learn by reading, and computers can read so much more than any given human.

MiroF12• 6 years ago

That's just not even close to the only or most useful application.

I use NLP and associates s2s techniques every day. I struggle to see how so many people don't see the obvious benefits deep inference is bringing to stuff all around them.

claudeganon• 6 years ago

This past week, I used this new edit distance library to identify quasi-duplicates in a large dataset:

https://github.com/Bergvca/string_grouper

Saved me hours of work because all the Levenshtein implementations are pretty slow and I’m going to need to rerun the analysis as the dataset grows.

I don’t know about consumer-facing tools, but NLP stuff has helped me solve all kinds of tedious data problems at work.

narag• 6 years ago

Has it happened that a "thought experiment" has become a real experiment ever?

dane-pgp• 6 years ago

Most historians think that this was actually a thought experiment:

https://en.wikipedia.org/wiki/Galileo's_Leaning_Tower_of_Pis...

An equivalent experiment was famously carried out for real in 1971 on the surface of the Moon.

simonh• 6 years ago

In a sense every experiment starts as a thought experiment.

exo-pla-net• 6 years ago

Not sure exactly what you mean, but Einstein developed relativity largely through thought experiments. And relativity has been verified by real experiments.

jvanderbot• 6 years ago

I'd go one step further: Humans themselves don't understand anything, we are just good at constructing logical-sounding (plausible, testable) stories about things. These are mental models, and it's the only way we can make reasonable predictions to within error tolerances of our day-to-day experience, but they are flat-out lies and stories we tell ourselves not based on a high-fidelity understanding of anything.

Rumination, deep thinking, etc is simply actor-critic learning of these mental models for story-telling.

soulofmischief• 6 years ago

Mental models are not lies.

"The car is red" is not a lie just because I didn't phrase it internally as "The car reflects photons of a frequency of around 700nm".

We have to be able to simplify and internalize simplified models in order to make any sense of anything. It's the same reason your eyes only have a focal point in the dead center: attention requires vast amounts of processing power.

To reiterate, a simplification is not a lie. Especially not a flat-out lie.

Turing_Machine• 6 years ago

Yes. Bridges are designed using old-school Newtonian mechanics, without (in general) considering relativistic or quantum effects at all, but they stand up nonetheless.

That wouldn't be the case if they were built based on "flat-out lies".

Edit: the sticking point here is the use of the word "lie", I think.

A lie is a falsehood told with intent to deceive.

Thus, novels are not "lies", despite being works of the imagination. Everyone knows that the people in Moby-Dick or Wuthering Heights were made up.

Neither is a simplified model a "lie" if it is close enough.

All engineers know that the steel and concrete components of a bridge have quantum and relativistic effects going on inside them, so ignoring those effects isn't a "lie" in any meaningful sense.

It just doesn't matter for the purpose at hand.

cscurmudgeon2• 6 years ago

But you claim to understand right in this comment how humans understand other things.

Isn't that self-contradictory?

p1esk6• 6 years ago

He's just not very good at constructing logical-sounding (plausible, testable) stories about things :)

xdavidliu2• 6 years ago

no he did not claim so. His claim is just as much a lie as all other claims. Lies can be valuable, even if they are not strictly true.

runT1ME• 6 years ago

Do current NLP systems understand arithmetic, and can the do it with unfamiliar numbers they've never seen on? If not, I'd think that your theory is demonstratably false, as a child can extrapolate mathematical axioms from just a few example problems, whereas NLP models are not able to do so.

glenstein2• 6 years ago

Do current NLP systems understand arithmetic, and can the do it with unfamiliar numbers they've never seen on?

I don't know if that question is rhetorical or not, but GPT-3 can do basic math for problems it has not been directly trained on, and there's been a fair amount of debate, including right here at hn, about what the takeaway is supposed to be.

Davidzheng• 6 years ago

I question the premise here. Can young children (say <5) who don't know arithmetic at all learn the axioms from just a bunch of examples? This isn't how children are taught arithmetic; they are taught the rules/algorithm not just a bunch of data-points. Do you have a source on the claim?

joe_the_user8• 6 years ago

The funny thing with GPT-3 is that it comes up with sentences that seem to make correct deductions using arithmetic or other submodels of reality but it will blithely generate further sentences that contradict these apparent understandings. It's impressive and incoherent all at the same time.

bananaface• 6 years ago

I get what you're saying, but I think this is a side-effect of saving thinking in abstractions. We use abstractions to plug holes to avoid having to drill down on every concept we're exposed to.

That can look like a surface-level linguistic understanding, but it's not, it's a surface-level abstraction. It's not arbitrary, it has structure, and when you flesh it out you're fleshing it out with actual abstract structure, not just painting over the gaps with arbitrary language.

mqus• 6 years ago

Not a single mention if this is only applicable to english or to other natural languages. Afaict this mostly lists advancements in ELP (english language processing), Especially the Winograd schema (or ar least the given example) seems to be heavily focused on english.

Relevant article for this problem: https://news.ycombinator.com/item?id=24026511

MiroF12• 6 years ago

But there's no reason the models are english specific...

woodson• 6 years ago

In a certain way, they could be, simply because their structure may work better on the type of language that English is. It may not work as well for languages that exhibit other grammatical patterns (e.g. morphologically rich languages, or those that exhibit superficially more flexible word order which nonetheless conveys information about topic/focus).

akerro• 6 years ago

I'm 100% sure NLP will do the same mistake with non-English language as face recognition did with faces of black people.

I'm taking bets.

m0zg2• 6 years ago

For some of them, there is. Russian (my mother tongue), for example, has the kind of morphology that makes linguists' hair stand on end. Also, verbs and nouns are gendered (and therefore they must agree, and you must know what everything refers to at all times, often pretty far away from where the noun was first mentioned), and there are declensions on various parts of speech, and they must agree as well. And words can be freely formed out of several roots, prefixes, and suffixes. For any of this to be understood or generated, your model has to be able to model all of that, which is much harder than modeling English.

YeGoblynQueenne20• 6 years ago

Yes, there is. A "model" is a set of parameters optimisted by some algorithm or system trained on the data in a specific dataset. Thus, a language model trained on a dataset of English language examples is only capable of representing English language utterances, not, e.g. French, or Greek, or Gujarati utterances. Diagrammatically:

  data --> system --> model

What is not necessarily English-specific are the systems used to train different language models, at least in theory. In practice, systems are typically hand-crafted and fine-tuned to specific datasets, to such a degree that most of the work has to be done anew to train on a different dataset.

bloaf• 6 years ago

I know that there are allegedly NLP algorithms for generating things like articles about sports games. I assume they have something more like the type signature (timeline of events) -> (narrative about said events)

What this article is about is more (question/prompt) -> (answer/continuation of prompt)

Does anyone know if there is progress in the (timeline of events) -> (narrative about said events) space?

082349872349872• 6 years ago

For an intermediate goal on the way to sports games, the financial press version of (timeline of events) -> (narrative about said events) could be tackled as a memoryless system.

YeGoblynQueenne20• 6 years ago

> The Winograd schema test was originally intended to be a more rigorous replacement for the Turing test, because it seems to require deep knowledge of how things fit together in the world, and the ability to reason about that knowledge in a linguistic context. Recent advances in NLP have allowed computers to achieve near human scores:(https://gluebenchmark.com/leaderboard/).

The "Winograd schema" in Glue/SuperGlue refers to the Winograd-NLI benchmark which is simplified with respect to the original Winograd Schema Challenge [1], on which the state-of-the-art still significantly lags human performance:

The Winograd Schema Challenge is a dataset for common sense reasoning. It employs Winograd Schema questions that require the resolution of anaphora: the system must identify the antecedent of an ambiguous pronoun in a statement. Models are evaluated based on accuracy.

WNLI is a relaxation of the Winograd Schema Challenge proposed as part of the GLUE benchmark and a conversion to the natural language inference (NLI) format. The task is to predict if the sentence with the pronoun substituted is entailed by the original sentence. While the training set is balanced between two classes (entailment and not entailment), the test set is imbalanced between them (35% entailment, 65% not entailment). The majority baseline is thus 65%, while for the Winograd Schema Challenge it is 50% (Liu et al., 2017). The latter is more challenging.

https://nlpprogress.com/english/common_sense.html

There is also a more recent adversarial version of the Winograd Schema Challenge called Winogrande. I can't say I'm on top of the various results and so I don't know the state of the art, but it's not yet "near human", not without caveats (for example, wikipedia reports 70% accuracy on 70 problems manually selected from the originoal WSC).

__________

[1] https://www.aaai.org/ocs/index.php/KR/KR12/paper/view/4492

YeGoblynQueenne20• 6 years ago

Separate comment because separate point.

To my mind, all this benchmark-beating is just pointless faffing about and not a little frustrating to see that this is where the current state of the art is at. NLP systems keep getting better and better at beating benchmarks and yet we still don't have artificial agents that can hold an intelligent conversation, even in a restricted context.

Or rather, we had such programs since the late 1960's, with no one else's than Terry Winograd's own doctoral thesis system, SHRDLU [1], but it seems that in the race to build powerful classifiers, we forgot how to build systems that can interact intelligently with a human.

Here's a small excerpt from SHRDLU (taken from the wikipedia page on SHRDLU, originally in an appendix in Terry Winograd's thesis):

    Person: Is there a large block behind a pyramid?
    Computer: YES, THREE OF THEM: A LARGE RED ONE, A LARGE GREEN CUBE, AND THE BLUE ONE.
    Person: Put a small one onto the green cube which supports a pyramid.
    Computer: OK.
    Person: Put the littlest pyramid on top of it.
    Computer: OK.
    Person: Does the shortest thing the tallest pyramid's support supports support anything green?
    Computer: YES, THE GREEN PYRAMID.
    Person: What color is it?
    Computer: BY "IT", I ASSUME YOU MEAN THE SHORTEST THING THE TALLEST PYRAMID'S SUPPORT SUPPORTS.
    Computer: RED

Note that this is not just mindless text generation. The conversation is held in the context of a "blocks world" where a robot with a single hand and eye ("Computer" in the excerpt above) moves blocks of various shapes and colours around, as directed by a human user in free-form natural language. When the Computer says "OK" after it's directed to "put the littlest pyramid on top of it" it's because it really has grabbed the smallest pyramid in the blocks world and placed it on top of the small block in an earlier sentence, as the Person asked. The program has a memory module to keep track of what ellipses like "it", "one" etc refer to throughout the conversation.

SHRDLU was a traditional program hand-crafted by a single PhD student- no machine learning, no statistical techniques. It included, among other things, a context-free grammar (!) of natural English and a planner (to control the robot's hand) all written in Lisp and PLANNER. In its limited domain, it was smarter than anything ever created with statisical NLP methods.

______________________

[1] https://en.wikipedia.org/wiki/SHRDLU

quotemstr• 6 years ago

Why is it surprising that a CFG can approximate a subset of English grammar?

trott• 6 years ago

Why is it surprising that a CFG can approximate a subset of English grammar?

"Colorless green ideas sleep furiously" is a famous example of a sentence that is grammatical, but meaningless. The goal of SHRDLU was far more ambitious than approximating English grammar.

liuliu• 6 years ago

We knew hand-crafted program in limited domains can work for NLP, computer vision and voice recognition long time ago. The challenge is always, that the limited domain can be extremely limited, and to get anything practically interesting requires a lot of human involvement to encode the world (expert system).

Statistical methods traded that. With data, some labelled, some unlabelled and some weakly-labelled, we can generate these models with much more efficient human involvement (refine the statistical models and labelling data).

I honestly don't see the frustration. Yes, current NLP model may not be the "intelligent agent" everyone looking for yet to any extent. But claiming it is all faffing and no better than 1960s is quite a stretch.

joe_the_user8• 6 years ago

The thing that qualifies as "faffing" in my opinion isn't the statistical NLP programs, which "are what they are" but claims of progress based primarily on benchmarks, as YeGoblynQueene rightly states.

And "limited domain" is relative. A program that gets many aspects of language right talking about a small world might be said to have a larger domain than a program that outputs stream of semi-plausible, semi-gibberish involving the whole of English language. Which again isn't saying modern NLP is nothing but rather we should a somewhat better to talk of it (and machine learning generally) than "hitting benchmarks".

YeGoblynQueenne20• 6 years ago

> We knew hand-crafted program in limited domains can work for NLP, computer vision and voice recognition long time ago.

Yes, we did. So- where are the natural language interfaces by which we can communicate with artificial agents in such limited domains? Where are the applications, today, that exhibit behaviour as seemingly intelligent as SHRDLU in the '60s? I mean, have you personally seen and interacted with one? Can you show me an example of such a modern system? Edit: Note again that SHRDLU was created by a single PhD student with all the resources of ... a single PhD student. It's no stretch to imagine that an entity of the size of Google or Facebook could achieve something considerably more useful, still in a limited domain. But this has never even been attempted.

Yes, it is faffing about. Basically NLP gave up on figuring out how language works and switched to a massive attempt to model large datasets evaluated by contrived benchmarks that serve no other purpose than to show how well modern techniques can model large datasets.

ericbarrett• 6 years ago

I read grandparent's post as saying that despite all the research and untold amounts of compute power poured into NLP over the decades, its practitioners have yet to address the original real-world goals that led us to study it in the first place. Missing the forest for the trees, if you will.

(I don't assert that it has hasn't. I know just enough about the topic to see how little I know. But it seems, from the outside, a valid criticism, and not one unique to the field.)

tmalsburg2• 6 years ago

Since this post is receiving downvotes, I'd like to know which part of the HN guidelines it is violating.

ascavalcante80• 6 years ago

NLP is great for many things, but, from my own experience as a NLP developer, machines are not even close to understand human language. They can interpret well some kind of written speech, but they will struggle to grasp two humans speaking to each other. The progress we are make on building chatbots and vocal assistants is mainly due to the fact We are learning how to speaking to the machines, and not the contrary.

FiberBundle2• 6 years ago

I found the science exams results interesting and skimmed the paper [1]. They report an accuracy of >90% on the questions. What I found puzzling was that they have a section in the experimental results part where they test the robustness of the results using adverserial answer options, more specifically they used some simple heuristic to choose 4 additional answer options from the set of other questions which maximized 'confusion' for the model. This resulted in a drop of more than 40 percentage points in the accuracy of the model. I find this extremely puzzling, what do these models actually learn? Clearly they don't actually learn any scientific principles.

[1] https://arxiv.org/pdf/1909.01958.pdf

wrs• 6 years ago

I would be interested in hearing the results from humans presented with adversarial answer options. You may say that a machine learning correlations between words isn’t really learning science, but I wonder how many human students aren’t either, just pretty much learning correlations between words to pass tests...

FiberBundle2• 6 years ago

They do give an example of a question, in which the model chose an incorrect answer in the adversarial setting:

"The condition of the air outdoors at a certain time ofday is known as (A) friction (B) light (C) force (D)weather[correct](Q) joule (R) gradient[selected](S)trench (T) add heat"

I assume this might be characteristic for other questions as well, although I don't know anything about the Regents Science Exam and whether there are multiple questions about closely related topics.

jacobwilliamroy5• 6 years ago

Most multiple choice math problems can be completely circumvented by simply finding the digital root of the expression in the problem. I was surprised to find this to be true, even on college entrance exams.

teej3• 6 years ago

That’s the thing. Machine Learning is a misnomer, the models don’t “learn” anything about the domain they operate in. It’s just statistical inference.

A dog can learn to turn left or to turn right for treats. But they don’t understand the concept of “direction”, their brain isn’t wired that way.

Machine learning models perform tricks for treats. The tricks they do get more impressive by the day. But don’t be deceived, they aren’t wired to gain knowledge.

plutonorm5• 6 years ago

My god if I hear this argument one more time I'm going to pop. What on earth gives you the idea that you are something more than statistical inference?

0-_-02• 6 years ago

Doesn't your brain derive knowledge through statistical inference? How else? I can imagine the AGI of the future (having perfected knowledge gathering through statistical inference) some day saying: "Humans perform tricks for treats. The tricks they do are impressive. But don’t be deceived, they aren’t wired to gain knowledge."

0-_-02• 6 years ago

Adversiarial training means that they specifically search for answers that the network would misunderstand. If this only leads to a 40 percent loss (the network still answers correctly 50% of the time) I still consider that remarkable.

Choosing the best from 8 answers where some of them were adversarially derived should be equivalent to choosing the best from all possible answers, of which there could be tens of thousands. How would a human do in that situation?

Although the kinds of mistakes the network makes seem like a mistake you would never do (i.e. you wouldn't call the condition of air outdoors gradient), the opposite could also be true, that it would easily answer questions you would have a problem with.

mannykannot4• 6 years ago

I admire the way that you have reinterpreted the issue, but I don’t buy it.

For one thing, I can imagine a knowledgeable human working through ten thousand alternatives and picking the best, (if, objectively, there is one) - it would just take a long time. On the other hand, It does not seem obvious to me that current NLP systems would do better than humans on such a task, and nor is it obvious to me that one can conclude that from the fact that the machine sometimes ignores adversarial examples (if the assumption is that most of the ten thousand would be de-facto adversarial, that is a lot to not make one mistake on; that’s not so much a problem for a human with an understanding of the issue, and who might well come up with the right answer unprompted.)

This is probably all moot, however, as the most obvious way to compare the machine results to those of humans is to give both the actual tests in question, rather than substitute a dubious “equivalent” test.

MiroF12• 6 years ago

Adversarial examples probably exist for humans too, they are just less easy to find since we can't easily backprop through human judgements like we can a bunch of array multiplications and dot products.

Doesn't mean that they are not "actually" learning any more than we are.

callmekit2• 6 years ago

These are "adversarial answers" - additional answers to a multi-choice test. If you know the correct answer to the test, adding more possible answers shouldn't make you change your selection.

rvense7• 6 years ago

I think the point about language being a model of reality was interesting. I have an MA in linguistics including some NLP from about a decade ago and was looking at a career in academic NLP. I ultimately left to become a programmer because (of life circumstances and the fact that) I didn't see much of a future for the field, precisely because it was ignoring the (to me) obvious issues of written language bias, ignorance of multi-modality and situatedness etc. that are brought up in this post.

All of these results are very interesting, but I'm not really feeling like we've been proved wrong yet. There is a big question of scalability here, at least as far as the goal of AGI goes, which the author also admits:

Of course everyday language stands in a woolier relation to sheep, pine cones, desire and quarks than the formal language of chess moves stands in relation to chess moves, and the patterns are far more complex. Modality, uncertainty, vagueness and other complexities enter but the isomorphism between world and language is there, even if inexact.

This woolly relation between language and reality is well-known. It has been studied in various ways in linguistics and the philosophy of language, for instance by Frege and not least Foucault and everything after. I also think many modern linguistic schools take a very different view of "uncertainty and vagueness" than I sense in the author here, but they are obviously writing for non-specialist audience and trying not to dwell on this subject here.

My point is, when making and evaluating these NLP methods and the tools they are used to construct, it is extremely important to understand that language models social realities rather than any single physical one. It seems to me all too easy, coming from formal grammar or pure stats or computer science, to rush into these things with naive assumptions about what words are or how they mean things to people. I dread to think what will happen if we base our future society on tools made in that way.

skybrian2• 6 years ago

These tools seem to be getting pretty good at fiction. In particular, playing around with AI Dungeon, it doesn't believe anything, or alternately you could say it believes everything at once. It's similar to a library that contains books by different authors, some fact, some fiction. Contradictions don't matter. Only local consistency matters, and even then, not that much.

Unfortunately, many people want to believe that they are being understood. But, on the bright side, this stuff seems artistically useful? Entertainment is a big business.

rvense7• 6 years ago

many people want to believe that they are being understood

Not only that, they also want to understand. I read some computer poetry a few years ago that worked. The computer did not intend any meaning, but I found it anyway.

And of course I did. This is how language works. We assume that people (and by extension, the texts they write) follow a set of rules which structures how we engage with them in a successful manner. Paul Grice formulated this as the cooperative principle in his study of everyday language, but I believe it is the exact thing that is at play when people meet NLP:

https://en.wikipedia.org/wiki/Cooperative_principle

joe_the_user8• 6 years ago

I'd claim actual involves language both a general shared reality and quite concrete and specific discussions of single physical and logic facts/models. Some portion of language certainl looks mostly like a stream of associations. But within it is also are references to physical reality and world model and the two are complexly intertwined (making logical sense is akin to a "parity check" - you can for a while without it but then you have to look at the whole to get it). I believe one can see this in a GPT paragraph, where the first two sentences seem intelligence, well written but the third sentence contradicts the first two sentences sufficiently one's mind isn't sure "what's being said" (and even here, our "need for logic" might be loose enough that we only notice the "senselessness" after the third logical error).

rvense7• 6 years ago

In human language I think physical reality is always a few layers out. Language is social, first and foremost, and naming something is not neutral. We can hardly refer to single objects directly, we mostly do it through their class membership, which always, always include a whole range of associations, reductions, metaphor, etc. that are cultural.

fernly• 6 years ago

it is extremely important to understand that language models social realities rather than any single physical one...

Oh gosh yes. Gorge the ML tool with the right streams of text and you'd have a glib anti-vax, white supremacist, flat-earth generator pouring out endless logorrheic twaddle.

Oh wait... could that already... it would explain so much...

hinkley• 6 years ago

It's probably worse than that.

It's a model of reality as filtered through the human brain, with all of its neuroses, most of which we have only begun to model.

Aren't we stuck on things like sarcasm? How are you going to model everything from confirmation bias to undisclosed PTSD, in order to have a prayer of filtering the noise from actionable information?

skybrian2• 6 years ago

Darn, based on the title, I was hoping for an overview of recent research.

Lots of people are having fun playing with GPT-3 or AI Dungeon, myself included, but it seems like there is other interesting research going on like the REALM paper [1], [2]. What should I be reading? Why aren't people talking about REALM more? I'm no expert, but it seems like keeping the knowledge base outside the language model has a lot going for it?

[1] https://ai.googleblog.com/2020/08/realm-integrating-retrieva... [2] https://arxiv.org/abs/2002.08909

nl• 6 years ago

REALM is amazing, and Google's PEGASUS[1] taught itself to count between 2 and 5 (which is mindblowing).

Basically the NLP groups from Google and FB[2][3] are always worth watching.

[1] https://ai.googleblog.com/2020/06/pegasus-state-of-art-model...

[2] https://ai.facebook.com/blog/covost-v2-expanding-the-largest...

[3] https://ai.facebook.com/blog/introducing-a-new-large-scale-d...

MiroF12• 6 years ago

I think there's a pretty large gap between what people in academia NLP/industry labs find interesting and what the bitcoing/VC/whatever people on twitter seem to find interesting.

realm is definitely cool, and i agree that techniques like this are the way forward.

_emacsomancer_2• 6 years ago

A bit I found rather strange, on the language-side:

This is to say the patterns in language use mirror the patterns of how things are(1).
(1)- Strictly of course only the patterns in true sentences mirror, or are isomorphic to, the arrangement of the world, but most sentences people utter are at least approximately true.

Presumably this should really say something like "...but most sentences people utter are at least approximately true of their mental representation of the world."

rllin• 6 years ago

the thing is humans have most efficiently encoded (in detail) reality in text. humans already highlight what is worth encoding about reality.

for example, you can finetune gpt-2 to have an idea of sexual biology by having it read erotica. just like how you can have a model learn the same by watching porn. but it is much more efficient to read the text, since there is much less information that is "useless"

rland3• 6 years ago

Models are transitive- if x models y, and y models z, then x models z. The upshot of these facts are that if you have a really good statistical model of how words relate to each other, that model is also implicitly a model of the world.

This right here is a great way of putting the success of GPT-3 into context. We think GPT is smart, because when it says something eerily human-like we apply our model of the world onto what it is saying. A conversation like this:

Me: So, what happened when you fell off the balance beam?
GPT: It hurt.
Me: Why'd it hurt so bad?
GPT: The beam was high up and I feel awkwardly.
Me: Wow, that sounds awful.

In this conversation, one of us is thinking far harder than the other. GPT can have conversations like this now, which is impressive. But only I can model the beam, the fall, and the physical reality. When I say "that sounds awful," I actually do a miniature physics simulation in my head, imagining losing my balance and falling off a high beam, landing, the physical pain, etc. GPT does none of that. In either case, when it asks the question or when it answers it, it is entirely ignorant of this sort of "shadow" model that's being constructed.

Generalizing a bit, our "shadow" model of reality in every single domain is far more powerful than language's approximation. That's why we won't be able to use GPT to do a medical diagnosis or create a piece of architecture or whatever else people are saying it's going to do now.

ilaksh• 6 years ago

Right. I recently saw a podcast with Dileep George of Vicarious AI. I don't know if his specific techniques will work fully generally but at the high level he is talking about something quite similar to what you are saying in terms of grounding language understanding and having real models of the world. So I am definitely following him.

More broadly there is a growing group of researchers who are working on trying to achieve better world modeling (although of course there have been many for decades, it just seems that the number who are coming from deep learning towards AGI with more emphasis on world modeling is increasing).

Such as Ferran Alet. Sort of the Tenenbaum school I guess. Or Melanie Mitchell has been saying similar for quite awhile. They have recent talks on Good AI's YouTube channel.

dwohnitmok2• 6 years ago

The author's quote is asserting the opposite of what you're saying here.

Filling in "GPT" for x, "text" for y, and "world" for z, the author is stating "if GPT models text, and text models the world, then GPT models the world."

In particular, the author directly addresses your point that "Generalizing a bit, our 'shadow' model of reality in every single domain is far more powerful than language's approximation."

Modality, uncertainty, vagueness and other complexities enter but the isomorphism between world and language is there, even if inexact.

My own personal view tends to lean towards the author's. GPT-3 , despite its many astonishing achievements, has many many limitations. Yet I don't see any indication as of yet that any of those limitations are fundamental ones inherent to the approach behind the GPT series, which is why GPT-3 both excites and frightens me.

akakww2• 6 years ago

GPT models the world through text*

There are many other things that go into modelling the world, like drawing pictures, visualising things, communicating in non-text ways essentially. GPT can only model the world in the ways that text can model the world, which is limited.

rland3• 6 years ago

Yes, fundamentally I agree, of course. If you could find a way to get all of natural knowledge into a sufficient enough network, you would probably have a very intelligent network. But getting text in is easy. It's the most manipulable proxy model. Getting other data in, data that represents what I'm talking about, is much much more difficult.

lmm• 6 years ago

In this conversation, one of us is thinking far harder than the other. GPT can have conversations like this now, which is impressive. But only I can model the beam, the fall, and the physical reality. When I say "that sounds awful," I actually do a miniature physics simulation in my head, imagining losing my balance and falling off a high beam, landing, the physical pain, etc. GPT does none of that. In either case, when it asks the question or when it answers it, it is entirely ignorant of this sort of "shadow" model that's being constructed.

GPT-3 could presumably write a paragraph like that one. You can claim to have a working physics model in your head, but why should I believe that unless it becomes evident from the things that you communicate to me? I've certainly met humans who could have a superficially legitimate conversation about objects in motion while harbouring enormous misconceptions about the physics involved.

Maybe the biggest takeaway from GPT-3 should be that we should raise our standards for human conversation, demanding more precise language and giving less credit to flourishes that make the meaning ambiguous.

akakww2• 6 years ago

wow that last point is profound, it would almost solve the problem of misinformation, bogus headlines, double speak, etc. Simply raising the standard for what good speech is.

rland3• 6 years ago

I mean the miniature physics simulation of me actually imagining another embodied human falling off of a beam. There is a huge knowledge of the physical world encoded into that. Think about it, you can viscerally imagine balancing on a beam, losing your footing, falling, and the subsequent forces and accelerations on all parts of your body during and after the fall, in real time. All of that information is in our model, and none of it is in GPT's. Everyone has this, it's builtin. I'm not talking about Newton's Laws here.

The physical mechanics of being an embodied thing isn't the only example of this. Any experiential knowledge, any knowledge with feedback suffers from this. And any knowledge that is difficult to put into words, like emotional knowledge, interpersonal knowledge, etc., will be off-limits.

That's my point.

walleeee• 6 years ago

A lot of the power of the thought experiment hinges on the fact that the room solves questions using a lookup table, this stacks the deck. Perhaps we be more willing to say that the room as a whole understood language if it formed an (implicit) model of how things are, and of the current context, and used those models to answer questions.

Some define intelligence (entirely separately from consciousness) precisely as the ability to develop an internal model. Coupled to a regulatory feedback the system can then modify itself in response to some set of internal and/or external conditions (Joscha Bach for instance suggests consciousness is a consequence of extremely complex self-models)

laurieg• 6 years ago

I find it a little bit strange that there is an unspoked assumption in almost all natural language processing: That speech and text are perfectly equivalent.

All of the examples in the article work on English text, not spoken English. I would consider spoken English to be a much better "Gold standard" of natural language.

I'm really looking forward to machine translation operating purely on a speech in/speech out basis, instead of converting to text as an intermediate step.

ragebol• 6 years ago

In my head- and maybe this was naive- I had thought that, in order to attempt these sorts of tasks with any facility, it wouldn’t be sufficient to simply feed a computer lots of text.

(Tasks here referring to questions in the New York Regent’s science exam)

Same for me.

But it makes sense of course that learning from text only is entirely possible. I certainly have not directly observed the answer to eg. 'Which process in an apple tree primarily results from cell division? (1) growth (2) photosynthesis (3) gas exchange (4) waste removal', I have been taught, from text books, what the answer should be.

I do have a much better grounding of what growth is, what apples and apple trees are though.

benibela2• 6 years ago

Rather than a generator, I could use a good verifier, i.e., an accurate grammar checker

Recent Advances in Natural Language Processing

Related Stories