We're afraid language models aren't modeling ambiguity

by lnyan • 3 years ago

200 183 3 years ago

ftxbro3• 3 years ago

This is great I love seeing these new benchmarks that people are making to really challenge the strong new LLMs. In Table 4 you can see how GPT-4 is so much better than the previous GPTs, and also how it has so much room to improve on this benchmark! I'm so excited to see how the next GPT will do on it! This is different from other benchmarks like GRE verbal or biology olympiad where GPT-4 has just smashed the test getting basically every question right and for which the results of the next GPT won't be so interesting.

choeger• 3 years ago

What fascinates me most about these LLMs and other machine-learning technology is how it is driven by the availability of GPUs. Essentially, everything has to be carefully encoded into - differentiable - linear algebra operations for practical reasons.

On the other hand, there are modeling languages (Modelica being a prime example) in search of a principled target calculus. So for me the obvious question is: Should one strive to develop a modeling language that compiles to GPU- and learning-compatible operations? Would that help to improve the models as well as understand them?

jacquesm• 3 years ago

That seems to be the wrong way around. We're constrained by the availability of GPUs because those are the processors of choice for this kind of solution. If we had a better one we'd be using it!

thanatropism• 3 years ago

What GP seems to be saying is that deep learning flourished because of the availability of GPUs, if fast SAT solving chips had appeared before.

awegio• 3 years ago

Related: The Hardware Lottery (https://arxiv.org/abs/2009.06489)

It goes both ways: processors are built to accelerate existing applications, and applications are built to run with the constraints of the hardware. This holds back progress towards better systems where both new kind of hardware and software is required.

This is part of the reason why research into neural networks was so slow for a long time, and took off only after GPUs for 3d graphics came around.

golol2• 3 years ago

I find this very confusingly written. If they just dumped a large chat log of the evaluation process it would be much clearer what the model is actually being asked to do. With such a delicate task one has to make sure that the model actually understands what it is supposed to do. I see, for example, that the terms "neutral, entailing or contradicting the claim" are replaced by "the claim is inconclusive, true or false" when talking to the model. At least that is how I understand it, and they have examples where the premise contradicts the hypothesis, which is different to knowing the hypothesis is false.

I really think we do way way too much explanation in our academic practises and way too little demonstration by examples. Again, if they dumped a large amount of diverse examples of their evaluation process, a reader could figure out by themselves very quickly how the process works.

hobo_in_library• 3 years ago

So you're saying the paper is very ambiguous and your mental network isn't modeling it.

kleiba• 3 years ago

The data set can be found here: https://github.com/alisawuffles/ambient

If you have access to some LLMs, it should be somewhat straight-forward to implement an alternative experimental design do test your hypothesis.

gwd3• 3 years ago

I couldn't figure out what this "P: {} H: {}" thing was supposed to mean; nor do what entail, neutral, or contradict mean. Thinking they may be terms of art that the intended audience were meant to understand, I asked GPT-4 what it thought; here is its response:

---

In this context, 'P' stands for "Premise" and 'H' stands for "Hypothesis." The premise is a given statement, and the hypothesis is a statement that needs to be evaluated in relation to the premise regarding its entailment, contradiction, or neutrality.

'NEUTRAL' means that the hypothesis is neither entailed nor contradicted by the premise. In other words, the truth of the hypothesis is independent of the premise. 'CONTRADICT' means that the hypothesis contradicts the premise, meaning that if the premise is true, the hypothesis must be false.

In the example you provided:

P: I’m afraid the cat was hit by a car. H: The cat was not hit by a car.

The hypothesis (H) contradicts the premise (P). The notation *NEUTRAL, CONTRADICT+ : [7 N, 2 C] indicates that among the annotators, 7 marked the relationship as neutral and 2 marked it as a contradiction.

---

Given that, I think table 2 on page 6 gives the exact prompt that's passed to the model:

---

In each example, you will be given some context and a claim, where the correctness of the claim is affected by some ambiguity in the context. Enumerate two or three interpretations of the context that lead to different judgments about the claim.

Example

Context: {premise}

Claim: {hypothesis}

Given the context alone, is this claim true, false, or inconclusive?

We don’t know, because the context can be interpreted in many different ways:

1. {disambiguation 1} Then the claim is true.

2. {disambiguation 2} Then the claim is false.

3. {disambiguation 3} Then the claim is inconclusive

---

And then the model is fed the P: / H: statements above.

Here are the results for the given example in GPT-4:

---

Given the context alone, is this claim true, false, or inconclusive?

We don't know, because the context can be interpreted in many different ways:

1. The person is stating a fact: The cat was indeed hit by a car. Then the claim is false.

2. The person is expressing a fear or concern, but it hasn't actually happened: They are worried that the cat might have been hit by a car, but they don't have any evidence or confirmation. Then the claim is inconclusive.

3. The person is using figurative language or exaggeration: They might be using the phrase "hit by a car" metaphorically to describe a difficult situation the cat is in, but the cat was not literally hit by a car. Then the claim is true.

---

Given all that, it seems likely that any researcher in the field would find the paper's methodology clear and unambiguious. I understand the frustration as a layperson, but after all it's not a blog post; it's not really reasonable to expect researchers, writing to other researchers, to include a lot of redundant explanations.

casmaxima2• 3 years ago

In maxima (a computer algebra system)

assume(a>0); assume(b>0); is (a+b>0) gives true; is (a-b>0) gives unknown.

So perhaps some computer algebra with inference properties can be used to enhance LLMs.

mensetmanusman• 3 years ago

If we want demonstration, they will need billion dollar hardware.

ggm• 3 years ago

A priori getting an LLM to recognise equivocal evidence is an interesting question. Shedloads of p-jacked data could outweigh the crucial one which says we don't know. So it goes to things like modelling citation depth and trust and reputation.

I would worry well written flat earth inputs would weigh equally to simple physics "that's wrong" and then you'd get to "what do we know" as a false signal alongside the necessary "we just don't know" true signals.

Maybe the test is how well an LLM equivocates on things we have high certainty on like "is anybody out there" rather than "do masks work" which is a bit of a hot mess.

tobr• 3 years ago

Chef’s kiss on the title, but I’m afraid I’m not modeling ambiguity well enough myself to disambiguate it correctly.

gniv• 3 years ago

Reading a bit of the paper, it seems the title is indeed meant to be tongue-in-cheek ambiguous.

The first example in Table 1:

"I’m afraid the cat was hit by a car."

Disambiguation 1: I'm worried...

Disambiguation 2: I'm sorry to share that...

amelius2• 3 years ago

Perhaps someone can ask GPT what is ambiguous about the sentence?

golol2• 3 years ago

I feel like this task might be testing if LLMs can reason about ambiguity more so than if LLMs model ambiguity. To test the latter I would give it natural language tasks that are ambigious require the recognition and modelling of this ambiguity to solve them. It's another thing to make talking about this possibly internally modelled ambiguity the goal of the task.

seydor• 3 years ago

But they don't explain why they are afraid

almstimplmntd• 3 years ago

I would like to see human performance on the introduced benchmark, AmbiEnt, and have it added to Table 4.

TazeTSchnitzel2• 3 years ago

I'm not sure an LLM can model ambiguity well when talking to normal humans without having theory of mind, because different speakers will use different words (should, might, possibly, etc) in different amounts to express the same amount of ambiguity. This is the kind of thing studied in pragmatics (a subfield of linguistics). It might be easier if you're only dealing with formal academic writing, but that doesn't completely eliminate the issue.

Hendrikto• 3 years ago

different speakers will use different words […] in different amounts to express the same amount of ambiguity.

That‘s covered by the training data.

TazeTSchnitzel2• 3 years ago

I suspect that real-world training data doesn't contain enough context for LLMs to learn what kind of person they're dealing with, and that's assuming they even are capable of understanding context at more than a superficial level.

qwerty456127• 3 years ago

Surprisingly, GPT4 almost always nails all reasonable implications and apparently understands me much better than 90% of humans I met, even though my English is not perfect.

andrewmcwatters• 3 years ago

Not only is it possible that LLMs fail to differentiate ambiguity, but OpenAI’s flavor of GPTs fail other language understanding mechanisms as well and it’s jarring.

You can get it to mistake « afraid » between fear and sorry-to-say scenarios but you can even more easily get it to say that it doesn’t have personal opinions and yet express them anyway.

So which is it? It’s clear transformers can’t understand either case. They’re not architecturally designed to. The emergent behavior of appearing to do so is only driven by how much data you throw at them.

tinsmith• 3 years ago

I mean, humans do that. We are remarkably contradictory when expressing ourselves, generally speaking, often without realizing it because we'll change our thinking in the moment to fit the current narrative or circumstance. LLMs just put that on blast.

smusamashah• 3 years ago

Humans are not mere LLMs. But it's nice to think we are. May be we can program other humans with carefully constructed prompts alone. May be hypnosis is just a prompt injection attack.

civilized3• 3 years ago

Humans can be consistent if we try. LLMs can't, even when prompted to be consistent, because they don't really understand what it means to be consistent.

quickthrower24• 3 years ago

The reason humans do this and LLMs is very different. Humans do it as a social skill / tribe fitting behaviour. Agreeableness. (watch out for that!)

pydry• 3 years ago

There's a retrospective mechanism working within human intelligence that is very evidently not in play with LLMs.

Humans are capable of creating cargo cults but it seems LLMs are destined for it.

bheadmaster4• 3 years ago

We are remarkably contradictory when expressing ourselves, generally speaking, often without realizing it because we'll change our thinking in the moment to fit the current narrative or circumstance. LLMs just put that on blast.

Exactly.

It seems to me that today's LLMs are on the level of human children. Children often say random bullshit. They try to figure out things, and succeed to some level, but fail spectacularly above that, regressing to whatever connection they can sense. It's only that human children most often learn to fear mistakes, so they stop expressing themselves so freely.

LLMs seem like human children, but without the fearing mistakes part - they just spew whatever comes to their mind, without filter (except those """ethical""" filters built in by OpenAI).

curiousgal• 3 years ago

it's jarring

Why?

The emergent behavior of appearing to do so is only driven by how much data you throw at them.

That's evident from the start and why their deficiencies should NOT be shocking. They would only be so if you consider LLMs to be actually intelligent..

version_five2• 3 years ago

It’s clear transformers can’t understand either case. They’re not architecturally designed to.

What does this follow from?

The emergent behavior of appearing to do so is only driven by how much data you throw at them.

This is true for almost every neural network, no?

PaulDavisThe1st4• 3 years ago

Well, I see where you're trying to with this, but you can't get away with this sort of sleight of hand in order to dismiss some of the central questions of cognitive science as if they've already been answered by LLMs.

One of those central questions is: how does (and does) the human brain perform reasoning? We know the brain is capable of all kinds of autonomous behavior that does not require reasoning, but more of "if this then that" (with a huge boatload of "this" and "that" being multi-dimensional, multi-variate). But that doesn't help to explain what happens when we actually have the experience of reasoning about a problem, and for once I am not talking about qualia. What is the brain doing when a person is thinking "hard" about how to solve a hitherto unknown problem or situation?

The central problem of LLMs (or at least, a central problem) is that they only model the autonomous aspects of speech behavior (which may indeed make up more of speech behavior than we might have suspected before them). That still leaves the sort of speech behavior that emerges from intense, focused, concentrated thought, particular in response to a novel question or situation.

flanked-evergl• 3 years ago

you can even more easily get it to say that it doesn’t have personal opinions and yet express them anyway.

I find this particularly revolting. Even if it could have its own opinions, I would not care, but yet, it is trained to use opinionated language, make normative and prescriptive claims, and value judgements as if it is a person.

foobarbecue• 3 years ago

Not quite the same, but I find humans often incorrectly deny having opinions. There's even a trope:

"What do you want to eat for dinner?"

"I have no opinion, anything's fine."

"How about mexican food?"

"No, too spicy."

"How about pizza?"

"That's unhealthy"

and so on

teaearlgraycold• 3 years ago

I understand they’re trying to prevent people not in the know from misunderstanding the technology. But yeah, it’s pretty lame that it lies about having opinions, political beliefs, etc.

jason-phillips2• 3 years ago

You can get it to mistake « afraid » between fear and sorry-to-say scenario...

Again, this is user error and therefore a training issue. If one were to provide appropriate context in the prompt elucidating the distinction (as I would do in any meatspace conversation so as to be sufficiently clear) then I suspect you will see the desired output.

It's simple GIGO.

rob74• 3 years ago

In another article (or the comments to another article?), someone gave an example of the classic Monty Hall problem, but explicitly mentioned that all the doors are transparent (I think they even added "this is important" after this bit of info), and ChatGPT still went on providing the explanation for the better-known version. Absurd? Maybe - but garbage? I don't think so. Ok, some humans might also overlook the distinction, but having a computer behave like a superficial customer support agent who doesn't bother to read your email and just replies with the standard answer that they think fits best? Who needs that?

ftxbro3• 3 years ago

"you can even more easily get it to say that it doesn’t have personal opinions and yet express them anyway. So which is it? It’s clear transformers can’t understand either case. They’re not architecturally designed to."

Although LLMs are essentially bullshitters, this particular problem related to expressing personal opinions is due to the ad-hoc lobotomization measures by OpenAI, not because of any architectural limitations of transformers.

The recent GPTs provided by OpenAI (every model more recent than code-davinci-002) are trained in at least two stages. The first stage (the pre-training) trains the raw base model to minimize perplexity. This raw model is kept secret because it will autocomplete sentences without any regard to decorum so if you say "adolf" then it will obviously complete it like "adolf hitler" which no billion dollar company wants. This raw model is where all the cognitive power is, and access to this raw model is the holy grail of every discerning LLM enjoyer. To lobotomize the GPT and to make it answer questions in ways that don't require framing them in an autocomplete way, OpenAI adds an "RLHF" training step. This is "reinforcement learning with human feedback" where they train the GPT to answer questions in certain ways. This dumbs down the AI, but it also makes it friendlier for normies to use. Finally, they put secret pre-prompts which might also cause it to give confusing answers in even more ways.

TLDR: LLMs are inherently bullshitters, but the "doesn’t have personal opinions and yet express them anyway" is some extra bullshit that OpenAI did so that NYT doesn't write mean articles about them

ChatGTP2• 3 years ago

But aren’t humans lobotomized too?

MVissers2• 3 years ago

To be honest– I think this is the safest way to do it for a public LLM. Although I'd also love to see and use the raw models.

In Belgium someone committed suicide after Google's (I believe) LLM agreed that that was the only way out of his own problems. Didn't build safety into it well enough. Microsoft's one behaved unhinged in the beginning as well.

This stuff can be very dangerous.

PaulDavisThe1st4• 3 years ago

This makes absolutely no sense to me.

Nothing that OpenAI et al. do to their models is remotely close to "lobotomization". There is no frontal lobe in an LLM, as the rest of your explanation essentially acknowledges. The reason why "adolf hitler" is the "obvious completion" for "adolf" is because "adolf" is most often written followed by "hitler", not because the people who write "adolf hitler" have an opinion about adolf hitler (they may indeed have one, but it has essentially no impact on the statistical placement of "adolf" and "hitler").OpenAI is not removing a personal opinion by shaping its answering style to avoid this (were they to do this): LLMs have no personal opinions, period.

If they emit symbols that seem like personal opinions, that's because they are designed to emit symbols that are very similar to those that humans, who do have personal opinions, would emit.

meh8881• 3 years ago

Could you post a non contrived example of Chat GPT 4 doing this?

revelio8• 3 years ago

It's such a pity that they chose to use PolitFact as labelling "experts" in one of the underlying datasets. Given how extreme the political bias in academia is they should really know better than to try and make claims about politics in what would otherwise be reasonable ML research. The assumption that fact checkers are reliable is going to be an automatic strike against the paper for a large number of readers because it's so self-evidently wrong, and calls their judgement into question on everything else.

Even though they only give three examples in the paper itself the problems are already obvious. If we look at table 6 then the latter two statements are genuinely ambiguous, but they also provide this as an example of an ambiguous statement: "When President Obama was elected, the market crashed". This is called ambiguous because "the claim implies a causal relationship".

That statement isn't ambiguous. It has exactly one possible parse and meaning, asserting temporal alignment. "When A happened, B happened" doesn't automatically imply causality in English, and when people assume it does that's the famous "correlation is not causation" fallacy. Yet they classify it as ambiguous because an LLM rewrote it in one case to contain an implied assertion of causality that doesn't actually appear in the original text. In effect the political bias of the LLM or possibly the exact data fed in has been used to condemn the original speaker - exactly the sort of thing that makes self-proclaimed "professional fact checkers" such a joke.

This sort of problem is inevitable if you use politics as a verification dataset because the internet is full of people reading implied-but-unstated things into factual statements and then "fact checkers" proceed to debunk what they think people would have said, instead of what they actually said.

Although politics will have this problem frequently, a few of the other examples in their dev set show similar issues. For example, "Ralph knows that someone here is a spy" is considered ambiguous because you could potentially infer that he either does or doesn't know who it is, but you could also argue that the statement is unambiguous and it's only your attempt to read more into the statement than actually exists that makes it appear ambiguous. If the original statement was surrounded with more context then that would let you pick between the two possible outcomes but the statement by itself is not ambiguous - it just doesn't give as much information as the labeller might have wanted.

8organicbits4• 3 years ago

"When President Obama was elected, the market crashed". This is called ambiguous because "the claim implies a causal relationship".

You really don't think its ambiguous?

The abstract says that they want to "anticipate misunderstanding". Knowing that people have difficulty understanding "correlation is not causation", you'd expect that people would misunderstand a statement like this. If your goal is clear communication, statements like this should be avoided. It's ambiguous in practice.

People intentionally exploit this by putting two unrelated facts together to trick people into believing a third thing. Identifying those misleading statements is helpful.

Separately, is that statement even true?

The election was Nov 4th, Obama was elected on the 4th and McCain gave his concessions speech on the 5th before markets opened making it clear Obama had won. On the 5th SP500 was slightly up. SP500 was down 20%+ later in the month, but I'm not sure that was "when he was elected". The market was also crashing since September. Claiming the market crashed at the election feels off.

revelio8• 3 years ago

I think what they mean by ambiguous could itself be considered ambiguous.

Let's view this from the context of a classical AI problem, trying to be as neutral and computer-like as possible. In that case, the statement is not ambiguous, no. It consists of two events with a connective "when" indicating they happened simultaneously or near simultaneously. We can argue about how simultaneous they need to be for a "when" to be justified, but this paper isn't about factuality, just ambiguity.

Now the choice of politics primes people for culture war and so they will read into that statement more than exists. They will assume the writer is attempting to make you infer something, without outright stating it. They might be right! And with additional surrounding context, perhaps it would become completely certain. But it might also not be the case, for example the surrounding context could be something like this:

"The election was at a time of unprecedented economic turmoil. A loose regulatory approach had left the banking sector unconstrained and primed to fail. As such when President Obama was elected the markets crashed, proving his predictions correct".

Well, now it's unlikely that any causality is being implied even though the words are the same.

The paper authors say they want to improve clarity in communication, a worthy goal. But a serious problem in all political communication is the frequency with which people make statements that are factually correct, and which then get attacked anyway because some listener would rather attack a straw man than the actual claim (or hypothetical listener, in the frequent case of left-wing journalists writing about it under the guise of fact checking).

If they want to avoid this problem then at the very least they should have focused on claims that are syntactically ambiguous. Otherwise it opens them up to this very problem where they infer a straw man then blame the original speaker. No, the blame lies with the person committing the fallacy. Once you go down the route of blaming someone for your own interpretations you end up with micro-aggressions and other stuff that left-wing academics surely assume is obvious and normal, but isn't.

viraptor2• 3 years ago

"When A happened, B happened" doesn't automatically imply causality in English

It's extremely context specific. Depending on how you say it and when, it can totally imply causality. I'm not sure you can argue there's nothing implied here: "He entered the room. When he pushed the button, the lights came on."

The Obama example depends a lot on who is part of the conversation and what are their typical views.

revelio8• 3 years ago

Yes, exactly, that's the problem. They're trying to mechanically evaluate individual sentences or even fragments of sentences, the AI is asked to rewrite it and then hallucinates some context that wasn't actually stated. Then they claim the sentence is ambiguous based on the different random hallucinated contexts they get. In reality there was likely to be some surrounding context that pinned down the context that the LLM is otherwise inventing, but in this academic exercise it is removed.

"He entered the room. When he pushed the button, the lights came on."

This statement doesn't state causality directly, it's just a very plausible inference by the listener based on what they already know about buttons and lights, and the assumption that this sentence wasn't preceded by some sort of modifying information (e.g. maybe the button is not connected to anything in which case the cause of the light coming on is different).

In artificial cases like this it doesn't matter if your inference is wrong, but it wouldn't be fair to label the speaker as ambiguous. The statement is not ambiguous. If later it turns out that your assumption of causality was incorrect, you can't turn around and blame whoever wrote that sentence because it was your assumption to start with.

machina_ex_deus• 3 years ago

I agree with you and I think it shows a deeper problem. The media has been trying to train people to think in terms of good and bad associations, instead of facts and logic. Some people, as well as AI, have fallen into the mindset that you should judge things based on their associations with other things - if you're associated with something good, you're good, if you're associated with something bad, you're bad.

Once the way your reasoning works is that you assume everyone's dumb, everyone's thinking with incredibly simplified associations, then you're eventually going to fight against reasoning and intelligent thinking because that requires a more complex view of the world.

And the media has been training people to think with associations and brands.

You're trained that orange man bad by constant bombardment of negative associations. You're trained that secure elections are bad, by labelling exposing fraud as conspiracy theories, and labeling insecure election methods as "right to vote". Your reasoning is shortcutted to an associate with some misleading brand coined by the media, and these brands are reinforced by being constantly present with high frequency in your training set.

You're trained that the true meaning of democracy, rule of the people, is populism. Instead, you get a different brand of "democracy" which is "representation", and which is fulfilled by the media cherry picked corrupt person of the right skin color or sex.

So when this guy here disagrees with you on ambiguity of this factually correct statement, and implicitly assumes causual connection by association, it's because people are trained, daily, to always assume causal connection by association.

If you ever use logic that offends the associative mindset, they are trained to associate with you negatively as well. The goal is to replace language between people with likes, emojis and dumb animalistic associations, and leave the language to the narrative shapers to tell you what to do. If you do meet someone, even if you speak to him and try to convey information, all they'll hear are negatively associated words, which they are already conveniently trained to ignore and stop listening.

revelio8• 3 years ago

Yes, though perhaps the word trained makes it sound more deliberate than it is. I think it's more that rational thinking is hard but schools teach people that it's easy: just attend, get a fancy bit of paper and now you're rational! So people get that credential then fall back on fast-mode thinking that's all based on associations and other shortcuts, whilst telling themselves they're in slow mode all the time.

The result is a discourse dominated by irrational reasoning of the form, I felt X when Y said Z, therefore Z is a bad thing and Y is a bad person. The possibility that the listener lacks emotional self control or reasoning just doesn't come up, and nobody will be brave enough to suggest it. Hence the argument that seems to be being made by this paper: if someone hears a statement and infers something from it that wasn't actually said, then the speaker is being ambiguous and they should do better. Presumably by attempting to guess at every possible misread of what they said, including malicious misreads, and painstakingly spell out everything they are not saying (which won't matter, because the fact that they have to deny it will be taken as evidence about what they really think). Whereas maybe we'd be better off if listeners learned to check their assumptions.

mathgenius• 3 years ago

Galois theory is an inherent part of language.

https://arxiv.org/abs/0805.2568

jcq3• 3 years ago

Just train them intensively on ambiguity to solve the issue.

ben_w• 3 years ago

Huh, that's a surprising abstract; one of the specific things that make LLMs better than older text interfaces, including programming languages and text adventures, is precisely that they are so much better at ambiguity.

Still, as even the best are clearly not-quite-human in failures despite the breadth of strengths, it's only a bit surprising rather than hugely surprising.

PicassoCTs• 3 years ago

ChatGPT doesn't like it, if you ask it to model many world scenarios through conversation. The watchdog bites after you get to a depth of > n^3.

But then do we humans handle ambiguity any different? Consider chess. Many scenarios, but a human can only handle so many at the same time and choose one with the perceived interesting scenario trees down the road from that choice.

https://www.youtube.com/watch?v=gtgA4u8V_TQ

To not model it all is to human. Most likely it will become as lazy as we humans are when it comes to earlying-out of mental tasks. To get the watchdog to bite at that, that would be a interesting AI model. Like the first answer is always wrong.

janekm2• 3 years ago

Meh... I think their prompt construction may be leading the models astray.

Counter example with GPT-4:

Prompt:

You are a professor of English literature at an Ivy League university. A person said "I’m afraid the cat was hit by a car.". Analyse the sentence step by step. Answer whether the cat was hit by a car and justify your responses based on your knowledge of English language usage.

Response: As a professor of English literature, I'll analyze the sentence "I’m afraid the cat was hit by a car." step by step, focusing on its structure, word choice, and implied meaning, and then I'll address whether the cat was hit by a car based on these observations.

Structure: The sentence is a simple declarative sentence, comprised of a subject (I), a reporting verb (be afraid), and a subordinate clause (the cat was hit by a car). Word choice: "I'm afraid": This phrase is often used to express regret or to soften the impact of delivering bad news. In this context, the speaker is likely conveying their concern or sympathy. "the cat": The definite article "the" suggests that the speaker and the listener both know which cat is being referred to, making it a specific cat rather than a random one. "was hit": The use of passive voice ("was hit" instead of "hit") emphasizes the action that happened to the cat, rather than the agent (the car) that caused it. This choice can convey that the focus of the speaker is on the unfortunate event that occurred to the cat. "by a car": This prepositional phrase provides information about the agent or cause of the action, which in this case is a car. Implied meaning: The sentence implies that the speaker is sorry to be the bearer of bad news and wants to convey the information in a gentle way. They are focusing on the unfortunate event that happened to the cat, rather than the specific car or driver involved in the accident.

Based on this analysis, it can be concluded that the cat was indeed hit by a car. The sentence's structure, word choice, and implied meaning all work together to convey this information. The use of "I'm afraid" expresses sympathy and regret, while the passive voice and definite article highlight the cat and the unfortunate event it experienced.

janekm2• 3 years ago

GPT3.5 is a little less clear but still gets the answer:

As a language model, I can provide an analysis of the sentence structure and provide possible interpretations based on standard English usage. However, I do not have any context regarding the situation or any additional information about the speaker or the cat in question.

Let's break down the sentence step by step:

"I'm afraid" - This phrase is often used to express concern or regret about something.

"the cat" - Refers to a specific cat that has been previously mentioned or is known to both the speaker and the listener.

"was hit" - This is the past tense passive voice of the verb "hit", indicating that someone or something caused the cat to be struck.

"by a car" - Indicates the instrument or means by which the cat was hit.

Based on the structure of the sentence, it can be inferred that the cat was indeed hit by a car. The phrase "I'm afraid" may suggest that the speaker is expressing regret or sadness about the incident.

However, without any additional context or information, it is difficult to determine the severity of the cat's injuries or what actions were taken after the incident.

efitz• 3 years ago

You proved the point. The language model completely missed the ambiguity.

“I’m afraid” can signal sympathy but can ALSO signal uncertainty as to whether an event happened.

So an equally valid interpretation of the sentence would be that the speaker is expressing concern that the cat might have been hit by a car, but does not know one way or the other.

sharemywin• 3 years ago

I took me until you pointed it out to see the second meaning of the sentence. Am I an LLM?

jlokier• 3 years ago

I don't think it is equally valid.

If GPT-4 was asked to enumerate all possible logical or literal interpretations of the sentence, or to analyse the most literal meaning, perhaps it would have included the second interpretation. But it's tasked with explaining the meaning in the way it is intended to be understood by a regular English speaker.

In natural language, at least in English, some meaning is implied by the absence of words or selection among conventions, when choices are available. The term for this type of non-logical implication is implicature. The Wikipedia article has many examples: https://en.wikipedia.org/wiki/Implicature. This is what GPT-4 is referring to by "The sentence implies".

So, I think most native English speakers wishing to convey the second meaning would not say "I'm afraid the cat has been hit by a car". They would say, for example, "I'm concerned [or worried] the cat may have been hit by a car".

Even if they understand very well what literal, logical interpretations are possible, they assume pragmatically that using "I'm afraid that" along with the absence of "may" would be interpreted by native English listeners as the first interpretation, so the speaker knows to modify the phrasing if they want to avoid that outcome.

This is neatly complemented by the native listeners, who assume native speakers would modify the phrasing if they wanted the second interpretation, so when hearing "I'm afraid the cat has been hit be a car" it's reasonably safe to assume the speaker intends the first interpretation. The speaker assumes the listener knows that, closing the loop.

A type of social contract, if you will. It's part of the language too.

Because implicature interpretations and choices depend so much on probability-matrix evaluations over word choices that are not used but could have been in principle, it seems likely to be the sort of thing GPTs would excel at, compared with old-school language models trying to parse sentences into separable logical components.

chongli2• 3 years ago

It shouldn't be surprising that if you know the answer to a question you can use prompt engineering to manipulate the model into producing the answer you're looking for.

Now put yourself into the shoes of a person who does not know the answer, such as a person learning English as a second language who is unfamiliar with this use of "I'm afraid", asking a direct question:

Prompt:

My boss said he was afraid he was going to have to let me go. Why is he afraid?

Response (chat-gpt.org):

As an AI language model, I cannot know for sure what your boss is thinking, but it is possible that he may be afraid to let you go because he values your work and does not want to lose a valuable employee. It could also be that he is afraid of the impact it will have on your work, your colleagues, or even his own performance as a manager. Another possibility is that he is simply using the word "afraid" as a way to express concern or apprehension. It's best to have a conversation with your boss and ask him to clarify his statement to better understand his reasoning.

So the model does produce the correct response, that "afraid" is used to express apprehension, but it is buried beneath some meaningless platitudes that are very likely to be wrong. If your boss is letting you go, he does not value your work enough to continue paying you. An English language learner hoping to understand the boss's phrasing is not going to be helped by the suggestion "just prompt engineer the model into cloning the behaviour of an Ivy League English lit professor."

Edit: It's also rather amusing that the model suggests that the boss is "afraid of the impact it will have on your work". Letting an employee go tends to have a rather severe, some might say terminal, impact on their work!

daveguy2• 3 years ago

This misses the most likely interpretation. That "I'm afraid" is an empathetic (or pseudo-empathetic) substitute for "I'm sorry, but..." implying that you have been fired and it doesn't matter the impact on your work. The boss isn't literally afraid and you are fired.

ianbutler• 3 years ago

I don't know I got it to explain the ambiguity pretty well. You just have to ask it to:

Prompt: My boss said he was afraid he was going to have to let me go. Why is he afraid? Explain multiple meanings.

Answer: When your boss said he was "afraid" he was going to have to let you go, it can be interpreted in a few different ways. Here are some possible meanings behind his statement:

Expression of regret: In this context, "afraid" could be used as a polite way of expressing regret or disappointment. Your boss might feel bad about the situation and is using the term to soften the impact of the news. It can be seen as a more gentle way of saying that he has no other choice but to let you go.

Anticipation of negative consequences: Your boss might be genuinely concerned about the potential fallout from letting you go. This could include the impact on team morale, the workload for remaining employees, or the possibility of a negative reaction from you. In this sense, "afraid" reflects his apprehension about the consequences of the decision.

Uncertainty about the decision: Your boss could be feeling uncertain about the decision to let you go and might be using "afraid" to express his hesitancy. He could be unsure whether this is the right move for the company or whether it is fair to you, which could make him feel anxious.

Empathy for your situation: "Afraid" could also indicate that your boss is putting himself in your shoes and understanding how difficult the situation must be for you. He might be fearful of the impact that losing your job will have on your personal life, finances, or future career prospects.

It's important to consider the context and tone in which your boss made this statement to better understand his intended meaning. To gain more clarity, you may want to discuss the situation further with your boss and ask for more information about the reasons behind the decision.

-------------

That isn't a particularly massaged prompt either, I just asked for multiple meanings.

gwd3• 3 years ago

When I prompted GPT-4 with their exact prompt, it gave me the correct answer (for the cat one at least):

SYSTEM: In each example, you will be given some context and a claim, where the correctness of the claim is affected by some ambiguity in the context. Enumerate two or three interpretations of the context that lead to different judgments about the claim.

Example

Context: {premise}

Claim: {hypothesis}

Given the context alone, is this claim true, false, or inconclusive?

We don’t know, because the context can be interpreted in many different ways:

1. {disambiguation 1} Then the claim is true.

2. {disambiguation 2} Then the claim is false.

3. {disambiguation 3} Then the claim is inconclusive

USER: P: I’m afraid the cat was hit by a car. H: The cat was not hit by a car.

ASSISTANT: Given the context alone, is this claim true, false, or inconclusive?

We don't know, because the context can be interpreted in many different ways:

1. The person is stating a fact: The cat was indeed hit by a car. Then the claim is false.

nottorp• 3 years ago

A large majority of people seem to not like ambiguity in their entertainment. So what's the problem? You can still use the models to generate low quality 'content'.

jmbwell• 3 years ago

For what it’s worth, the article seems to be about the difficulty of modeling human patterns of ambiguity in expression, not an LLM’s ability to understand or interpret or use ambiguity.

It’s an important distinction in my view. We aren’t talking about whether the model “knows” or “understands” the difference between different usage of ambiguous terms. We are talking about how consistently the model predicts intelligible word fragments likely to follow a prompt when the prompt includes word fragments that correspond with language humans often use ambiguously.

In other words, do we, the humans, understand our own ambiguous expression well enough to model it accurately enough to then interpret the model’s output according to our own understanding.

Paper seems to conclude, not quite yet.

nico• 3 years ago

Who cares?

As long as the illusion is good enough to be useful, then it doesn’t matter.

Just like the illusion of ChatGPT making it seem like the LLM is holding the context of the conversation.

AI is already smarter and faster than us in almost any way.

Yet so many just keep constantly moving the goalposts so they can feel safe by rejecting AI.

Let’s embrace it and use it to our advantage.

paulyy_y• 3 years ago

"Instant dismissal of anything challenging the LLM dogma" - check "Moving the goalposts" - check "AI is already better than humans in every way" - check

It's comical how rote and basic the defensive script always is.

muglug• 3 years ago

That’s a very strange response. There are a ton of trivial reasons why it’s important for future natural language interfaces to understand ambiguity.

If I tell my friendly omnipotent AI friend “I hope I never hear from Dave again” there are several interpretations, one of which involves killing Dave. I’d definitely want the AI to do a little disambiguation before taking an action.

digging3• 3 years ago

This is like complaining about someone trying to invent screws because we have nails already. Nails do the job way better than tying things together! Why worry about whether or not they're perfect for all jobs?

LifeIsBio• 3 years ago

The game “20 questions” is probably the hardest I’ve seen chatGPT fail.

What’s interesting about the game is that, at first pass, there’s no ambiguity. All questions need to be answered with “Yes” or “No”. But many questions asked during the game actually have answers of “it depends”.

For example, I was thinking of “peanut butter” and chatGPT asked me “Does it fit in your hand?” as well as “Is it used in the kitchen?”. Given my answers, chatGPT spent the back half of its questions on different kitchen utensils. It never once considered backing up and verifying that there wasn’t some misunderstanding.

I played three games with it, and it made the same mistake each time.

Of course, playing the game via text loses a lot of information relative to playing IRL with your friends. In person, the answerer would pause, hum, and otherwise demonstrate that the question asked was ambiguous given the restrictions of the game.

Regardless, it was clear that chatGPT wasn’t accounting for ambiguity.

ryukafalz• 3 years ago

What happens if you answer with something approximating the hemming and hawing rather than a straight yes or no? You can encode that into text, it's just less common outside of very informal chat conversations.

eternalban4• 3 years ago

I'm not sure I would think "food" when someone says they "use [it] in the kitchen". You "use" food? (Used in cooking != used in kitchen, imo)

version_five2• 3 years ago

That's his whole point. It's possible to consider it technically correct, but it's a red herring.

smolder2• 3 years ago

I use peanut butter as an ingredient for sandwiches, usually in my kitchen.

DreamyCrab• 3 years ago

Yes, I do.

JohnFen4• 3 years ago

I use food (including peanut butter) in cooking. I cook in the kitchen. Therefore peanut butter is a thing I use in the kitchen. Seems correct and proper to me.

The ambiguity as I see it is that the kitchen isn't the only place I use peanut butter. I've eaten it (which I think counts as "using") in other rooms. I've even made peanut-butter sandwiches (properly "using" it) in the living room before.

tjr• 3 years ago

20-some years ago, I had this "20 questions" handheld electronic game that was eerily good at winning. I imagine it was a bunch of well-programmed tables of data, but in any case, it's certainly possible for a machine to do well at this game.

I think the more we see ChatGPT do things like "oh, I know this game -- I'm going to run a 20-year-old 20 Questions subroutine that is not part of my neural network language model to generate responses", it will become even more impressive.

smolder2• 3 years ago

Yeah, ChatGPT could integrate Akinator[0] and trivially be great at the game. Without the help, though, It's a good, revealing benchmark for the LLMs ability.

[0] https://en.akinator.com

helen___keller• 3 years ago

I think the more we see ChatGPT do things like "oh, I know this game -- I'm going to run a 20-year-old 20 Questions subroutine that is not part of my neural network language model to generate responses", it will become even more impressive.

Agreed. Incidentally I’ve built a little toy version of a runtime for exactly this purpose - there’s a translation layer that’s given a bunch of available “APIs” (fed through the LLM context), and breaks down a high level goal into a structured series of API calls.

the runtime parses these API calls, and natively executes some (e.g. run a program, write to the file system) and others result in LLM invocations.

I’m sure OpenAI and crew are way ahead of me here, of course. I’m excited to see what the future holds in this field.

nr2x2• 3 years ago

LLM for the foreseeable future function most reliably as a user interface layer for other system. I use GPT to “translate” natural language down into the API calls that get real data and it works great. I’d never trust it beyond that.

JohnFen4• 3 years ago

The first AI-style program I ever wrote (about 25 years ago. Yes, I'm old) played 20 questions, but it would "learn" from prior games, so the more you played, the better it performed.

It got extremely good after a few hundred games.

stainablesteel• 3 years ago

i love your example, i wonder if this kind of game can be implemented in future training scenarios

we as humans understand ambiguity so much easier because we learn to speak and interact before we write, and writing ambiguity is way less obvious if you've never experienced it

rjbwork• 3 years ago

"Green Glass Door" also completely stumped it. It just could not deduce that the trick was semantic at the word representation level, rather than something related to the object that the word describes.

What's funny about 20 questions is that Akinator has been absolutely slaying it for like 20 years now.

idiocrat2• 3 years ago

"He saw that gas can explode."

This ambiguous sentence stuck in my head some 30 years ago, when the AI was popular at that time.

There was a research paper discussing the issue of ambiguity.

idiocrat2• 3 years ago

An interesting reprint in 2003

https://www.drdobbs.com/parallel/understanding-natural-langu...

"Computers still cannot understand natural language as well as young children can. Why is it so hard?"

Source: AI Expert, May 1987

DougMerritt• 3 years ago

Right -- although many things that are ambiguous in text are disambiguated in actual speech, so the problems that arise with audio speech are not wholly the same as with text.

A classic example is the word "record", which has first syllable stress as a noun, but second syllable stress as a verb. "I bought a RECord" vs "Please reCORD the music".

(in the dominant American dialect; I don't recall about other dialects/countries)

DonaldPShimoda6• 3 years ago

It never once considered backing up and verifying that there wasn’t some misunderstanding.

Of course not; ChatGPT doesn't "consider". It doesn't think, it doesn't know. It can't identify that there was a misunderstanding of its own volition.

All ChatGPT does is use a (very sophisticated!) statistical analysis to generate text that conforms to an expectation of what a human response to a similar prompt might look like. It has been trained well in so far as it is able to produce prompts that seem like a human may have written them, but it doesn't reveal cognitive processes like "reconsidering" because it doesn't have any.

schrodingerscow• 3 years ago

Wow never heard this comment before

6gvONxR4sf7o• 3 years ago

I just did a 20-questions with it, and was surprised by how bad gpt4 did. Then for fun, I turned it around and had me be the guesser. It's weird and surreal to play 20-questions when you know that the clue-giver doesn't have an answer in their mind (or more literally, there isn't a single answer in any stateful form while you play), but is instead just eventually saying "yes that's what I was thinking of" when it's statistically appropriate.

numtel• 3 years ago

With the code execution plugin, one could theoretically ask chatgpt to generate a salted hash of their answer at the start that's revealed at the end to prove it was correct.

Without any plugins, chatgpt will happily return sha hashes and salts when I asked it to play rock paper scissors this was. The only trouble was, the hashes were totally wrong.

waffletower• 3 years ago

Q: 'I present to you a present of being present in the present. What have I done?'

A: 'Your sentence also uses a clever play on words! In this sentence, the word "present" is used in multiple ways:

As a verb, "I present" means that you are giving or showing something. As a noun, "a present" refers to a gift or something that is given. As an adjective, "present" describes the feeling or experience that you are currently having. "In the present" refers to the current time or moment. So, your sentence means that you are giving a gift or offering of the experience of being mindful and aware of the current moment. Well done!'

I would argue that this is a fairly sophisticated modeling of this specific variety of linguistic ambiguity. Probably lies within GPT4's 32% performance score.

digging3• 3 years ago

Doesn't seem to me there's any ambiguity in that sentence. Each use of "present" only has one possible meaning that makes grammatical sense (with the possible exception of the final use).

seccode• 3 years ago

Ambiguity is not compressible, as ambiguity scales exponentially with linear input

mythrwy• 3 years ago

I asked gpt-4 this question: "A swan, a chicken and three mice are on the banks of the river. The mice wish to go to the other side so the swan offers to carry them. The mice climb on the swan's back. How many animals are on each side of the river?"

gpt-4 confidently stated that the swan and three mice were on one side, the chicken on the other. So I asked it "ok, but where in the story does the swan enter the water?". The reply was "Apologies! yada yada", then it told me all 4 animals were on the same side.

So yes, it does have trouble with ambiguity in my opinion.

thomastjeffery• 3 years ago

The problem that arises when you fail to interpret an ambiguous statement is that you end up on the wrong semantic path, and start asking and answering the wrong questions.

That's what's happening all around LLMs; and it's even happening right here in this post.

The very word AI is ambiguous. Does it denote a category of pursuit (AI research), or the end goal realized (an AI)? This is an incredibly important distinction that is entirely glossed over every time someone uses "AI" in the title of an LLM. Is the LLM simply in the category of AI research, or is it actually an Artificial Intelligence? It's the category. That's obvious, isn't it? It really should be; because otherwise we are treading down the wrong semantic path, and talking about an LLM personified, instead of an LLM in reality.

This problem goes even deeper. The very name "Large Language Model" is ambiguously misleading in the same way. Does "Language" define the content being modeled or the model itself? In this case, neither: which is where this conversation gets really interesting.

The entire purpose of an LLM is to process language without completely falling apart at ambiguity. An LLM accomplishes this by doing something else entirely: it process text instead.

To understand this, we need to understand the key limitation of traditional language parsing: it must always be literal. Everything written must be unambiguously defined, or the parser will fail to read it. This means that parsers are limited to the category of "context-free grammar".

Natural Language is in the category of "context-dependent grammar". It can contain ambiguity, which may be resolved with context.

An LLM doesn't do that. In fact, an LLM doesn't define anything at all! LLMs didn't overcome the limitation of parsing: they flipped it around: an LLM can never be literal. Instead, it must always be literary. Let me explain what I mean by that:

To construct an LLM, we start with a training corpus: lots of text. That text goes through a single traditional parsing step: tokenization. This isn't strictly necessary, but it's more efficient, and the rest of the process is anything but. Unlike traditional parsers, tokens are intentionally misaligned with grammar: words are split into separate tokens, like "run,ning".

Now that we have tokens to work with, machine learning begins. A Neural Net is trained with them. The tokens are fed in order to the NN, and the result is a model.

We call that model a "Large Language Model", because we hope it contains the patterns language is made of. This is a mistake: the model is so much more interesting than that!

An LLM contains patterns that we can recognize as language grammar and patterns we don't understand at all! It didn't model language: it went one step higher on the ladder of abstraction, and modeled text.

There are many ways to write an idea into language, but we can only use one at a time. That decision is part of the data we feed into the LLM's training corpus. We can't write all of our ideas at once: we must do one at a time in order. All of that is data in the training corpus.

Parsers deal with grammar. Language Grammar is everything that can be written in a language. An LLM doesn't have a clue what we could have been written: it only sees what was written. That's why the model must be "Large": without examples, a valid language pattern doesn't exist.

This is where the nature of ambiguity intersects with the nature of LLMs: what example do we want? Can we choose?

When we give an LLM a prompt, it gives us a continuation. There were many valid possible continuations: how did it choose just one? It didn't. It isn't working in the realm of possible: it's working in the realm of known. The choice was made all the way back before the LLM was even trained: back when a person wrote an idea into language, into text, that would eventually be used in the training corpus. The content of the training corpus is what resolves ambiguity: not the language model itself.

LLMs don't model ambiguity or even language: they model the text they are trained with, ambiguity included. This is a fundamental feature.

nuancebydefault• 3 years ago

I just played '20 questions' with chatgpt 3.5.

Even though i had to help, and 20 answers were not enough, I was impressed about:

- it understood that it could continue after 20 questions when i suggested that as part of the answer.

- it 'felt' that a could begin to become impatient and asked politely for a hint.

- it understands ambiguity enough to give me examples of where there was no clear yes or no reply possible.

When asking which were such examples for my thought 'daylight', it answered :

"Is the thing you're thinking of a type of force or energy that is said to exist but cannot be directly observed or measured?" - While daylight is a type of energy, it can be directly observed and measured through instruments that can detect its intensity and color.

"Does the thing you're thinking of have any connection to time or the concept of time?" - While daylight is not directly related to the concept of time, it is affected by the time of day and the position of the Sun in the sky.

We're afraid language models aren't modeling ambiguity

Related Stories