GPT-3 vs. 3M free-text trivia responses

https://www.watercoolertrivia.com/blog/gpt-3-vs-water-cooler-trivia-participants-a-human-vs-robot-showdown

by cowllin • 5 years ago

103 43 5 years ago

minimaxir3• 5 years ago

Credit where due: friend of WCT Dennis wrote the script to feed Water Cooler Trivia questions to GPT-3 and access the OpenAI API.

It should be noted it's against OpenAI's rules to share access to GPT-3, although they've been inconsistent about enforcing it.

cowllin2• 5 years ago

We gave him a spreadsheet of questions and he gave us a spreadsheet of GPT-3 responses, so probably not actually shared access :)

But point well-taken!

sinned• 5 years ago

And yah, I hope I didn’t violate their terms by doing this —- it seemed like a fun thing to do!

agravier• 5 years ago

2. Clues confuse GPT-3.

They should probably have been removed. This gives me the overall impression that the testers treat GPT-3 a bit too much as something like an artificial human, and not enough like an algorithm (which will work better with sanitized input). This is not a major criticism, the experiment is still interesting.

Could it be that the marketing from OpenAI it to blame? From the OpenAI front page:

Discovering and enacting the path to safe artificial general intelligence.
Our first-of-its-kind API can be applied to any language task, and currently serves millions of production requests each day.

Does that seem misleading?

cowllin2• 5 years ago

I thought about removing those two-word clues but ultimately, they exist for users so I wanted to be closer to apples-to-apples!

minimaxir3• 5 years ago

OpenAI's marketing has been fair, and it's not misleading (GPT-3 can be applied to any language tasks but your mileage will vary as this submission demonstrates).

However, OpenAI's endorsement of the demos amplifying and anthropomorphizing GPT-3 as a sentient mind don't help, and it's been disappointing that OpenAI doesn't really push back on that. (the hype is what prompted my rebuttal on GPT-3 expectations, which still holds up: https://news.ycombinator.com/item?id=23891226)

I do believe the future of AI text generation is more bespoke, algorithmic-input-friendly models which is what I've been working on as my side project.

2sk21• 5 years ago

Very good write-up. It seems too soon to think of genuine commercialization at this time.

wyldfire• 5 years ago

"Two'fer Goofer", "Tough Training" - why would GPT-3 give the question back as its response in these cases?

fpgaminer2• 5 years ago

GPT-3 has a habit of repeating its input. GPT-2 had the same problem, as did all language models before. LSTMs were really bad in this respect and would often get stuck in single word loops. GPT-3 just does it less often, and tends to repeat sentences or ideas, rather than single words.

A larger corpus and more training seemed to have helped reduce the issue, and I think they tweaked the temperature algorithms as well (I think top-p is an option now, whereas most language models used top-k historically).

In my experience GPT repeats itself more often when it doesn't understand the prompt. I mean, if you don't understand _anything_ about an input sequence, what is the most likely next word? One of the previously used words.

It's likely that the following occurs: Given the phrase "1+1=? ", the first layer assigns a baseline probability to "1", "+", "=", and "?", because those occur in the input and its a good fallback. But then the layers after recognize the input as a math equation and quickly assign a high probability to "2". Since the probability on "2" is so high the temperature algorithm always picks "2" as the output token.

But if we imagine GPT doesn't understand math, then all the layers after the first have no clue what the input is. They can't assign higher probability to anything. They might make a few weak guesses, say "duck". But it isn't confident in them, so really "duck" is as likely as "1". The temperature algorithm then has a fairly decent chance of selecting "1" as the output token. Now GPT is given "1+1=? 1" and it seems like "+" is the next logical token. "1+1=? 1+". And so forth until it GPT is stuck in self reinforcing a loop saying "1+1=? 1+1=? 1+1=? 1+1=?" etc.

Funny thing is, it's not like humans don't do this. When you get asked a hard question that you don't have an answer for, don't you find yourself repeating the question to yourself over and over again, hoping you'll better understand it? I'm not necessarily creating a parallel between that behavior and GPT's behavior. Just food for thought, I suppose.

aabhay• 5 years ago

Given the parallel you draw at the end, perhaps future models will adopt some kind of common phrase to denote thinking, such as ‘Umm’, ‘er’, etc. Maybe there’s even a neurologically concise explanation for why humans do that!

notahacker2• 5 years ago

Isn't it also that in human writing (and training) the most common or correct response/followup to a prompt structured as a question often is a complete "x is y" sentence? GPT-3 appears to consistently parse these question prompts as requiring it to include certain strongly associated words in its output, but not consistently as trivia questions implying terse, associated-word-only answers

It's the opposite of the 'know all' the blog describes, it's GPT-3 knowing where to find the answers without even understanding it's a quiz!

jon-wood• 5 years ago

We don’t have to imagine GPT3 doesn’t understand maths. It can reliably answer 1+1= because there are so many instances of it in the corpus, but if you give anything more complicated it will just throw out a random, incorrect, answer in most cases.

darepublic• 5 years ago

How is GPT3 considered publicly available? I applied months ago and have received no response.

nealdotpy• 5 years ago

I definitely agree it's not publicly available. I too applied in Dec 2020 and just the other day (09 March) received a key. Hang in there, it'll come :)

flemhans• 5 years ago

Applied in June and haven't heard anything!

fpgaminer2• 5 years ago

Somewhat related, but I can't stop thinking about what's going to happen when GPT-4 reads articles like this.

Articles like these that discuss GPT-3's output, or comments and articles directly generated by GPT-3, are going to be fed into GPT-4 as part of a theoretical WebText3 dataset. Will that help or hurt GPT-4's learning? What effect will it have?

Theoretically you can feed GPT-3's responses back into itself and ask it to introspect about them; whether it was right or wrong and to comment on why it gave the response it did. But I doubt GPT-3 is particularly good at self introspection. GPT-3 was trained before GPT-3 existed, obviously. So it was never trained on articles analyzing the output of an AI of its caliber.

But GPT-4 is going to be trained on a corpus filled with people analyzing GPT-3's outputs, like this article. We would expect GPT-4 to be able to write an article like this. So it should be theoretically possible to give GPT-4 its own output, and then ask it to provide introspection, and for that introspection to be insightful.

EDIT: Follow up thought. It's almost as if the internet is being filled with a training corpus on GPT-3's failings. Every fact that GPT-3 failed to learn from WebText2 is now going to be repeated, alongside the correct answer, in WebText3. Humans are globally working, unknowingly, to build a curated dataset by which GPT-4 can learn from GPT-3's mistakes.

mrfusion• 5 years ago

Somewhere in it’s billions of parameters it will have a sense of self. A model of itself. That’s pretty mind boggling.

bjourne2• 5 years ago

All text generated by current language models is defective. For reasons that are not fully understood, text language models emit is less interesting than text emitted by humans. Unlike humans, they can not score well on both coherence, as in grammatically correct sentences, and interest, as in "having something to say". They are however good at "coherent rambling".

Since most language models work by using its outputs as its inputs, this means that the emitted text becomes worse the more words that are emitted. Suppose you want your model to emit a novel. The first 30 words might come out ok. The next 30 are so-so. Eventually, after a few hundred words, the model will begin to ramble and will get stuck in loops: "then she went to visit her mother and she went to visit her mother and she vent to visit her mother and she ..."

All language models suffer from this problem; Markov models, RNNs, Transformers, GPT-2/3... In fact, one could argue that GPT-3 has "only" decreased the deterioration speed. Markov models deteriorate after a dozen words while GPT-3 can keep going for perhaps a hundred words before it begins to ramble. However, the statistical properties of the text GPT-3 generates are different from what a human emits (and what those "statistical properties" are is unknown - it appears that capturing what makes human prose human-like in a mathematical formula is very difficult) from the very first word so you can't use GPT-3-generated text to train GPT-4. The paper "The Curious Case of Neural Text Degeneration" describes the problem in more detail and with math.

A major breakthrough in language modeling may come if someone is able to invent a model that doesn't degenerate. One whose thousandth word is as human-like as its first word.

minimaxir3• 5 years ago

That's a No True Scotsman way of looking at language models. GPT-3 or any Transformer model doesn't need to be innovative to be successful. But that's why it's important to run experiments and build test cases.

In some cases, degeneration can actually be a feature: https://twitter.com/dril_gpt2

wpietri• 5 years ago

Great points. I suspect a non-degenerative model is orders of magnitude harder, in that good long prose requires much deeper complexity.

For example, I was just reading this excellent long piece from Rebecca Traister: https://nymag.com/intelligencer/article/andrew-cuomo-miscond...

Even if a language model had all the raw facts needed to write a long piece, there's a ton of structure here. She develops ideas, builds themes, assembles a thesis. She is not just going somewhere, but carefully taking us along with her. That requires modeling not just textual relationships, but taking conceptual relationships and expressing them in ways that require deep awareness of how the audience processes information.

So I'd be willing to bet a true non-degenerative model requires an increasingly complex and accurate theory of mind.

Der_Einzige• 5 years ago

The solution proposed in that paper, nucleolus sampling, effectively solves this problem though...

sprobertson• 5 years ago

For reasons that are not fully understood, text language models emit is less interesting than text emitted by humans.

What's not understood? As you describe, every standard language model is statistically predicting the next word given the previous words, ad infinitum. GPT3 has more parameters to build a more precise "thought" vector, but the goal is still only to produce the next word.

mannykannot• 5 years ago

That paper includes this observation:

Why is human-written text not the most probable text? We conjecture that this is an intrinsic property of human language... Grice’s Maxims of Communication (Grice,1975) show that people optimize against stating the obvious.

I don't think that should be at all surprising, if one purpose of speech is to convey information: predictability is the converse of being informational.

Furthermore, to be informative, it is not sufficient for a passage to avoid predictability; it also needs to be pertinent.

knuthsat• 5 years ago

The whole point of a model is to take advantage of data with more information. If GPT-3 output sucks for predicting human output then it will be ignored by the model during descent. Otherwise the learning algorithm is defective.

notahacker2• 5 years ago

The first part of your statement feels like it's anthropomorphising GPT-4. GPT-4s response to learning that GPT-3 is bad at word play will to store the concept of GPT-3 being bad at word play to recall if prompted about GPT-3 in future, not to swot up on word play. If you want GPT-4 to be better at word play you'll need a corpus of puns, rhymes and higher level concept association and a lot of training on replicating them and when to recognise a prompt might require them.

tsimionescu2• 5 years ago

Your ascribing way too much intelligence to a very simplistic text prediction algorithm. Neither GPT-3 nor GPT-4 (assuming it will just be a bigger GPT-3) have any concept of identity that could influence their predictions in the way you seem to assume.

If GPT-4 would be trained on text produced by GPT-3 and on text that analyzes that, what GPT-4 would learn is how to sound like GPT-3 and like detractors or admirers of GPT-3.

While impressive achievements, GPT-2/3/4/X are just models that predict what the most likely next word is based on a huge corpus of text. There is no model of the world in there, no objects, no identities. Just probabilities of glyphs in certain orders, without any meaning attached.

haolez• 5 years ago

I mostly agree with you, but I wonder what's the distance between what you described and our actual intelligence.

johbjo2• 5 years ago

One intuition about the large GPT-models is that they are learning "facts" like they are learning grammar rules. It's just that their capacity is so immensely large as to learn facts as part of the language model.

gwern2• 5 years ago

"The robot was best at Fine Arts and Current Events, worst at Word Play and Social Studies. ...This one’s not so surprising. We have a type of question called a “Two’fer Goofer” which asks for a pair of rhyming words that satisfy a given clue. It’s similar to the Rhyme Time category in Jeopardy or the old newspaper puzzle Wordy Gurdy. We had three of these questions in the showdown and GPT-3 missed all three of them. For Word Play questions that were more like vocabulary quizzes, GPT-3 performed admirably:"

More evidence, if it was needed, that the use of BPEs sabotages GPT-3 in a lot of subtle ways (https://www.gwern.net/GPT-3#bpes). GPT-3 can understand things like vocab definitions which do not depend on the internal spelling of a word or phonetics (which are erased by the BPE encoding of the data it was trained on), but as soon as you have to do things like puns... The BPEs are deadly. Ah well. Eventually OA or someone will train a proper character-level model, and then I'll be able to generate rhyming poetry without hacks like rhyming dictionaries.

hakuseki• 5 years ago

The more important takeaway: dozens of technologists at IBM spent more than three years and untold millions of dollars building the program specifically trained for Jeopardy! prowess. Less than 10 years later, a general-purpose open-sourced technology without the massive mainframe or cooling fans can compete on the same level.

This statement from the article confused me. GPT-3 is general-purpose, but not open source, nor does it run without massive hardware, nor was it developed without spending millions of dollars.

atleta• 5 years ago

Yep, it's not only open sourced, it's not even open access. (I get the reason, but somehow, maybe due to the fact that it's from an organization called OpenAI, a lot of people seem to think that everyone could use it if they wanted to.)

gwern2• 5 years ago

You could compare it to T5 instead. T5 does awesome at trivia, both the code and trained models are available, and it'll run on ordinary hardware.

(Not sure if one could argue 'the millions of dollars' part, but that GB R&D wasn't spent for the purpose of trivia playing like the Watson Jeopardy system was aimed at doing, so seems reasonably fair to me - the trivia playing is merely the byproduct of the target capabilities of translation, knowledge graph construction etc).

blueblisters• 5 years ago

Could GPT-3 lower barriers of entry for search engines? Assuming, it could update its "index" (weights) at regular intervals, it's not too far-fetched to see it competing with Google with high-quality relevant answers to queries.

ronsor• 5 years ago

I've tried asking GPT-3 factual questions and sometimes it does great. Other times, well... let's just say it doesn't quite understand. Bad answers are in italics.

Q: How tall is the Eiffel Tower? A: About twenty feet.

Q: What color are the clouds? A: They're a light blue.

Q: Which way does the sun rise in Paris? A: You'd have to be standing on top of a building to see it because it doesn't rise that high.

Q: Who invented the telephone? A: Alexander Graham Bell.

Q: Who invented Google? A: Larry Page and Sergey Brin.

Q: What's the capital of California? A: It's not located in California. It's Sacramento.

ctl0• 5 years ago

There are some works resonate with your idea, here's the first one I can recall.

https://arxiv.org/abs/2012.14983

dumbsqr• 5 years ago

I think than human communication is better than any ai model because human goes from general to concrete, adding more information to convert general context into insightful ideas. We use language to communicate what is important for us. Ai is about navigating in an ocean trying not to be captured by concrete things, because it knows nothing about being real.

As a mathematical methafor, each word is a window in the tangent space of an n-dimensional sphere and the normal points to the center of the sphere, as time goes on communication is about shrinking the radious of the sphere so that the center is like the kernel of what we wish to communicate. The sphere in an ai system has a very big radius and is not able to shrink because it lacks a kernel.

You could simulate a human conversation with an ai, for example to sell something following steps like description of a product, price advantages and so on, but in a real context those are ad hoc methods and there is no general rule to shrink that is to penetrate deep into the meaning of what you want to communicate.

johbjo2• 5 years ago

Less than 10 years later, a general-purpose open-sourced technology without the massive mainframe or cooling fans can compete on the same level.

First, are GPT-3 models generally available?

Secondly, GPT-3 is many orders of magnitude larger than any model existing 10 years ago. There are certainly cooling fans involved.

vessenes• 5 years ago

It would be helpful to see the prompt structure used for gpt-3: state of the art prompting methods do significantly better than naive ones, and randomness and other settings can make a big difference in quality as well.

There are also cyclical prompt methods which can help derandomize gpt3, for example, you could generate three to five answers per question then feed the set back to gpt-3 and ask it which is the most correct; the first round would use high randomness settings the last very low.

My own experience is that you sort of have to work with gpt3 to get on the same page sometimes, and when you do the results can be remarkable.

Anyway, fun idea!

kashyapc• 5 years ago

There was recently a thread[1] about an open-source alternative to GPT-3 here, called "GPT-Neo".

I hope it takes off. But the language on their website undermines some confidence: "GPT-Neo is the code name for a series of transformer-based language models [...] we plan to train and open source."

PS: The original URL that was submitted to HN is now 404; it moved to here[2].

[1] https://news.ycombinator.com/item?id=25819803

[2] https://www.eleuther.ai/projects/gpt-neo/

GPT-3 vs. 3M free-text trivia responses

Related Stories