Large language models can do jaw-dropping things. But nobody knows why.

https://www.technologyreview.com/2024/03/04/1089403/large-language-models-amazing-but-nobody-knows-why/

by branko_d • 2 years ago

65 23 2 years ago

sublinear• 2 years ago

one of the biggest scientific puzzles of our time
a hidden mathematical pattern in language that large language models somehow come to exploit... the fact that these things model language is probably one of the biggest discoveries in history

Is it? Excuse my ignorance. If we were talking about simple Markov chains, aren't these just stationary distributions?

psyklic• 2 years ago

Even if LLMs are thought of as Markov chains, there are still substantial unsolved scientific questions.

For a given prompt/"state", an LLM essentially computes the next-state probabilities. This is done by compressing language and storing particular patterns/distributions. To date, we still don't understand what statistics are stored by LLMs. Or even what stats might be necessary to produce natural-sounding language. (A canonical Markov chain only works with n-gram statistics, but we know these are insufficient.)

IMO figuring this out to a human-level understanding could be a major breakthrough in science. It would reveal a deeper structure to language. It may even help understand how our brains process language; we'll at least know one plausible algorithm. Prior to LLMs, many scholars assumed language was unique to the brain and could only poorly be "computed".

yetihehe• 2 years ago

Is it? Excuse my ignorance. If we were talking about simple Markov chains, aren't these just stationary distributions?

"- You're just a robot, you can't write a poem

- Can you?"

It seems that GPT3.5 is already a much better writer than I am, and I'm still much better than a Markov chain, so LLM's are verifiably better.

dmz73• 2 years ago

(Most) LLMs have been trained on more data than any human will ever be able to read, let alone process. Then they have been tweaked to produce human like output (most of the time). They can just as easily be tweaked to produce incomprehensible rubbish. This means that they can generate convincing human like output for any possible combination of data they were trained on and this output will usually be as good or better than that of average modern human that has consumed and comprehended tiny amount of data (in comparison). But even the dumbest of humans can be assumed to understand what they know where LLMs are just giant repository of numbers. LLMs have no capacity to understand what these numbers mean or what their output represents, that requires a human like intelligence. "AI" term has been grossly overloaded (just a couple of decades ago, a complex if/then/else statement was touted as "AI") and as we gain better and better understanding of information science, we assign that term to more sophisticated information systems we build but these systems are still far below of any threshold we would accept as sign of intelligence from another human being. Better term for these system would be Complicated Parrots rather than Artificial Intelligence. Intelligence requires understanding and understanding, I believe, requires sentience.

phero_cnstrcts• 2 years ago

Had to press close on 4 popups before I could read anything and then another one as I quickly pressed the back button.

sibeliuss3• 2 years ago

And why do you not have an adblocker installed?

mdavid6262• 2 years ago

Mobile?

stephenr3• 2 years ago

I'm 100% behind ad blockers in a browser, but this is some grade A victim blaming right here.

astrange• 2 years ago

It's not surprising that people don't know how GPT-4 works, because they don't have access to the model. Thus making it not an LLM but instead an API to a text generating oracle.

refulgentis• 2 years ago

I believe the article discusses LLMs in general, not GPT-4.

mo_42• 2 years ago

Large language models can do jaw-dropping things. But nobody knows exactly why. And that's a problem.

LLMs at least the most common ones can talk but they don't do things.

But even if they could. Humans can do it too, even your pet does jaw-dropping things. So far we never said that it's a problem in these cases.

So for me, such statements mostly communicate some sort of fear or skepticism. And I'm not saying that we shouldn't investigate why LLMs can do it. We should rather call it a research problem.

an_aparallel• 2 years ago

Yes, im sure OpenAI, and Google are paying incredibly high salaries to experts who dont know how LLM's work. I'm far from a computer scientist, or mathematician, but how the hell does this narrative keep being spouted by folks with a straight face? Oh, I know...no one can think critically...

branko_d• 2 years ago

No one knows how LLMs work in a sense that nobody knows how human brains work. Yes, we know a lot about individual cells, but the complexity that emerges when billions of them get connected is something that we do not understand. There is an argument that we'll never be able to fully understand our brain, because that would require more complex brain, which would make it more difficult to understand, which would require more complex brain etc...

Similarly, LLMs have "hidden layer". We know everything about individual "neurons", but we couldn't connect them together ourselves in a way that training somehow does. The network that is the result of that training is "hidden" from our understanding even though its individual nodes are plain to see.

Since AI can increase in complexity while human brain can't, there may come a day when AI understands the human brain, without the human brain ever being able to understand AI.

Large language models can do jaw-dropping things. But nobody knows why.

Related Stories