Meta AI chief says large language models will not reach human intelligence

https://www.ft.com/content/23fab126-f1d3-4add-a457-207a25730ad9

by belter • 2 years ago

42 26 2 years ago

belter• 2 years ago

winternett• 2 years ago

If you, like me, have witnessed the shear idiocy and disinformative mess of comments and content on Reddit, TikTok, Instagram and other so-called data sets for Ai training that seek engagement and watch time, it's easy to see that training Ai on that data will never work right. besides that, no amount of scripting will make Ai tools be able to mimic human intelligence, it's going to be something far different that can build and improve on itself after a point, rather than just being scripting that is continually updated by humans.

Based on the shear malfunction I observe in mobile OS assistant apps, we're a long way from true Ai... Coupled with so many fraudsters trying to capitalize on the ideal, and how everyone is converting free services to paid services with common street=level-drug-dealing tactics, I think the biggest issue these efforts will face is loss of credibility and shear burnout on all the overblown marketing of Ai. Current products are vastly underwhelming, criminally overpriced, and low-value.... There's a long way to go.

pyuser583• 2 years ago

It’s really bad. Especially when you consider how old ideas hang around after they’ve been disproven.

rgrmrts2• 2 years ago

I have a fairly skeptical view on the current AI hype cycle and the people pushing it as a panacea, and it has been somewhat personally frustrating to see AI being shoved into every product.

To me it seems Yann is one of the few leaders in this space (at least one who is as public as he is) who is being realistic about LLMs, their capabilities, and their impact on computing. I’m glad he’s a countermeasure to the exaggeration and, dare I say, grift.

reducesuffering2• 2 years ago

The biggest grifts here are the VC’s and megacorps looking to make billions without a damn to the consequences.

Yann, unlike Geoff Hinton and Yoshua Bengio, is a consequence of

“It is difficult to get a man to understand something, when his salary depends on his not understanding it.” - Upton Sinclair

The greatest prank these $-focused people ever pulled on you is that the $ agnostic scientists and researchers who never pursued fortunes are the grifters…

FFS Kokotajlo was giving up 85% of his wealth just for the potential to criticize OpenAI and Scott Alexander just gives his blog proceeds away to research https://www.astralcodexten.com/p/acx-grants-results

staticman22• 2 years ago

As far as I know Scott Alexander isn't an A.I. expert, or even a pretend one on the internet.

coffeefirst• 2 years ago

Yes... my new favorite use of ublock origin is to find the selector around the little sparkle icon that let's you autoreply "sounds good!" and banish it from my site.

pennybanks1• 2 years ago

I feel like its very deliberate what the companies are doing. They want to build hype. as much hype and money as they can afford. because look what its doing, look at the markets, the evaluations, its all part of some wheele to get consumers to get excited about their brands their future products etc. they are generating hype to generate money.

i mean at the end of the day after its all said and done. who cares if no one wants to use any ai feature anymore. i meaan openai became a trillion dollar company and they move onto the next. microsoft doubles in value they move on. etc.

its just annoying they even have the capabilities to do what they are doing. and its almost like we have no choice in the matter. until of course we choose with our wallets eventually but still

flembat2• 2 years ago

On the other hand, any software that does not include AI asistance seems lifeless to me now. Why isn't this text box suggesting ideas to me...

rgrmrts2• 2 years ago

Interesting! I’m very much the opposite. I never liked autocomplete or suggestions in text boxes (to a certain extent, e.g. I have common things like name and address saved in my password manager to suggest autofill for online shopping or whatever).

az226• 2 years ago

They already have in many areas. Beyond human intelligence in some as well.

What a dumb take.

Half his opinions are dead wrong.

LLMs will evolve from single model inference to multi-stage inference, which may involve a collection of models and data integration.

ohthatsnotright• 2 years ago

Maybe you can try showing how LLMs score better than human intelligence and what metrics you're using to quantify that, and then refuting his arguments with research and supporting evidence. Right now your take looks like you're a shill hoping to cash in on this bubble before it pops.

x86x87• 2 years ago

I do not believe you know what you are talking about. Let me ask you: what is intelligence? If this thing is intelligent and you remove humans completely what are you left with?

croes• 2 years ago

Against what human intelligence are you measuring?

appplication2• 2 years ago

Yes. The root here is that large language models produce human-like passages of text, but they are devoid of any specific intelligence, per se. They can sound quite convincingly like your average internet denizens, but fundamentally - and critically - lack the ability to differentiate between truth and fiction.

There are a lot of cool things you can do with models that don’t require you to differentiate fact from fiction. For example, creative prompts, and to some degree customer service chats etc. But these are not intelligence, not do they demonstrate anything resembling it.

It seems as though there are a lot of very smart people who have a hard time wrapping their heads around this, including, somewhat incredibly, some of the top researchers in the field. I think there is, in some researchers, a deep, Frankenstein-like desire for their creations to truly be something, and we see this manifest in unfounded, religion-like delusions. All despite these same folks carrying a deep academic understanding of the true nature of these models.

throwaway482945• 2 years ago

Does anyone truly understand these models? I don't think we have any proofs about the upper limits of what LLMs are capable of. How can you be so confident?

To be clear, I am not saying there are no limits to what LLMs can do, I just don't get how people can be so sure one way or the other. Especially when you consider that this technology is evolving at such an unpredictable pace.

appplication2• 2 years ago

We do actually understand generally well enough what is happening. Attention isn’t some mysterious unexplained mechanism. We know how it works and why. When people describe these models as a black box, they typically mean that there are too many layers and weights to explain to you exactly why it chose, for example, a specific sequence of words. But we can certainly explain exactly why it would chose some sequence, and why that sequence would be expected to be relevant.

Simplifying a bit, but attention provides a way for the model to build context on one word based on how often it is seen with others. It doesn’t have a concept of correct or incorrect. It doesn’t have a concept of reasoning.

What is impressive is that even without these concepts of correctness and reasoning, the model can still perform quite well on tasks where correctness and reasoning would be expected. But this is more a statement on the corpus of knowledge and the power of language in general than it is on the models capabilities itself. It’s important not to confuse the ability to seem correct and seem well reasoned with any actual mechanism to do so.

opprobium• 2 years ago

Recent example of a proof regarding theoretical limitations of Transformers: https://aclanthology.org/2023.tacl-1.31.pdf (also extended to cover SSMs https://arxiv.org/pdf/2404.08819)

blurbleblurble• 2 years ago

Tool augmented LLMs will reach wild levels of unhuman synthetic intelligence though.

more_corn• 2 years ago

You sure? People are pretty dumb. Wasn’t there a research article recently suggesting that people who aren’t concentrating aren’t general intelligences?

flembat2• 2 years ago

I think we imagine the models have emergent properties, because we have never in human history been able to have a meaningful two way conversation with non human entities before. So we believe in them. Remember how in the 1970s people believed that computers must be correct. There was no understanding of how they actually worked.

ofrzeta2• 2 years ago

'LLMs had “very limited understanding of logic . . . do not understand the physical world, do not have persistent memory, cannot reason in any reasonable definition of the term and cannot plan . . . hierarchically”'

So now back to symbolic AI?

Meta AI chief says large language models will not reach human intelligence

Related Stories