Yann LeCun on why LLMs and AGI are "total BS" [video]

https://www.youtube.com/watch?v=7u-DXVADyhc

by Der_Einzige • 7 months ago

2 1 7 months ago

jascha_eng• 7 months ago

I don't have 2 hours but I asked Gemini for a summary because it has a good YouTube integration some interesting points imo but not sure I wanna watch the whole thing:

This video features an in-depth interview with Yann LeCun, Chief AI Scientist at Meta and a Turing Award winner, hosted on The Information Bottleneck podcast. LeCun discusses his new startup, the limitations of current Large Language Models (LLMs), his vision for "World Models," and his optimistic outlook on AI safety.

Executive Summary Yann LeCun argues that the current industry focus on scaling LLMs is a dead end for achieving human-level intelligence. He believes the future lies in World Models—systems that can understand the physical world, plan, and reason using abstract representations rather than just predicting the next token. To pursue this, he is launching a new company, Advanced Machine Intelligence (AMI), which will focus on research and productizing these architectures.

Key Insights from Yann LeCun 1. The "LLM Pill" & The Limits of Generative AI LeCun is highly critical of the Silicon Valley consensus that simply scaling up LLMs and adding more data will lead to Artificial General Intelligence (AGI).

The "LLM Pill": He disparages the idea that you can reach superintelligence just by scaling LLMs, calling it "complete bullshit" [01:13:02].

Data Inefficiency: LLMs require trillions of tokens to learn what a 4-year-old learns from just living. He notes that a child sees about 16,000 hours of visual data in four years, which contains far more information than all the text on the internet [25:23].

Lack of Grounding: LLMs do not understand the physical world (e.g., object permanence, gravity) and only "regurgitate" answers based on fine-tuning rather than genuine understanding [36:22].

2. The Solution: World Models & JEPA LeCun advocates for Joint Embedding Predictive Architectures (JEPA).

Prediction in Abstract Space: Unlike video generation models (like Sora) that try to predict every pixel (which is inefficient and hallucinatory), a World Model should predict in an abstract representation space. It filters out irrelevant details (noise) and focuses on what matters [15:35].

The Sailing Analogy: He compares sailing to running a world model. You don't simulate every water molecule (Computational Fluid Dynamics); you use an intuitive, abstract physics model to predict how the wind and waves will affect the boat [01:30:29].

Planning vs. Autocomplete: True intelligence requires planning—predicting the consequences of a sequence of actions to optimize an objective. LLMs just autocomplete text [07:26].

3. A New Startup: Advanced Machine Intelligence (AMI) LeCun is starting AMI to focus on these "World Models" and planning systems.

Open Research: He insists that upstream research must be published openly to be reliable. Closed research leads to "delusion" about one's own progress [04:59].

Goal: To become a supplier of intelligent systems that can reason and plan, moving beyond the capabilities of current chatbots.

4. AI Safety is an Engineering Problem LeCun dismisses "doomer" narratives about AI taking over the world, viewing safety as a solvable engineering challenge akin to building reliable jet engines.

Objective-Driven Safety: He proposes "Objective-Driven AI". Instead of trying to fine-tune an LLM (which can be jailbroken), you build a system that generates actions by solving an optimization problem. Safety constraints (e.g., "don't hurt humans") are hard-coded into the objective function, making the system intrinsically safe by construction [01:02:04].

The Jet Engine Analogy: Early jet engines were dangerous and unreliable, but through engineering, they became the safest mode of transport. AI will follow the same trajectory [58:25].

Dominance vs. Intelligence: He argues that the desire to dominate is a biological trait tied to social species, not a necessary byproduct of intelligence. A machine can be super-intelligent without having the drive to rule humanity [01:35:13].

5. Advice for Students Don't Just Study CS: LeCun advises students to focus on subjects with a "long shelf life" like mathematics, physics, and engineering (control theory, signal processing).

Avoid Trends: Computer Science trends change too rapidly. Foundational knowledge in how to model reality (physics/math) is more valuable for future AI research than learning the specific coding framework of the month [01:36:20].

6. AGI Timelines He rejects the term "AGI" because human intelligence is specialized, not general.

Prediction: Optimistically, we might have systems with "cat-level" or "dog-level" intelligence in 5–10 years. Reaching human level might take 20+ years if unforeseen obstacles arise [51:24].

Yann LeCun on why LLMs and AGI are "total BS" [video]

Related Stories