It Is Beginning: AI Improves Itself (auth: Sabine Hossenfelder) (+abridgement)

https://www.youtube.com/watch?v=QADKN3hantI

by mdp2021 • 1 month ago

2 2 1 month ago

mdp2021• 1 month ago

What happens when AI begins to improve AI? [...] Jimmy Ba, the co-founder of ExaAI, recently tweeted that recursive self-improvement loops likely do live in the next 12 months and that 2026 is going to be insane and likely the busiest and most consequential year for the future of our species. [...] Indeed, Anthropic just created an internal think tank as a fire drill to prepare for the intelligence explosion and to look for early warning signals for recursive self-improvement. They called it the Anthropic Institute. Itʼs led by co-founder Jack Clark, who put the odds above 60% that by 2028, an AI system could be told to make a better version of itself and then do it autonomously.
Though, seeing how things are going with the scientific literature, I think thatʼs an intelligence implosion. But I digress. My point is that there are many ways AI can self-improve in some sense, and yet itʼs not the sense that anyone had in mind when they were thinking of the intelligence explosion.
It was supposed to be a runaway effect caused by AI improving their own code that leads to exponential or super-exponential intelligence increase.
So, how close are we to that? I already last year about the beginning of self-improving AI with large language models that tune their own hyper parameters or on the more applied side AI thatʼs improved microchips used for AI training. Self-improvement, yes, but not the runaway effect weʼre waiting for. But, these examples have multiplied in the past months and some of these AIs do now indeed write new AI code. The clearest recent example is probably Andrej Karpathyʼs auto research. Itʼs a small Python setup where an artificial intelligent agent modifies language models training code, trains the model for 5 minutes, checks whether the result improved, keeps or discards the change, and repeats. Metaʼs hyperagents work like this, too. These are element-based research agents that write code, run experiments, debug failures, keep works, and toss what doesnʼt. These are not yet models directly rewriting their own brains, but it is artificial intelligence developing new artificial intelligence, albeit on a small scale.
Weʼre indeed getting closer to the self-recursive part. Meta, a nonprofit institute for model evaluation and threat research, is keeping track of this closely. In November, they asked whether we have already reached the point where AI can do AI research well enough to speed up the next AI. To answer this question, they tested GPT-5.1 codex max on some research engineering tasks like fixing a damaged small language model. In November, their answer was, "We have not yet reached the self-acceleration point." But, that was in November and at the pace that things are going, thatʼs basically Stone Age.
Already in February, one meta researcher published a model predicting that if current trends continue, AI could automate more than 99% of AI research and development around 2032.
Personally, I [Sabine Hossenfelder] think itʼll happen much sooner than that. So, this is no longer just someone on the internet says the singularity is near. The people who work in the field are now discussing when and how the recursive loop is going to close and how to prepare for that event. That said, so far artificial intelligence has not replaced scientists. Itʼs merely automated the part where we try 700 things that donʼt work and pretend that this was the plan all along.

Note: this is not the transcript - they are selected excerpts from the transcript.

Now who the f. silently hit the abridgement: it's clearly for everyone's convenience. The content is a video...

In case the voiceless (already enough to stay fully silent) hitter thought this was an automation: no, it's really made through one's "hands".

In case it's an autofilter: @dang, it would need finetuning.

pestatije• 1 month ago

cool...all those AI personnel can now be fired too

It Is Beginning: AI Improves Itself (auth: Sabine Hossenfelder) (+abridgement)

Related Stories