Explainer: What's r1 and everything else?

https://timkellogg.me/blog/2025/01/25/r1

by Philpax • 1 year ago

262 121 1 year ago

The benchmarks for the different models focus on math and coding accuracy. I have a use-case for a model where those two functions are completely irrelevant and I’m only interested in writing (chat, stories, etc). I guess you can’t really benchmark ‘concepts’ as easily as logic.

With distillation, can a model be made that strips out most of the math and coding stuff?

ben302• 1 year ago

Yes, you can create a writing-focused model through distillation, but it's tricky. *Complete removal* of math/coding abilities is challenging because language models' knowledge is interconnected - the logical thinking that helps solve equations also helps structure coherent stories.

mercer• 1 year ago

I understood that at least some of these big models (llama?) is basically bootstrapped with code. is there truth to that?

NitpickLawyer2• 1 year ago

completely irrelevant and I’m only interested in writing (chat, stories, etc)

There's a person keeping track of a few writing prompts and the evolution of the quality of text with each new shiny model. They shared this link somewhere, can't find the source but I had it bookmarked for further reading. Have a look at it and see if it's something you'd like.

https://eqbench.com/results/creative-writing-v2/deepseek-ai_...

comrade12342• 1 year ago

Those outputs are really good and come from deepseek-R1 (I assume the full version, not a distilled version).

R1 is quite large (685B params). I’m wondering if you can make a distilled R1 without the coding and math content. 7B works well for me locally. When I go up to 32B I seem to get worse results - I assume it’s just timing out in its think mode… I haven’t had time to really investigate though.

afro88• 1 year ago

Here's a better link: https://eqbench.com/creative_writing.html

The R1 sample reads way better than anything else on the leaderboard to me. Quite a jump.

polotics• 1 year ago

Interesting article, but the flourish ending """AI will soon (if not already) increase in abilities at an exponential rate.""" is not at all substantiated. Would be nice to know how the author gets to that conclusion.

FridgeSeal• 1 year ago

It’s basically the line for all the AI-hype people: “all the problems are going away!”, “soon it’ll all magically make things exponentially good-er-er!”

brookst6• 1 year ago

Alternatively, it’s a restatement of the obvious empirical truth that technology tends in improve on an exponential and not linear curve. Seems like a simpler explanation that doesn’t even require insulting people.

sandworm1013• 1 year ago

The key "ability" that will grow exponentially is AIs ability to convert investment dollars into silicon+electricity and then further reduce those into heat energy. Such schemes only seem wasteful to outsiders, those whose salaries are not tied to their ability to convert money into heat. A fun startup would be one that generates useful electricity from the AI investment cycle. If we put the Ai machine under a pot of water, we might then use the resulting steam to drive a turbine.

grues-dinner• 1 year ago

Some datacentres do in fact recover the heat for things like municipal heating. It's tricky though because being near population centres that can use the heat is often (not always) inversely related to things that are good for datacentres like cheap land, power and lack of neighbours to moan about things like construction and cooling system noise.

There was also a startup selling/renting bitcoin miners that doubled as electrical heaters.

The problem is that computers are fundamentally resistors, so at most you can get 100% of the energy back as heat. But a heat pump can give you 2-4 times the energy back. So your AI work (or bitcoin mining) plus the capital outlay of the expensive computers has to be worth the difference.

immibis4• 1 year ago

Due to Carnot's law, you can't get much electricity that way without a big temperature difference. Think about it: the AI machine would have to run at at least 100 degrees Celsius to boil the water, and that's the bare minimum.

But if we can make computers that run at, say, 2000 degrees, without using several times more electricity, then we can capture their waste heat and turn a big portion of it back into electricity to re-feed the computers. It doesn't violate thermodynamics, it's just an alternative possibility to make more computers that use less electricity overall (an alternative to directly trying to reduce the energy usage of silicon logic gates) as long as we're still well above Landauer's limit.

ozten2• 1 year ago

Orbital Materials is designing wafer substrates that capture carbon and reuse excess heat.

tkellogg3• 1 year ago

Author here. I do believe it's going to be exponential (not yet), but that's out of scope for the article. However, if someone has a good explainer link for that, please put it here and I'll link it into the post.

jcgrillo3• 1 year ago

Care to justify those beliefs or are we just supposed to trust your intuition? Why exponential and not merely quadratic (or some other polynomial)? How do you even quantify "it"? I'm teasing, somewhat, because I don't actually expect you're able to answer. Yours isn't reasoned arguments, merely religious fervor dressed up in techy garb. Prove me wrong!

samsartor4• 1 year ago

All past data shows is exponential growth in the cost of AI systems, not an exponential growth in capability. Capabilities have certainly expanded, but that is hard to measure. The growth curve is just as likely to be sigmoid-shaped. Just a phase transition from "computers process information strictly procedurally" to "computers use fuzzy logic sometimes too". And if we've exhausted all the easy wins, that explains the increased interest in alternative scaling paths.

Obviously predicting the future is hard, and we won't know where this stops till we get there. But I think a degree of skepticism is warranted.

ioulaum• 1 year ago

The exponential part may be iffy, but it is self improving.

And this same RL is also creating improvements in small model performance.

So, more LLMs are about to rise in quality.

immibis4• 1 year ago

It's self-improving? So, we can ask AI how to improve AI, and the suggestions actually work?

the84722• 1 year ago

The drumbeat of AI progress has been fairly steady, on log scales.

https://time.com/6300942/ai-progress-charts/

amazingamazing• 1 year ago

that doesn't mean ai is improving itself though

JoshTko• 1 year ago

AI will improve at an exponential rate once it can independently improve AI performance. For example. Once AI can organically identify, test, confirm, deploy an improvement like R1 vs o1 (in terms of perf/watt) then we'll see exponential improvement. Honestly though, that still seems possible within 5 years or less, maybe 3.

amelius• 1 year ago

Only if the AI can do it faster than humans.

And if the improvements it makes are not asymptotically diminishing.

ksec• 1 year ago

Honestly though, that still seems possible within 5 years or less, maybe 3.

If that is a normal human estimation I would guess in reality it is more likely to be in 6-10 years. Which is still good if we get it in 2030 - 2035.

boznz• 1 year ago

Currently AI is getting better at sorting the data that already exists, but if enough reddit, wiki posts are wrong its answer is inevitably wrong. Without being able to experiment to test its theories against reality, the AI curve will likely not lead to super-intelligence without humans to assist. That's my 5 cents.

justlikereddit• 1 year ago

Short version: It's Hype.

Long version: It's marketing efforts stirring up hype around incremental software updates. If this was software being patched in 2005 we'd call it "ChatGPT V1.115"

Patch notes: Added bells. Added whistles.

Havoc• 1 year ago

people re-creating R1 (some claim for $30)

R1 or the R1 finetunes? Not the same thing...

HF is busy recreating R1 itself but that seems to be a pretty big endevour not a $30 thing

littlestymaar• 1 year ago

This is indeed a massive exaggeration, I'm pretty sure the $30 experiment is this one: https://threadreaderapp.com/thread/1882839370505621655.html (github: https://github.com/Jiayi-Pan/TinyZero).

And while this is true that this experiment shows that you can reproduce the concept of direct reinforcement learning of an existing LLM, in a way that makes it develop reasoning in the same fashion Deepseek-R1 did, this is very far from a re-creation of R1!

m3kw9• 1 year ago

Maybe they mistake recreation for the cp command

1123581321• 1 year ago

Nice explainer. R1 hit sensational mainstream news which has resulted in some confusion and alarm among family and friends. It’s hard to succinctly explain this doesn’t mean China is destroying us, that Americans immediately started working with the breakthrough, cost optimization is inevitable in computing, etc.

huqedato3• 1 year ago

I know that I'll get a lot of hate and downvotes for this comment.

I want to say that I have all the respect and admiration for these Chinese people, their ingenuity and their way of doing innovation even if they achieve this through technological theft and circumventing embargoes imposed by US (we all know how GPUs find their way into their hands).

We are living a time with a multi-faceted war between the US, China, EU, Russia and others. One of the battlegrounds is AI supremacy. This war (as any war) isn’t about ethics; it’s about survival, and anything goes.

Finally, as someone from Europe, I confess that here is well known that the "US innovates while EU regulates" and that's a shame IMO. I have the impression that EU is doing everything possible to keep us, European citizens, behind, just mere spectators in this tech war. We are already irrelevant, niche players.

travisporter2• 1 year ago

But Anthropic is a French company isn’t that in the EU

throw310822• 1 year ago

No it's American, you probably confuse them with Mistral AI.

huqedato3• 1 year ago

Anthropic is not French. You probably meant Mistral, that is a French company, but a niche player in this AI game.

hackandthink2• 1 year ago

"We are living a time with a multi-faceted war between the US, China, EU, Russia and others."

The only way to win this war is to deescalate. Everybody wins.

And AI competition is a good thing for Europe especially when it lags behind technologically.

quesera• 1 year ago

The only way to win this war is to deescalate

But there is absolutely no way that will happen, so the pragmatic question is which horse to bet on.

huqedato3• 1 year ago

I don't follow this "And AI competition is a good thing for Europe especially when it lags behind technologically." How that? Please explain your rationale.

whimsicalism3• 1 year ago

as an american, i never voted for this war, would like it to end, and think now is the time when international coordination is most critical

rerdavies• 1 year ago

The top three largest supercomputers in the world are located at Lawrence Livermore Labs, Oak Ridges Labs, and Argonne Labs. Each of them provides an architecture that's ideal for running AI.

One can't help wondering what kinds of classified AI results the US military is getting when running on El Capitan.

brookst6• 1 year ago

Is El Capitan really ideal arch for AI? It’s massively parallel of course.

didibus3• 1 year ago

You said a lot of controversial things. I'll just zoom in on the last bit:

I have the impression that EU is doing everything possible to keep us, European citizens, behind, just mere spectators in this tech war. We are already irrelevant, niche players.

Citizens of the US are just as irrelevant, if not more, since none of the productivity gains trickles down to them. Their real wage growth has stagnated since the 1970s, and each year that goes by, their actual power to purchase more goods or services goes down.

The victories in AI only matter to those who will profit from it.

When it comes to the citizens of a country benefiting from AI or not, being the leader in AI tech is not very important. It is more a matter of if AI benefits them or not. That their country has the leading AI tech can as easily results in them having less jobs, and being paid worse, as it could the opposite, depending on the policies of that country.

But given that, it can very much be better to live in the EU, with second grade open-source models, but where the productivity benefits of AI benefit the general citizens, then to live in the US, where the productivity benefits of AI benefit only the few.

simonw• 1 year ago

From that article:

ARC-AGI is a benchmark that’s designed to be simple for humans but excruciatingly difficult for AI. In other words, when AI crushes this benchmark, it’s able to do what humans do.

That's a misunderstanding of what ARC-AGI means. Here's what ARC-AGI creator François Chollet has to say: https://bsky.app/profile/fchollet.bsky.social/post/3les3izgd...

I don't think people really appreciate how simple ARC-AGI-1 was, and what solving it really means.
It was designed as the simplest, most basic assessment of fluid intelligence possible. Failure to pass signifies a near-total inability to adapt or problem-solve in unfamiliar situations.
Passing it means your system exhibits non-zero fluid intelligence -- you're finally looking at something that isn't pure memorized skill. But it says rather little about how intelligent your system is, or how close to human intelligence it is.

dartos2• 1 year ago

That's a misunderstanding of what ARC-AGI means

Misunderstanding benchmarks seems to be the first step to claiming human level intelligence.

Additionally:

> ARC-AGI is a benchmark that’s designed to be simple for humans but excruciatingly difficult for AI. In other words, when AI crushes this benchmark, it’s able to do what humans do.

Doesn’t even make logical sense.

ethbr13• 1 year ago

This feels like a generalized extension of the classic mis-reasoned response to 'A computer can now play chess.'

Common non-technical chain of thought after learning this: 'Previously, only humans could play chess. Now, computers can play chess. Therefore, computers can now do other things that previously only humans could do.'

The error is assuming that problems can only be solved via levels of human-style general intelligence.

Obviously, this is false from the way that computers calculate arithmetic, optimize via gradient descent, and innumerable other examples, but it does seem to be a common lay misunderstanding.

Probably why IBM abused it with their Watson marketing.

In reality, for reliable capabilities reasoning, the how matters very much.

antonvs2• 1 year ago

Misunderstanding benchmarks seems to be the first step to claiming human level intelligence.

It's known as "hallucination" a.k.a. "guessing or making stuff up", and is a major challenge for human intelligence. Attempts to eradicate it have met with limited success. Some say that human intelligence will never reach AGI because of it.

9dev• 1 year ago

Doesn’t that turn the entire premise on its head? If passing the benchmark means crossing the lower, not the upper threshold, that invalidates most claims derived from it.

cootsnuck• 1 year ago

Correct. Hence many people constantly bemoaning the hype driven narratives that dominate many AI discussions.

tkellogg3• 1 year ago

Ah! My bad, I edited the article to simply quote Francois. Thanks for catching this, Simon.

bikamonki• 1 year ago

So, is AI already reasoning or not?

ozten2• 1 year ago

Yes. ARC AGI benchmark was supposed to last years and is already saturated. The authors are currently creating the second version.

brookst6• 1 year ago

Depends on your definition of reasoning. Creating valid chains of thought? Yes. Sentient? No.

jug• 1 year ago

Yes, that's what OpenAI o1 does, and DeepSeek R1. Also Google Gemini 2.0 Thinking models. It's a way to significantly improve benchmark scores, especially in math.

It's funny to watch too. I played with Gemini 2.0 on Google AI Studio and asked it to "come up with your favorite song as you take a long walk to really think this through".

The reasoning can then be shown, and it talked to itself, saying things like "since I'm an AI, I can't take walks, but with a request like this, the user seems to imply that I should choose something that's introspective and meaningful", and went on with how it picked candidates.

erlendstromsvik• 1 year ago

I just tried that prompt with gemini-2.0-flash-thinking-exp-01-21

In the reasoning process it concludes on: From the brainstormed genres/artists, select a specific song. It's better to be concrete than vague. For this request, "Nuvole Bianche" by Ludovico Einaudi emerges as a strong candidate. Craft the Explanation and Scenario: Now, build the response around "Nuvole Bianche."

Then in the actual answer it proposes: "Holocene" by Bon Iver.

didibus3• 1 year ago

No. AI learns to predict reasons, and doing so as it predicts the answer improves its accuracy at predicting the answer.

In summary, even though they are called "reasoning" models, they are still based on prediction and pattern matching, not true logical reasoning. The improvement in accuracy is likely due to better leveraging of the model's statistical knowledge, rather than any deeper understanding of the problem's logic. And the reasons you see it output have nothing to do with the actual reasons it used to determine the answer.

In fact, R1.Zero hints that, it might be even better to let the AI follow a chain of thought that doesn't actually make logical sense or is understandable, and that doing so could even further improve its ability to accurately predict solutions to code, math and logic problems.

whimsicalism3• 1 year ago

this is a pretty hype-laden/twitter-laden article, i would not trust it to explain things to you

thih9• 1 year ago

Sure. But perhaps some hype is justified? Here's what a senior research scientist from nvidia says:

We are living in a timeline where a non-US company is keeping the original mission of OpenAI alive - truly open, frontier research that empowers all. It makes no sense. The most entertaining outcome is the most likely.
DeepSeek-R1 not only open-sources a barrage of models but also spills all the training secrets. They are perhaps the first OSS project that shows major, sustained growth of an RL flywheel. (…)

https://x.com/DrJimFan/status/1881353126210687089

rahimnathwani• 1 year ago

  Most important, R1 shut down some very complex ideas (like DPO & MCTS) and showed that the path forward is simple, basic RL.

This isn't quite true. R1 used a mix of RL and supervised fine-tuning. The data used for supervised fine-tuning may have been model-generated, but the paper implies it was human-curated: they kept only the 'correct' answers.

Philpax• 1 year ago

I think what you're saying is consistent with the quote: human curation of SFT data is indeed not complex. There might be extra work on top of that RL, but it's the same work that's been done throughout LLM development.

tkellogg3• 1 year ago

Additionally, in the following days, I've seen evidence suggesting that the SFT part might not even be necessary. I'd argue that work wouldn't have happened if R1 wasn't released in the open.

fullstackchris• 1 year ago

So the conclusion is AI is about to "increase in abilities at an exponential rate", with the only data point being that R1 was sucessfully able to acheive o1 levels as an open source model? In other words, two extremely unrelated themes?

Does this guy know people were writing verbatim the same thing in like... 2021? Still always incredible to me the same repeated hype over and over rise to the surface. Oh well... old man gonna old man

raincole2• 1 year ago

Does this guy know people were writing verbatim the same thing in like... 2021?

Given how far gen AIs have improved since 2021, these people were quite spot on.

WesleyJohnson• 1 year ago

Someone should author an ELI5 (or slightly older) guide to how LLMs, RL, Agents, CoT, etc, all work and what all these acronyms mean. And then, add to it, daily or weekly, as new development arise. I don't want to keep reading dozens of articles, white papers, tweets, etc, as new developments happen. I want to go back to the same knowledge base, that's authored by the same person (or people), that maintains a consistent reading and comprehension level, and builds on prior points.

It seems like the AI space is moving impossibly fast, and its just ridiculously hard to keep up unless 1) you work in this space, 2) are very comfortable with the technology behind it, so you can jump in at any point and understand it.

alotaluv• 1 year ago

haha just have gpt operator do it

esafak• 1 year ago

Just ask an Internet-enabled LLM like You.com to do it. This is what they are good that. Wikipedia satisfies your repository requirement.

raincole2• 1 year ago

People keep saying that DeepSeek R1's training cost is just $5.6M. Where is the source?

I'm not asking for the proof. Just the source, even a self-claimed statement. I've read the R1's paper and it doesn't say the number of $5.6M. Is it somewhere in DeepSeek's press release?

richardatlarge2• 1 year ago

T or F?

Nobody really saw the LLM leap coming

Nobody really saw R1 coming

We don’t know what’s coming

Explainer: What's r1 and everything else?

Related Stories