mxwsn

This is the same reasoning behind why Yann Lecun thought test-time scaling would not work for LLMs: compounding error.

Instead, the more tokens LLMs use, the better their performance on many tasks. LLMs can self-correct, evidenced by the power of getting models to question themselves by emitting "Wait," in S1. https://arxiv.org/abs/2501.19393

No, there are more training tokens than parameters in LLMs. They are in the classical first descent setting.

Here’s a thought experiment: suppose that a mathematician solved a major problem by having a long exchange with an LLM in which the mathematician played a useful guiding role but the LLM did all the technical work and had the main ideas. Would we regard that as a major achievement of the mathematician? I don’t think we would.

This is a cultural choice. It makes sense that in the mathematics culture we currently have, this is alien. But already, other fields, and many individuals, would disagree and say that the human did have a major achievement here. As long as human-AI collaborations are producing the best results, there is meaningful contribution by the humans, and people that are deeper experts and skilled LLM whisperers should be able to make outsized contributions. The real shoe drops when pure AI beats humans and human-AI collaboration.

Great summary. The fact that the auto encoding task is not grounded in thoughts, and their initial training on guessed internal thoughts, raise serious concerns on faithfulness. Feels like they might get better results by just training a supervised model on activations and "internal thoughts" measured by some different behavioral way.

Diffusion and flow matching models generate samples by iterative denoising. Iterative denoising means passing input to the neural network, running a forward pass, and taking the output back as input and rerunning the neural network. Often you do this 100 times, which is slow and expensive.

Flow maps / consistency models / shortcut models instead try to learn to compress this iterative work into 1 forward pass. This makes inference 100x faster as you'd only need to run the neural net forward pass once. Beyond speeding up inference, there are other advanced benefits to this, such as improved ability to perform inference-time steering.

Mathematically, learning a flow map corresponds to learning to solve an ordinary differential equation, i.e., learning the time integral of the velocity field. This mathematical foundation provides the basis for various training objectives for learning flow maps, which involve self-referential identities or identities such as the transport equation, which are discussed in the blog post.

Hope that helps! I'm an ML researcher currently researching flow maps.

How do you know that width scaling has been the driving force of improvement?

The Jacobian is first derivatives, but for a function mapping N to M dimensions. It's the first derivative of every output wrt every input, so it will be an N x M matrix.

The gradient is a special case of the Jacobian for functions mapping N to 1 dimension, such as loss functions. The gradient is an N x 1 vector.

Wow! The title suggests introductory material, but in my opinion this has strong potential to win test of time awards for research.

That's really interesting. What if they RAG search related videos from the prompt, and condition on that to generate? That might explain fidelity like this

Why is not the diffusion training objective? The technique is known as self-conditioning right? Is it an issue with conditional Tweedie's?

AI with ability but without responsibility is not enough for dramatic socioeconomic change, I think. For now, the critical unique power of human workers is that you can hold them responsible for things.

edit: ability without accountability is the catchier motto :)

Has anyone come across any really cool artifacts? I'd be curious to see

Stablecoins transferred $27 trillion in 2024 - more than Visa and Mastercard combined. This is right in the article.

Stablecoins operate using decentralized ledgers on e.g. Ethereum which use decentralized compute. This isn't mentioned explicitly because the target audience knows this already.

Gemini has beat it already, but using a different and notably more helpful harness. The creator has said they think harness design is the most important factor right now, and that the results don't mean much for comparing Claude to Gemini.

Huh, I imagined this was because of relaxing regulation.

Good read, thanks for sharing

But what is the original purpose of AI research? I will speak for myself here, but I know many other AI researchers will say the same: the ultimate goal is to understand how humans think. And we think the best (or the funniest) way to understand how humans think is to try to recreate it.

Eh. To riff on Dijkstra, this is like submarine engineers saying their ultimate goal is to understand how fish swim.

This ought to be called the qwerty effect, for how the qwerty keyboard layout can't be usurped at this point. It was at the right place at the right time, even though arguably its main design choices are no longer relevant, and there are arguably better layouts like dvorak.

Python and React may similarly be enshrined for the future, for being at the right place at the right time.

English as a language might be another example.

What's surprising about this is how sparsely defined the rewards are. Even if the model learns the formatting reward, if it never chances upon a solution, there isn't any feedback/reward to push it to learn to solve the game more often.

So what are the chances of randomly guessing a solution?

The toy Countdown dataset here has 3 to 4 numbers, which are combined with 4 symbols (+, -, x, ÷). With 3 numbers there are 3! * 4^3 = 384 possible symbol combinations, with 4 there are 6144. By the tensorboard log [0], even after just 10 learning steps, the model already has a success rate just below 10%. If we make the simplifying assumption that the model hasn't learned anything in 10 steps, then the probability of 1 (or more) success in 80 chances (8 generations are used per step), guessing randomly for a success rate of 1/384 on 3-number problems, is 1.9%. One interpretation is to take this as a p-value, and reject that the model's base success rate is completely random guessing - the base model already has slightly above chance success rate at solving the 3-number CountDown game.

This aligns with my intuition - I suspect that with proper prompting, LLMs should be able to solve CountDown decently OK without any training. Though maybe not a 3B model?

The model likely "parlays" its successes on 3 numbers to start to learn to solve 4 numbers. Or has it? The final learned ~50% success rate matches the frequency of 4-number problems in Jiayi Pan's CountDown dataset [1]. Phil does provide examples of successful 4-number solutions, but maybe the model hasn't become consistent at 4 numbers yet.

[0] https://www.philschmid.de/static/blog/mini-deepseek-r1/tenso... [1]: https://huggingface.co/datasets/Jiayi-Pan/Countdown-Tasks-3t...

I used sublime from 2013 to 2021. It was great. Since, I've switched to VS Code and haven't looked back.

Context is a challenge for LLMs, but the challenge feels of a different quality to me, than the challenge of incorporating local context into automated decision-making AI like algorithmic hiring, banking decisions, and real estate valuation like Zillow. These examples are more like "pre-LLM" machine learning, and it's not clear to me that LLMs are inherently limited in the same way. If anything, I think there's potential for LLMs to more flexibly handle a much broader variety of local contextual information by ingesting natural language rather than non-LLM machine learning systems where how to featurize or represent this information is typically quite bespoke. Take the neighbors' practicing death metal in their garage every Sunday and its impact on house valuation - it feels harder to get a non-LLM ML system to "understand" this, as a very sparse "feature", than an LLM.

My interest was piqued, but the extrapolation in [1] is uh... not the most convincing. If there were more data points then sure, maybe

OK - there's always a nonzero chance of hallucination. There's also a non-zero chance that macroscale objects can do quantum tunnelling, but no one is arguing that we "need to live with this" fact. A theoretical proof of the impossibility of reaching 0% probability of some event is nice, but in practice it says little about whether we can exponentially decrease the probability of it happening or not to effectively mitigate risk.

I think so. Parents can also make it happen at their convenience by asking doctors. We have technology to induce birth or control its timing over a few days

This article is timely and pairs well with Sequoia's $600B question: https://www.sequoiacap.com/article/ais-600b-question/ calculated simply from NVidia run rate revenue, which is the cost that genAI companies are paying. Where's the profit?

Meta's open source LLM stance makes things more spicy, making it challenging for anyone generate differentiated and lasting profit in the LLM space.

At the current pace, the LLM bubble is poised to pop in a year or two - negative net revenue can't keep growing forever - barring a transformative, next-generation capability from closed-source AI companies that Meta can't replicate. All eyes on GPT-5.

The cover of Bayesian Data Analysis 3 shows that empirically, birthdays are not uniformly distributed. The fall has 10-20% more births than other months, and holidays are significantly underrepresented.

https://lh3.googleusercontent.com/proxy/OZtu7ACWp4X283a4e5Pg...

This essay by Neal Stephenson was first published in 1999. https://en.m.wikipedia.org/wiki/In_the_Beginning..._Was_the_...

The analogy of OS as cars (Windows is a station wagon, Linux is a tank) is brought up in the recent Acquired episode on Microsoft, where Vista was a Dodge Viper but Windows 7 was a Toyota Camry, which is what users actually wanted.

Ah right, thanks

Essentially all AI work I've seen in games aims for game theory optimal play, but I think it could be really interesting to consider AI for exploitative play. Does this exist? Poker with imperfect information, human pressure and fallibility means that players will inevitably stray from Nash equilibrium. The decision on how to exploit without getting exploited back oneself seems really fascinating to consider from an AI perspective. At a glance it seems to require considering how others view you..

In SF, waymo isn't particularly slow, and drives remarkably humanlike to me, unlike Cruise which regularly took 2x as long to drive anywhere as a human would, and drove very much not like a human would.