HN user
Studying a DPhil in Robotics World Models for Nuclear Fusion
World models, reinforcement learning, evolutionary methods and fast ML code
It's telling that in their release post, Moonshot themselves said that K3 lags Fable and GPT-5.6 in "user experience". I took that to mean the stuff you can't push directly via RL, what some people call "big model smell".
"Despite being a highly competitive model overall, K3 nonetheless exhibits a noticeable gap in user experience compared with Claude Fable 5 and GPT 5.6 Sol."
I appreciate people thinking this is a marketing ploy, but at the same time, OpenAI have just had to delay a model release because of government attention on cybersecurity risk. This incident will increase the attention on them specifically.
Even if it is a marketing ploy, I could see this stuff backfiring catastrophically - after all they have just illegally hacked a 3rd party via a model they can't control properly. Any serious person in government (US or otherwise) will look at this and say "these guys have no idea what they're doing"
HuggingFace posted an incident report a week ago, which makes it much more likely that this happened. I understand people are suspicious of OpenAI, but I don't think there's any reason to believe this is a made-up event.
I've been sharing music on SoundCloud for a decade and I am genuinely interested in this topic, but I cannot stand how strongly this post reeks of LLM writing. Do people releasing this kind of content not notice all the clichés?
There is a field of hierarchical RL in which the optimisation occurs over a range of time scales/abstraction. But I'm not aware of much practical success for these approaches so far.
Do you have a good source on this information theory framing? I don't remember it being covered in Sutton & Barto.
We're still far from solving real-world physical interaction, even with the world knowledge of LLMs incorporated into VLAs. It's the reason robotics startups trying to deploy into people's homes are so reliant on teleop; the hardware is sufficient (we know that because the robots can solve tasks when piloted by a human), but the intelligence is still insufficient.
I don't think JEPA is necessarily the solution (although I'm bullish on world models), but I don't understand why people feel so infuriated about LeCun. The reason he is famous today is because he spent years taking a contrarian stance, working on neural networks when they were seen as dead and buried. He was eventually proven right. Nowadays he holds strong views which contradict the LLM zeitgeist --- what's shocking about that?
One of the lead authors, Daniel Kokotajlo, worked at OpenAI for years before quitting. In 2021 he wrote a remarkably accurate forecast of how LLMs would develop over the following 5 years[0].
I think it should be obvious that he understands that LLMs are trained via next-token prediction.
[0] https://www.lesswrong.com/posts/6Xgy6CAf2jqHhynHL/what-2026-...
Yann is a big SSL guy but I don't think he was involved in the original DINO - he's not listed as a co-author or anything.
The most basic machine learning-related query gets flagged for me. For example:
In flax nnx, what's the idiomatic way to store state on a Module. For example, if I'm handling the carry manually for an nnx.RNN.
How do I restore one of the orbax checkpoints into NNX from this script?
It's a shame - I like Fable for writing tasks over ChatGPT and I do believe Anthropic is a more ethical outfit than OpenAI. But with the safeguards (and Fable access expiring in a few days) there's no reason to pay for draconian guardrails and harsh rate limits.
There have been papers about model collapse, but the underlying assumption is that you constantly train on only the outputs of the previous model. Later research has shown that as long as you retain some "real" data, training on largely synthetic data is ok.
And in the case the previous poster describes, the other model doesn't generate datasets, it generates environments which the next generation interact with to learn from.
I think you'll be waiting a while for the former, unless you're ok with strangers teleoperating a robot around your house whenever it gets confused.
Wouldn't modern SLAM or VSLAM address that problem?
Thank you for expanding! I come from more of an ML background so still learning a lot on these topics:
Agreed that the neocortex uses fewer layers because of looping - I also suspect it's partly because neurons are more complex than the neurons used in ANNs, so in principle they should be capable of more sophisticated computation in a single forward pass (especially considering that handling multiple neurotransmitters could mean superposed functions).
The point about there being one correct way to model something does seem backed up by the Platonic Representation hypothesis (https://arxiv.org/pdf/2405.07987). I've even seem some work that shows you can find a bijective map between the latent spaces of different transformers (https://arxiv.org/pdf/2505.12540).
Your point about Finsler spaces is fascinating - I hadn't come across the term before. It'd be interesting to see if there's work that specifically allows latents to exhibit that kind of directional metric behaviour, and whether that improves generalisation or something.
On the entire brain being dedicated to prediction, that's more of a high-level comment. I was thinking of work like Andy Clark on predictive processing, which suggests that even regions of the brain which we think of as receptive (eg. the visual cortex) may be implementing a generative/predictive model which is corrected by sensory input.
I'm afraid the precise connection you're making isn't totally obvious to me.
As far as prediction - I mean sure the cortex and LLMs do prediction, but then so can RNNs or diffusion models or any other generative model. Really any ML architecture is learning to compress its environment in pursuit of modelling. More broadly, the predictive brain model would suggest that all of the brain, not just the neocortex, is dedicated to prediction. What would you say makes LLMs similar to the neocortex, rather than the basal ganglia or Broca's area?
Similarly, if you agree with the Manifold Hypothesis, then all machine learning models operate on manifolds. I agree it's an exciting thought, but then I don't know what would distinguish an LLM from a VAE or SVM in terms of operating over a low-dimensional manifold embedded in high dimensional spaces - maybe just scale?
Could you provide more detail? My understanding is that the neocortex is predominantly focused on forwards simulation, which seems distinct to how transformers operate.
As an ML researcher, I know box2d because it underpins many of the standard reinforcement learning environments (in OpenAI Gym) that we use to benchmark methods, like Lunar Lander or Car Racing: https://gymnasium.farama.org/environments/box2d/car_racing/
Thanks to Erin for such a useful piece of software!
This method is cool and the post explains it well. It would, however, be good to get more detail on the energy efficiency they flag as their motivation: is this model actually more energy efficient than the comparators they highlight?
They've said that Mojo is still on track to be open-sourced this year, post-acquisition.
Indeed. The world models research many labs are now chasing was to some degree ignited by David Ha and Schmidhuber's 2018 paper.
More broadly, Sakana is pursing a refreshingly distinct research path, with their focus on evolutionary methods, biological intelligence (e.g. continuous thought machines) and open publication.
Basically all the images and videos on their website are AI-generated. It doesn't inspire much confidence.
Tri Dao's lab must have saved countless watts with FlashAttention. Great to see them continuing to open-source massive efficiency gains.
Here's one that was flagged for me: a question about a niche Reinforcement Learning paper from 2012
I've been reading the option-option model paper by David Silver. It appears that they achieved quite an effective result. Why hasn't there been more work on it since?
I agree it's an oversimplification. The example I think of is something like Newton's law of gravitation vs Ptolemaic epicycles: one simple explanation replaced many layers of tweaks.
It's also a relevant example for AI - one paper tested the ability of Transformers to model planetary orbits: unlike Newton's Law, the implicit forces they learn are nonsense.
Anthropic's claim was that Deepseek collected ~150k conversations.
https://www.anthropic.com/news/detecting-and-preventing-dist...
I think the extent of distillation by Deepseek specifically is overstated. For comparison, Minimax collected over 13m 'exchanges', which starts to sound a lot more like large-scale distillation.
In some sense, science is the most extreme form of compression - Newtonian mechanics explains an incredible number of phenomena in a few lines of text.
AISI did also say that GPT-5.5, which has been public for months, scores basically the same as Mythos on their cybersec evaluation. But there wasn't as much media about about that for some reason.
https://www.aisi.gov.uk/blog/our-evaluation-of-openais-gpt-5...
Token prices have increased, but it's not really the whole story at this point, given some models will use far more tokens to complete a task than others. One of the charts in Anthropic's blog posts shows Fable at 'low' reasoning achieving better results for less money than Opus on 'high'.
They say it's because the EU's DMA would require them to open up device data to third-party assistants, and they'd no longer be able to guarantee user privacy.
https://www.apple.com/newsroom/2026/06/due-to-dma-siri-ai-de...
As WIRED reported[0], despite constantly writing about how an AI collapse is just about to come, Zitron privately does PR for AI firms on the side. The man is an obvious hack, and it's disappointing that he has become one of the mainstream faces of AI skepticism.