yogthos

where it gets really fun is when you add a temporal dimension to the noise :) https://yogthos.net/posts/2026-06-17-perlin-flow.html

it's a big club, and you ain't in it

Chinese version was the one I found.

https://www.kimi.com/user/agreement/zh/userPrivacy

Anybody who's tried it knows that what they're saying is true though.

oh that's very cool, exactly the kind of stuff we need

That's my thinking as well. The whole distillation thing is a distraction from the actual innovation happening in this space. What will be interesting to see going forward is what types of new techniques people manage to come up with to over come the current architecture limits.

Indeed we will, my prediction is that we'll see a model from China that definitively surpasses any US model by the end of the year. China has an absolute population advantage here along with having a much better education system. And China now dominates in published AI papers.

The US enjoyed an early advantage due to excessive money being poured into AI which led to the current bubble, and access to the hardware that was needed to train these models initially.

At this point, neither of these factors actually matter that much. The naive approach of simply making models bigger has hit a wall, and now you need ingenuity in figuring out better architecture for them. Precisely because Chinese companies have had to deal with more limited resources, they put a lot more effort into researching different kinds of optimizing techniques. And of course, China is also catching up in chip making, and Huawei clusters are already competitive with Nvidia for training. So, that gap is closing as well.

The big difference is that an absolutely insane amount of money has been spent in the US, while China managed to do this on a fraction of the budget. The AI Investment Surge graph here puts things in perspective. https://hai.stanford.edu/news/inside-the-ai-index-12-takeawa...

I'm saying it's a cope to claim that the only reason Chinese models are catching up is due to distillation, while pointing out that distillation itself is in no way unique to Chinese companies. I'm sorry this was too complex of an idea for you to follow.

My prediction is that they're going to angle to become a vendor of record for the government and get bailed out. That's the only path at this point because there won't be any competition from China in this niche.

The process takes time because even when you're distilling answers, you still need to actually do reinforcement training on the model. And given that Fable and GPT 5.6 just came out there simply hasn't been much time to do that. On top of that, Kimi also does better than Fable or GPT on a lot of tasks, distillation alone can't explain that, meaning there is a difference in architecture. You can watch a talk from Kimi founder to see how Kimi was actually trained and why it performs well. https://www.youtube.com/watch?v=5CkCW1P-g88

Not to mention that US companies models constantly distill each other as Musk was forced to admit under oath. This whole narrative has just been a massive cope.

This is precisely why we need more projects like this https://github.com/bigscience-workshop/petals

There's no fundamental reason why models couldn't be developed and trained using community efforts. It might not be as fast and efficient, but it's definitely possible.

And given that Chinese models are closing the gap there are basically two thing that could be happening. One is that they are moving faster than US companies developing closed models, and two that we're starting to hit a plateau for model capabilities where all the easy gains have been plucked, and now it's not really possible to move forward at the same rate on the frontier. Of course, both things could be happening at the same time.

Sorry, misread your comment. Again though, it seems pretty obvious that our ability to accumulate knowledge across generations by recording it through the use of language is a categorical advantage over other animals.

The fact that we are able to accumulate knowledge and apply it to shape our environment in a way that no other species is able to kind of does suggest that we have a profound communication advantage.

Sure, lifespan is a factor, but it's hard to argue that lifespan has been the main factor for humans. Or that lifetime of many other organisms isn't sufficient. And I already pointed out that we co-evolved with language, that's what it means for it to be a dialectical process. However, for language even get bootstrapped, it needs the prerequisite brain structures to support recursive grammars and abstractions we use.

What we can look at is what other species are able to accomplish and how much they are able to shape their environment. I'm not arguing the fact that we don't have a good grasp on how other animal communicate and different kinds of intelligence we have in the animal kingdom. What I'm saying is that our unique combination of being able to abstract concepts and package them up using language, and especially writing, is what allowed up to rapidly accumulate knowledge and pass it across generations. We are demonstrably the only species that does this. Although, there were other hominids originally with similar capabilities that went extinct.

I expect what's most likely to happen is that either Anthropic or OpenAI end up becoming a vendor of record for the government and get a bailout. And then their whole business model will be serving use niches which would be considered too sensitive for Chinese models. That's the only plausible business model I can see here.

Yup, and just look at how difficult it is for us to recognize different kinds of intelligence on our own planet too. We've just started figuring out that animals like crows have complex cognition.

They used two of my books and I'm still waiting for my cheque here.

We kind of do though since we don't see crows accumulating knowledge at any scale comparable to humans and using this knowledge to shape their environment in increasingly complex ways. And crows very obviously do make tools and even teach each other to do it, so it's not like they don't have the inclination for it.

The language itself is a a human invention, and a product of how our brains are wired. However, there's a dialectical process here where the language shapes us in turn, and both our minds and our language evolve together. The reason we have more things to say stems from us accumulating knowledge and expanding out horizons through the use of our language.

Sure, intelligence is a gradient, it's not something exclusive to humans. And different biological systems need to solve problems and create models of their environment in order to respond to it intentionally. However, that's tangential to the point I was making, which was that we are able to rapidly accumulate knowledge across generations giving us mastery of our environment that's qualitatively different from any other organism on the planet. And the next logical step here is machine intelligence where human style intellect could be implemented on a non-biological substrate which would open up completely new niches for postbiological life to inhabit.

The difference is that we are able to accumulate information across generations to grow our collective knowledge. Other animals are not able to do that at scale. So, while you are correct that other animal communicate and even teach each other, it's a qualitatively different situation from human communication.

The assumption that biological life will be doing galactic colonization seems myopic in the extreme. Let's just consider the progression here. Life on Earth appears around 4.5 billion years ago. Humans start evolving around 2.8 million years ago. Use of language appears around 100,000 years ago. Writing is invented around 5500 years ago.

Inventions of language and writing are the landmark moment here. Before language was invented the only way information could be passed down from ancestors to offspring was via mutations in our DNA. If an individual learned some new idea it would be lost with them when they died. Language allowed humans to communicate ideas to future generations and start accumulating knowledge beyond what a single individual could hold in their head. Writing made this process even more efficient.

So, after millions of years of life on Earth no technological development happened. Then when language was invented humans started creating technology, and in a blink of an eye on cosmological scale we went from living in caves to visiting space in our rocket ships. It’s worth taking a moment to really appreciate just how fast our technology evolved once we were able to start accumulating knowledge using language and writing.

Now let’s take a look at how technology itself has been evolving. Once we discovered radio communication we went through a noisy period where we were leaking a lot of our broadcasts into space, and within a span of a 100 years we started using more efficient communication, and encryption. If somebody intercepted our broadcasts today they would look like noise because they’re designed to look like noise. Our society today is utterly and completely unrecognizable to somebody from even a 100 years ago. If we don’t go extinct, I imagine that in another thousand years future humans will be completely alien to us as well.

So the period during which intelligent life would be recognizable to us during its course of evolution is infinitesimally small. The time between creating language and becoming an advanced technological society is measured in thousands of years, while evolution of life is measured in millions of years. The chance of two different intelligences finding each other at exact same stage of development where they might be able to communicate is incredibly unlikely.

Based on that, I would imagine that the biological phase for intelligent life is rather short. We’re likely to develop human style AIs within a century, and they will be the ones to go out and explore the universe. Meat did not evolve to live in space, we’re adapted to gravity wells. An artificial life form could be engineered to thrive in space without ever needing to visit planets. This is the kind of life that’s most likely to be prolific in space. Furthermore, post biological intelligences would likely be running at much faster speeds than our mental processes operate on. What we consider real-time would be might we consider to be geological scales. Such beings might consider what we view as real time akin to the way we look at continental drift. We’re aware that it’s happening, but it’s of little interest to use on day to day basis. It’s quite possible that advanced civilizations become solipsistic and care little for the outside universe.

For all we know the Universe may be teeming with intelligent life and we just don’t recognize it as such. We might be like an ant hill next to a highway looking to see if there are other ant hills around.

When the model is open, then anybody can download and trying doing things with it. Distill it, change its architecture, inspect how its layers work, and so on. A lot of research ends up being published as a result which the original authors of the model can integrate back to improve it. This is precisely why China now tops AI publications https://www.science.org/content/article/china-tops-world-art...

The models themselves aren't what's valuable. The goal is to get the models to become the global standard which everyone uses, and people are familiar with. This further ensures that standards and hardware will be developed around these models going forward. It ensure these models are what most people are familiar with using, and so on. This becomes common infrastructure like Linux that's effectively impossible for commercial offerings to compete with.

The money is going to come from providing custm integrations, customization, robotics applications, and operating cloud services like AWS. That's where Chinese companies are aiming. There's a good interview with Alibaba Cloud founder where he explains the strategy. https://www.youtube.com/watch?v=JPzevOpIzPg

While American companies are betting on the idea that if one model can pull away it's going to keep self improving and nobody will catch up, Chinese companies are betting there will be a plateau to this tech, and it's more important to focus on market dominance because they will catch up later.

And it's becoming clear that Chinese companies made the right bet because Chinese models are closing the gap now, which means there is no singularity effect being observed with frontier models. We are starting to get to the point of diminishing returns already where companies have to put ever more effort into squeezing just a bit more capability. The easy gains appear to be over.

I'm also working on Jolt which uses Chez Scheme as the compiler. https://jolt-lang.github.io

I've already got enough of JVM compatibility to run Ring apps, and have some fun libraries like a Reagent style library on top of GTK https://yogthos.net/posts/2026-07-02-jolt.html

Has anybody been held liable? https://www.amnesty.org/en/latest/news/2026/06/usa-four-mont...

Also, we shouldn't underestimate the power of developing things in the open. Chinese open models benefit from the wisdom of an entire global research community while American engineers working on proprietary closed models are working in their own insular silos. It should be no surprise that the scientific community at large would pull ahead of these small teams. On top of that, doing research in the open amortizes the cost. Incidentally, this is exactly the same logic that led open source to dominate in recent years.

it sure is bootlicker

I find Orwell appeals to the pseudo-intellectual crowd because he says things that don't really have any profound insight, but seem deep to superficial people who want to feel smart. When you look at it critically, his writing has a really strong 'I'm 13 and this is deep' energy to it.

https://huggingface.co/Soofi-Project/Soofi-S-Base

https://huggingface.co/Soofi-Project/Soofi-S-Instruct-Previe...