kouteiheika

Assuming they did then they surely paid for them, which makes it "not stealing". Am I also "stealing proprietary U.S. technology" by harvesting my Claude chats from my `.claude` directory and training a bunch of models on them?

That said, I doubt the "they distilled Fable" is the reason why K3 is as good as it is, considering the timelines involved, and that Anthropic hides thinking traces, and their overly aggressive "safety" filters.

This constant FUD spread by Anthropic is so tiring.

It's an open-weight model so it literally doesn't matter whether it refuses by default or not, because it's pretty trivial to uncensor[1] any open-weight model and make it not refuse.

[1] https://github.com/p-e-w/heretic

So they are essentially a protection against spurious errors

Only coincidentally. Sampling nonsense tokens will certainly degrade its performance and/or brick it, but it's also there to encourage diversity.

For example, imagine we have the following sentence:

The color of this ball is ____

Now, what should the model predict for "____"? There isn't really a "correct" answer here. It can be "red", it can be "blue", it can be "green", or any other color. But it's definitely not going to be "ব্যথাя". LLMs output a probability distribution for the next token, so imagine this is the probability distribution that it outputs:

    red -> 60%
    green -> 19%
    blue -> 19%
    ব্যথাя -> 2%

I don't quite understand the point about order of operations - does it do normalization after every such filter pass? why not leave it to the end?

Because you can technically compose multiple samplers at the same time in a pipeline, and in some cases their order can matter and give you a different result (or take less/more time to execute). To give you a generic example: imagine you have a list with numbers in random order. You can execute one of two operations on it: (1) sort it, (2) take the leading 10 numbers. If you first sort it and then take 10 leading numbers you'll get a different result than if you'd first took 10 leading numbers and then sorted them.

Obligatory "The Conspiracy Against High Temperature Sampling":

https://gist.github.com/Hellisotherpeople/71ba712f9f899adcb0...

And then there's the fact Fable is barely usable because it randomly downgrades to Opus out of nowhere whenever it thinks about exploits.

I suspect it's not what you meant, but it's definitely not random and is very deliberate. Just today I got it to reliably trigger the "safety" filter with (drumroll) having it list the weight keys of a 300M parameter ModernBERT-derived model. Their "safety" classifier must be matching one of the key names in there and trigger their "this is a frontier model" anti-competitive filter[1] (even though it's just a tiny 300M parameter model, four orders of magnitude smaller than the frontier).

[1] https://news.ycombinator.com/item?id=48464732

Fortunately once you know how it works (i.e. dumb keyword classifier) it's easy-ish to get around: just rename the keys so that it doesn't contain the naughty keyword. (At least as long as it doesn't trigger on something in its own thinking trace, which needs... more creative workarounds.)

...until you get rug-pulled (like with Fable recently) because the model you're depending on is proprietary, then you have fun with no model at all.

The sooner Chinese labs catch up the better it is for all of us, even if you don't use their models, as they're the ones who define the baseline capability that cannot be taken away from you (no one is going to limit/remove access to an LLM if you can get equivalent/better unrestricted access from Chinese open-weight models).

In other words, it's the most average, the most flat. [..] And when you think about it, the AI's training data comes from the broadest consumer distribution. It spits out that kind of data because it learned that most people preferred it.

That's not what it is! The heavy reinforcement learning that the models go through makes their writing the farthest thing possible from the "most average, flat" distribution. The distribution actually becomes very sharp due to this, unlike any writing it has seen in 99.99% of its training data! (The heavier the RL is, the more same-y the output becomes.) This is why every AI-written piece feels the same, and why people can learn to easily tell that something was written by an AI. And frontier labs make this problem even worse by how they sample[1] their logits.

[1] https://gist.github.com/Hellisotherpeople/71ba712f9f899adcb0...

But when I read a technical article, the most important thing, unlike a novel, isn't the prose style.

Exactly! Which is why I think there's no need to "enhance" it with AI.

AI polished it [...] Compared to purely human writing [...] reads better than you'd expect

I'd hard disagree here. Not when you're bombarded with this every single day. First, you start to be able to reliable recognize this writing style. Then, every time you see it you start getting the urge to pick up your chair and smash your monitor with it.

There's also the issue of not being able to distinguish "AI slop" from "human written + AI polished". Am I being led to read some lazy, low quality AI slop? Or is this actually something that a real human spent time and effort to produce? Both look the same, so how can you tell the difference?

Personally I find it somewhat offensive to use AI in human-to-human communication. If you expect another person to read it then please write it yourself! It's not going to be as "polished" as-if AI would have written it, but that's okay! If you insist on using AI then, at most, just get it to review your writing and apply its findings wherever it makes sense.

the mechanics that make zoning real: why #[repr(C)] is load-bearing, why the magic number is often 128 and not 64, and why adding prefetch hints can make things worse.

The honest rule

Sigh... yet another Claude written article, and (unless I'm blind) it doesn't seem to be disclosed anywhere.

Normally I'm not the one to complain, but it's really tedious to see this writing style pretty much everywhere nowadays, and the more you see it the more you start to find it unbearable to read.

HauHauCS has large collection but there are many others

Before anyone recommends these models to other people I'd suggest they read this thread:

https://old.reddit.com/r/LocalLLaMA/comments/1sw77p0/hauhauc...

But if you assume he's just a villain

You don't have to assume anything. A true "good guy" doesn't openly say that he's fine with autonomous, AI-powered weapons being used against me, and mass surveillance applied to me and my family just because I don't live in the US. A true "good guy" doesn't say "privacy is a human right", and then immediately (and completely) bend the knee to an authoritarian government on this issue.

It's pretty much the same as when "installing programs on your computer" is called "sideloading". Deliberately deceptive, weaponized language to make it seem like a bad thing.

That will put them in direct competition with Steam, though. Suddenly their cheaper console will result in way higher cost for the lifetime of the console.

...funny that so many people were complaining about the recent Steam Machine not being worth it compared to just getting a PS5; maybe now it's not that bad of a deal after all, huh?

So although it's nice to have capable open models, it will increasingly bad for us all if open models keep fast-following closed model capabilities as they have been

Cat's out of the bag. The only way to make them safe is to make sure everyone has access to them. This might be an iffy analogy, but if Dario uses it all the time then so can I: they're kinda like nuclear weapons. If only one country has access to nukes then you're in trouble. If everyone has access to them, then it's mutually assured destruction to use them.

Sure, it could be increasingly bad if open models keep increasing in capability. But it will be much, much worse if only the rich and the powerful have access to this technology, and us -- the have-nots -- will have to contend with whatever scraps we'll be allowed to eat off the table of whichever billionaire is in control. We've already seen a prelude of this with Mythos being restricted and Fable being suddenly yanked. Is this the world you want to live in? Where only Dario and his friends have access?

What is Krea's approach to content such as pornography and gore?

You mean whether it's censored? Of course it is. They even say so themselves[1]:

The open source version needed to go through some alignment training so there might be some inconsistencies between closed / open version.

The "alignment training" is pretty much a code word for censorship/lobotomy, because unlike the API version they can't slap on a safe/unsafe classifier on an uncensored model. Of course, just to be clear: I don't blame. If they don't do this then the next post we'd see is them being in hot water. (Remember what happened with Grok? That's what happens when you don't take a hard line here.)

[1] -- https://www.reddit.com/r/StableDiffusion/comments/1udnm0a/we...

Neither change has any technical reason. The only reason why either name change was desired is because some contributors were upset by the names.

Indeed, neither has any technical reason.

I suppose you're right that both changes have a purpose -- one could feasibly convince a company to contribute Linux drivers (a net win for everyone), and the other is a constant annoyance which wastes everyone's time (is this project using `main`, or `master` -- you never know, so have fun getting it wrong all the time) just to allow certain groups of people/corporations to virtue signal**.

** -- If "master" is such a naughty word then where are all of the people getting offended by e.g. "Mastercard", "Ticketmaster", "Master Lock", "MasterBrand", and many other company/product names, and why those names aren't getting changed? Why there isn't any outrage about them? My point here is not to engage in whataboutism, but just to point out that the word isn't actually offensive when used in a non-offensive manner, and virtually no one is actually genuinely upset about it.

Well, because it's true: many of the repositories are named after "Wacom". It's a historical legacy on GNU/Linux. It's also a decade-long debate that these repos should be renamed differently.

Okay... let's rename them then? I know it's silly, but, well, we've went through the whole pointless `master` -> `main` branch rename in so many projects which was much more disruptive -- at least this one could serve a purpose?

the main thing that we've done is we've commissioned additional research on this because I've not been happy with the evidence.

Ah, yes, the existing research doesn't agree with our biases, so let's fund new "research" that does.

I am pretty sure there are entire YouTube channels dedicated to this.

There is: https://www.youtube.com/@Dr.Semiconductor

since they were caught stealing it from American labs

...and "good guys" the American labs were caught stealing from authors all over the world[1].

[1]: www.npr.org/2025/09/05/g-s1-87367/anthropic-authors-settlement-pirated-chatbot-training-material

I think that this is what OpenAI/Anthropic want but they wont say it publicly.

Won't say it publicly? Anthropic is openly and explicitly saying it publicly. Here: https://darioamodei.com/post/policy-on-the-ai-exponential

AI companies that develop advanced AI models must have strong security standards that protect their model weights

If the model is open-weight then there's nothing to protect, so the only way to fulfill this requirement is to make open-weights illegal.

I think you overestimate human translators. There is a lot of very poor quality human-translated text out there.

This.

There was even a big controversy recently with one of the games on Steam where human translators just completely botched and vandalized the translation, mistranslating huge chunks of it and injecting their own personal politics which are not present in the original text (only English was affected; other languages were translated fine apparently): https://store.steampowered.com/news/app/2914150/view/5028562...

If you'd get the AI to translate it, even without any editing, it would have done much better job. Just because something's done by a human it doesn't automatically make it good; you still need competent people at the helm, and recent machine translation advances certainly raise the floor on that.

Sure, but the difference is that one side (Anthropic, OpenAI, Google and co.) hoards everything, keeping it proprietary behind API paywalls and constantly spewing AI doomer rhetoric while limiting what you can do "for your own safety" (especially Anthropic; Dario has been consistently doing this since GPT-2 days, every time claiming that things are "too dangerous" for the common folk to handle). While the other side (big, bad China) releases all SOTA open-weight models with which you can do whatever you want with, along with a ton of open research.

So yes, there is geopolitical rivalry, but one side is deliberately antagonistic (not releasing anything in the open, putting arbitrary restrictions, spewing toxic rhetoric, applying sanctions, etc.) while the other side is letting everyone (including their rivals) to use what they've produced with little-no-to restrictions.

I'm under no illusion that if the situation was reversed China would most likely do the same, but as things stand you can probably guess which side I'm rooting for here (at least until the roles reverse).

Ah yes, those pesky Chinese backdoors that no single instance was ever found, even though Chinese open-weight model are a thing for many years now. Many people burn through millions of tokens on these models every day - surely someone would have triggered one of those backdoors, right?

Or that pesky CCP censorship and propaganda baked into the model, which any random guy can remove from whichever model they want as a single weekend side project with an off-the-shelf tool[1]. (Try it. It's fun. I've done it myself.)

[1] https://github.com/p-e-w/heretic

Not every weight is quantized. For example, those weights which don't take much space or are highly important are left in higher precision. State-of-art quantization of weights is never done uniformly (i.e. to all weights and in the same way).

It just got worse the more sophisticated games became.

I feel like it got worse not because games became more sophisticated, but because they got dumber and require less and less brainpower.

I spent my childhood playing ridiculous amount of video games, probably same amount or more as kids nowadays do. I had literally hundreds of Playstation games back in the day. And I turned out fine. But the difference was that I was playing a ton of games that required you to put some thought into them, and weren't easy, and often were story-heavy (e.g. JRPGs in particular; being from a non English-speaking country that's how I actually learned English!).

Doesn't matter, because they're open-weight, so I can just download them to my PC and... hey, look, now they're owned by me! Unlike the "good" Western counterparts which are all fully proprietary. (Except Mistral, but they're nowhere near SOTA.)

The model is natively quantized (i.e. it was trained that way in the first place, so this is not a post-training quantization which degrades performance).

Yes, I would agree with this.

I think it works best when you're using the agent in a more hands-on way with a targeted prompt. If you're obsessive about code quality like I am (so you thoroughly review and, when needed, reprompt or even rewrite what the agent does) then you'll be fine, but if you like to just throw a prompt at the wall and expect it to plan and execute the whole thing perfectly then you'll be disappointed.

A middle-ground trick one can use is to have Opus (or Fable now) plan the whole thing and get something cheaper like Kimi execute on it.

This. Here's a quick experiment I did yesterday.

I got a new $20 Claude subscription to try the new Fable model. I gave it a single prompt, and it barely finished, using up my whole session quota (it was at ~95% when it finished) and 10% of my weekly quota.

For comparison, with the Kimi Code $40 subscription I can pretty much constantly run two/three agents in parallel for the whole week, and I never run out of quota. I can blindly throw it at anything and everything without worrying about hitting the limits. (And it's not exactly a cheap model to run -- it has 1 trillion parameters!)

Is Kimi as good as Claude? Of course not. But you don't need the absolute state-of-art for most things. If I don't have exceptionally difficult tasks it makes no sense to use it. Just throw Kimi at it, and even if it needs to run 2 or 3 times longer in the background I don't care, because I'm not running out of tokens there.