_aavaa_

It’s an adaptation.

The difference between subscription rates and API rates?

Infinite $ since you’re trying to steal their proprietary information, in their view.

Also API usage also has ToS.

What is there to talk about the KV Cache, they’re handing off to a different model, I thought that you can’t reuse KV cache between entirely different models?

I mean the choice is: 1) we pass laws that explicitly say training models like this is legal (the original quote, 2) say it’s illegal and requires licenses for the data and ability to opt out, 3) we ignore it and continue because the companies are too big to jail.

Tesla had to build the charging infrastructure (and other fragmented companies followed), they had to educate customers, they had to fight dealership requirements, they had to build up and secure the supply chain, etc. etc. The government then comes in and gives them subsidies after they’ve made it though all those filters.

The Chinese government prioritized critical minerals and made sure there was domestic mining and processing. Then they mandated that regional electricity companies install EV chargers. Then they provided consumer incentives to buy an EV (bypass license plate lotteries). Then they didn’t play favorites; so much so that when the domestic manufacturers were crap they allowed Tesla to come in and set up production. That raised the bar on suppliers and spurred actual competition from the local brands.

They build the conditions for actual competition to occur and then are letting the companies win or fail on their own merits, someone will be bictorious and they’ll be lean and mean. A true capitalist free market compared to the sweet protectionist deals the Big Three get.

Would BYD be allowed to build a car in the USA? Even a joint venture? Of course not, Washington is mulling not allowing Chinese cars to even be driven across the boarder for those silly Mexicans and Canadians who want to buy one.

Fair use requires more than you accessing the material legally.

In the US one of the factors is “ the effect of the use upon the potential market for or value of the copyrighted work”.

If anthropic Hoovers up the world’s books and trains on them, and then spits them out verbatim on command, then it will clearly impact the value of the work; nobody will buy the original, they’ll just ask Claude.

Others also argue that even if it’s not reproducing it exactly that the training runs afoul of that factor, specifically the “market for” portion. A rights holder can no longer license their book for training of LLMs if Anthropic goes ahead and just trains on it anyway.

Who cares, a LLM isn’t a person.

No, we’re talking about an intimate set of tensors, not a human being.

A tree falling and killing someone isn’t tried for manslaughter.

So I don’t care about a hypothetical teacher.

There was, and to a large degree Tesla had to do a lot by themselves since the US government picks winners and saves losers rather. The Chinese government prioritized EV and battery productions through onshoring, industrial policy, and blanket incentives (bypassing license plate lottery for EVs), rather than favoring a specific company.

Why does the US have 1 successful EV company while China has a dozen? Because on government cares about it and the other doesn’t. And now that they’re losing a race they couldn’t be bother to compete it they complain.

Same complaining about AI, now that the two chosen champions are facing actual competition, there’s complaints that it’s unfair. we were supposed to win, it’s unfair that they’re beating us at our own crooked game.

Give me a break.

Yeah, in this case it's the USA side that's on the losing end. Just because the US government wants small government and no intervention (expect when it comes to the donor class, or their voting base, or their own financial interests) doesn't make it a universal truth.

We're happy to prop up companies that should have failed after they get big an dominant, but having an industrial policy to invest in a field as a whole is somehow a big problem.

You see this crying and threatening to take their toys and go home on every issue as soon as someone else is in the lead. Just look at TVs and solar panels; China invested in growing that sector since they saw it was important for the future; the USA does their best to deny climate change and demonize anything not running on fossil fuel. And now that nobody wants to buy American's overpriced and uncompetitive cars, it's the fault of other companies for planning ahead. But the same politicians complaining about it are very happy to set up their own protectionist tariffs and eventually bail out the laggards, again; all while touting the "free market"

Doctorow keeps saying it of all the tech companies: every pirate wants to be an admiral.

Dumping is such a loaded term. This is investor-backed scaling to capture market share, standard VC playbook.

I have serious concerns about how these models might reflect Chinese government perspectives (try asking them about Tiananmen Square).

And I have serious concerns about the American ones. Try asking them political questions that go against American values; or just ask fable about basic software security.

distillation: why exactly is it bad? After all, what are large language models but the distillation of all of the knowledge on the open Internet, scraped by the frontier labs and distilled into the models that are themselves being distilled? Who is exactly being wronged here? ... The U.S. should pass a law that (1) makes explicit that collecting data for training models is fair use, and (2) bars terms of service that forbid distillation

Sounds great to me; live by the sword, die by the sword.

Are we talking about the large Covid spike in early 2020? I don’t think that has anything to do with the sale.

I don’t think we need to blame that sale. The same steady decline had been going since 2016. Itms clearly there in the graph, but everyone trumpeting the ChatGPT angle conveniently ignores the preceding 5+ years of continual decline.

If there wasn’t anything interesting in it, we wouldn’t be hearing about Zuck’s continual effort to silence her.

For a video on the subject, see the always good Jacob Geller: https://youtube.com/watch?v=v5DqmTtCPiQ

So was alcohol.

The problem is doing it in a way that doesn’t removal all privacy.

To say that paid off nuclear is profitable without subsidies and then mention France is deeply funny.

Putting aside the actual subsidies, there’s the uncomfortable truth that nuclear operators are not on the hook for long tail risks and costs. The government (read us, the taxpayers) in most countries will foot the bill for decomisioning if the company goes under, and it will foot the bill for accidents.

To talk about them not getting subsidies once they are paid off is to ignore the fact that no reactor would ever get built without the government providing them a guaranteed backstop.

This ignores the pollution to make that fuel and the pollution to build the plan (both of which you are already counting on the battery side). Plus the pollution of all the backup generators that they require and must test regularly.

The same can be said about Apple. Several companies have complained about them taking a meeting with apple, presenting their product, only to have Apple then rip it off and build it in house. To say nothing of sherlocking.

He's just old enough to be part of congress. Give him another decade and he'll be ready to run for president.

Won't be for long given all the id restrictions coming for the internet.

I'm not calling it slop because it's long. I'm calling it slop because it's poorly written and reeks of AI with no editing.

Instead of

Is the loop just more attempts? One objection deserves an answer up front. A verification loop spends extra inference per task, so is the lift just a bigger compute budget in disguise? Partly it has to be. The loop does more work. But the retries the benchmark grants are blind: the model sees a failure signal and guesses again. The loop’s iterations are guided by evidence from the running application, which is a different kind of attempt, not just another one. Whether guided iteration beats an equal budget of blind retries at matched cost is exactly the ablation this framing demands, and it is planned for a future post: DeepSeek alone with a larger retry budget, against DeepSeek with the loop, dollar for dollar. Until that runs, read the results below with this open question in mind.

It could have been

These loops are not just retries. Each iteration provides the model with evidence from the previous run. Some of the uplift may come from the extra tokens, so a follow-up post will compare the guided loop against cost-matched blind retries.

We can argue over exact wording, but the original is far too long.

Or the point about "measuring cost honestly". It's not clear why you wouldn't be using the published rates and do the basic multiplication yourself. There's nothing subtle about this, and it doesn't need to a whole paragraph.

Or the people writing this could spend more effort to make it not slop. If they can’t be bothered, I won’t waste my time figuring out if this is worth it, there’s are 1000 other articles to read and techniques to try.

And if this is worth a look, I’m sure I’ll hear about it again from someone who wrote it better.

Well I don’t know what the method is, there’s a mention of ironbee, but I’m not willing to spend the time digging through this to figure out what’s needed and how to set it up.

Nobody is forcing you to play a game with kernel anti-cheat.

I want to play the game, not deal with mass cheating. Unless the people against kernel-level anti cheat can provide an alternative with similar level of protection, there isn’t anything to discuss as far as I’m concerned.

Seems interesting, but buried under a mountain of written slop.

So I would rather share a match with the occasional cheater than run un-auditable ring-0 software on the same machine I use for anything private.

Yeah except that’s not the options here. Even with ring-0 there are lots of cheaters. Without it the game would be completely infested with them.