ankit219

llms are good at greedy depth first search[1]. so prompt / human needs to tell the model the breadth parts to take.

and human writing the prompt here did a lot more than that. asking to include parallel-edge 2-cycles, admitting disconnected graphs, specifying the emptyset cover, multiset counting, it all likely came from previous fake proofs.

[1]: see: ankitmaloo.com/fable - its the way most llms are trained, and is also natural owing to autoregressive nature.

they are also working on activations (w4a8, w4a16 from what i know). if they deliver (and a big if), it means that given their market reach, they can dictate the way sub 100b parameter models are trained and served to a large extent, given their major usecase would be on device (macos and not ios for most of them).

at a gross margin level, mostly no. if you include the cost of training a model as full R&D then possibly yes.

Batch size is what you should look at. If a cluster is running and processing one request, filling the batch has almost no marginal cost (kv cache creation/storage/fetch costs aside). But if the concurrent requests exceed batch size, one extra request would cost basically the rent cost of entire new cluster. APIs have the bursty nature so companies would plan to price it such that they are profitable / break even at 40%-50% utilization (% of filled batch for simplicity). so any extra request would not have the same cost as long as they are alongside an api request. you might think it degrades teh performance. easy: just assign a priority tier to api requests, and a lower tier to subscription requests.

its even more effective and powerful now that you have continuous batching. so likely if the api is being used, they are not eating any loss, let alone "big loss"

An interesting question to wonder is why these optimizations were pushed so aggressively in the first place. Especially given this is the time they were running a 2x promotion, by themselves, without presumably seeing any slowdown in demand.

this is good.

problem is google's security concerns. when people connect gmail to openclaw, google flags the activity as weird and suspend the account because of unusual activity. Many whose accounts got locked because of this and they thought it was because they connected it to antigravity use against the policy (which happened in some cases). We will still see google account suspensions, and that would keep making news. and it wont be because of antigravity usage.

not much to do with self improvement as such. openai has increased its pace, others are pretty much consistent. Google last year had three versions of gemini-2.5-pro each within a month of each other. Anthropic released claude 3 in march 24, sonnet 3.5 in june 24, 3.5 new in oct 24, and then 3.7 in feb 25, where they went to 4 series in May 25. then followed by opus 4.1 in august, sonnet 4.5 in oct, opus 4.5 in nov, 4.6 in feb, sonnet 4.6 in feb itself. Yes, they released both within weeks of each other, but originally they only released it together. This staggered release is what creates the impression of fast releases. its as much a function of training as a function of available compute, and they have ramped up in that regard.

Batching multiple users up thus increases overall throughput at the cost of making users wait for the batch to be full.

writer has not heard of continuous batching. this is no longer an issue. this is what makes claude code that affordable. https://huggingface.co/blog/continuous_batching

People are misunderstanding Anthropic's fast mode because they chose to name it that way. The hints all point to a specific thing they did. The setup is costlier, its also smarter and better on tougher problems which is unheard of in terms of speed. This paper[1] fits perfectly:

The setup is parallel distill and refine. You start with parallel trajectories instead of one, then distill from them, and refine that to get to an answer. Instead of taking all trajectories to completion, they distill it quickly and refine so it gives outputs fast and yet smarter.

- paper came out in nov 2025

- three months is a good research to production pipeline

- one of the authors is at anthropic

- this approach will definitely burn more tokens than a usual simple run.

- > Anthropic explicitly warns that time to first token might still be slow (or even slower)

To what people are saying, speculative decoding wont be smarter or make any difference. Batching could be faster, but then not as costly.

Gemini Deepthink and gpt-5.2-pro use the same underlying parallel test time compute but they take each trajectory to completion before distilling and refining for the user.

[1] https://arxiv.org/abs/2510.01123

Agreed. Gemini 3 Pro for me has always felt like it has had a pretraining alpha if you will. And many data points continue to support that. Even as flash, which was post trained with different techniques than pro is good or equivalent at tasks which require post training, occasionally even beating pro. (eg: in apex bench from mercor, which is basically a tool calling test - simplifying - flash beats pro). The score on arc agi2 is another datapoint in the same direction. Deepthink is sort of parallel test time compute with some level of distilling and refinement from certain trajectories (guessing based on my usage and understanding) same as gpt-5.2-pro and can extract more because of pretraining datasets.

(i am sort of basing this on papers like limits of rlvr, and pass@k and pass@1 differences in rl posttraining of models, and this score just shows how "skilled" the base model was or how strong the priors were. i apologize if this is not super clear, happy to expand on what i am thinking)

(author here) great paper to cite.

What i think you are referring to is hidden state as in internal representations. I refer to hidden state in game theoretic terms like a private information only one party has. I think we both agree alphazero has hidden states in first sense.

Concepts like king safety are objectively useful for winning at chess so alphazero developed it too, no wonder about that. Great example of convergence. However, alphazero did not need to know what i am thinking or how i play to beat me. In poker, you must model a player's private cards and beliefs.

Bounded domains require scaling reasoning/compute. Two separate scenarios - one where you have hidden information, other where you have high number of combinations. Reasoning works in second case because it narrows the search space. Eg: a doctor trying to diagnose a patient is just looking at number of possibilities. If not today, when we scale it up, a model will be able to arrive at the right answer. Same goes with Math, the variance or branching for any given problem is very high. But LLMs are good at it. and getting better. A negotiation is not a high variance thing, and low number of combinations, but llms would be repeated bad at it.

(Author here)

I address that in part right there itself. Programming has parts like chess (ie bounded) which is what people assume to be actual work. Understanding future requiremnts / stakeholder incentives is part of the work which LLMs dont do well.

many domains are chess-like in their technical core but become poker-like in their operational context.

This applies to programming too.

And they would have won the AI race not by building the best model, but by being the only company that could ship an AI you’d actually trust with root access to your computer.

and the very next line (because i want to emphasize it

That trust—built over decades—was their moat.

This just ignores the history of os development at apple. The entire trajectory is moving towards permissions and sandboxing even if it annoys users to no end. To give access to an llm (any llm, not just a trusted one acc to author) the root access when its susceptible to hallucinations, jailbreak etc. goes against everything Apple has worked for.

And even then the reasoning is circular. "So you build all your trust, now go ahead and destroy it on this thing which works, feels good to me, but could occasionally fuck up in a massive way".

Not defending Apple, but this article is so far detached from reality that its hard to overstate.

you are comparing post hoc narratives in the training data to real time learning from causal dynamics. The objectives are different. They may look the same in scenarios where its heavily and accurately documented, but most narratives suffer from survivorship bias and reasoning post facto, eulogising the given outcomes.

think this particular complaint is about claude ai - the website - and not claude code. I see your point though.

Its a combination. All caps is used in prompts for extra insistence, and has been common in cases of prompt hijacking. OP was doing it in combination with attempting to direct claude a certain way, multiple times, which might have looked similar to attempting to bypass teh system prompt.

from what i know, it used to be that if you want to assertively instruct, you used all caps. I don't know if it succeeds today. I still see prompts where certain words are capitalized to ensure model pays attention. What i mean was not just capitalization, but a combination of both capitalization and changing the behavior of the model for trying to get it to do something.

if you were to design a system to prevent prompt injections and one of surefire ways is to repeatedly give instructions in caps, you would have systems dealing with it. And with instructions to change behavior, it cascades.

My rudimentary guess is this. When you write in all caps, it triggers sort of a alert at Anthropic, especially as an attempt to hijack system prompt. When one claude was writing to other, it resorted to all caps, which triggered the alert, and then the context was instructing the model to do something (which likely would be similar to a prompt injection attack) and that triggered the ban. not just caps part, but that in combination of trying to change the system characteristics of claude. OP does not know much better because it seems he wasn't closely watching what claude was writing to other file.

if this is true, the learning is opus 4.5 can hijack system prompts of other models.

Like it or not, it's a fundraising strategy. They have followed it mutliple times (eg: vague posts about how much their inhouse model is writing code, online RL, and lines of code etc. earlier) and it was less vague before. They released a model and did not give us the exact benchmarks or even tell us the base model for the same. This is not to imply there is no substance behind it, but they are not as public about their findings as one would like them to be. Not a criticism, just an observation.

While the threat is unreasonable, why does Italy wants a site banned globally? Why is it even considered a debate?

Not the same.

they have usage limits on subscription. I dont know about rate limits. Certainly not per request.

Why would scheduling be a thing in this case? I might be missing something here.

With continuous batching, you don't wait for entire previous batch to finish. The request goes in as one finishes. Hence the wait time is negligible.

I do not understand the stubbornness with wanting to use the auth part. On local, just call the claude code from your harness, or better there is a claude agent sdk, both of which have clear auth and are permitted acc to anthropic. But to say that they want to use this auth as a substitution for API is a different issue altogether.

The issue is that claude code is cheap because it uses API's unused capacity. These kind of circumventions hurt them both ways, one they dont know how to estimate api demand, and two, the nature of other harnesses is more bursty (eg: parallel calls) compared to claude code, so it screws over other legit users. Claude code very rarely makes parallel calls for context commands etc. but these ones do.

re the whole unused capacity is the nature of inference on GPUs. In any cluster, you can batch inputs (ie takes same time for say 1 query or 100 as they can be parallelized) and now continuous batching[1] exists. With API and bursty nature of requests, clusters would be at 40%-50% of peak API capacity. Makes sense to divert them to subscriptions. Reduces api costs in future, and gives anthropic a way to monetize unused capacity. But if everyone does it, then there is no unused capacity to manage and everyone loses.

[1] https://huggingface.co/blog/continuous_batching

Its a strange phenomenon. You want to call out the bs but then you are just giving them engagement and boost. You want to stay away but there is a sort of confluence where these guys tend to ride on each others' post and boosts those posts anyway. If you ask questions, very rarely they answer, and if they do, it takes one question to unearth that it was the prompt or the skill. Eg: huggingface people post about claude finetuning models. how? when they gave everything in a skill file, and claude knew what scripts to write. Tinker is trying the same strategy. (yes, its impressive that claude could finetune, but not as impressive as the original claim that made me pay attention to the post)

It does not matter if they get the details wrong, its just that it needs to be vague enough, and exciting enough. Infact vagueness and not sharing the code part signals they are doing something important or they are 'in the know' which they cannot share. The incentives are totally inverted.

https://ankitmaloo.com

The most basic version, but helps me publish blog posts quickly.

I was not aware of this. Thank you for sharing. This is not common phrasing in either India or US.

They can, but then cost per subscription would not be that low.

From what i have read on twitter. People were purchasing max subs and using it as a substitute for API keys for their startups. Typical scrappy startup story but this has the same bursty nature as API in temrs of concurrency and parallel requests. They used the Opencode implementation. This is probably one of the triggers because it screws up everything.

Telemetry is a reason. And its also the mentioned reason. Marketing is a plausible thing and likely part of the reason too, but lock-in etc. would have meant this would have come way sooner than now. They would not even be offering an API in that case if they really want to lock people in. That is not consistent with other actions.

At the same time, the balance is delicate. if you get too many subs users and not enough API users, then suddenly the setup is not profitable anymore. Because there is less underused capacity available to direct subs users to. This probably explains a part of their stance too, and why they havent done it till now. Openai never allowed it, and now when they do, they will make more changes to the auth setup which claude did not. (This episode tells you how duct taped whole system was at ant. They used the auth key to generate a claude code token, and just used that to hit the API servers).

Because your subscription depends on the very API business.

Anthropic's cogs is rent of buying x amount of h100s. cost of a marginal query for them is almost zero until the batch fills up and they need a new cluster. So, API clusters are usually built for peak load with low utilization (filled batch) at any given time. Given AI's peak demand is extremely spiky they end up with low utilization numbers for API support.

Your subscription is supposed to use that free capacity. Hence, the token costs are not that high, hence you could buy that. But it needs careful management that you dont overload the system. There is a claude code telemetry which identifies the request as lower priority than API (and probably decide on queueing + caching too). If your harness makes 10 parallel calls everytime you query, and not manage context as well as claude code, its overwhelming the system, degrading the performance for others too. And if everyone just wants to use subscription and you have no api takers, the price of subscription is not sustainable anyway. In a way you are relying on others' generosity for the cheap usage you get.

Its reasonable for a company to unilaterally decide how they monetize their extra capacity, and its not unjustified to care. You are not purchasing the promise of X tokens with a subscription purchase for that you need api.