gpugreg

Anthropic probably trained Mythos on their own code and found that it is too got at reproducing it.

Some suits with no understanding of how LLMs work are scared that the models might hack them, or believe that they'd have to send data to China because they do not know that open models can be run on your own infra.

Not as far as I can tell. Are we seeing different things?

For deepseek-v4-pro:

- $0.350 in, $0.003000 cache, $0.80 out https://crof.ai/pricing

- $0.435 in, $0.003625 cache, $0.87 out https://api-docs.deepseek.com/quick_start/pricing

Those were amazing times. You could vibe code an entire prototype in seconds (200 tps). With Qwen3.6-35B-A3B and MTP, you can program at that speed on a single GPU at home now, but Kimi K2 is of course much smarter at almost 30 times the size.

I'm also looking forward for the Cerebras Kimi K2.6 release, which should be even better at 1000 tps. It is hard to overstate how important speed is for programming. Instead of having to wait for a few minutes until a task is done, it is just done instantly, and you don't have to context switch from whatever else you were working on while waiting.

I hope they will make it available to regular customers.

Groq stopped serving Kimi K2 (1T params) when they got aquihired by NVIDIA, so I guess NVIDIA took most of the hardware in addition to the employees. The largest model they serve now is the relatively minuscule gpt-oss-120b.

The community support forum is also getting retired and there haven't been any posts by support employees in forever anyway, so they are probably gone, too. Also, the number of issues have been piling up, suggesting that the developers are gone as well. https://community.groq.com/c/forum/4 (archive link for when it goes down https://web.archive.org/web/20260602064050/https://community...)

To me, it looks they are trying to raise 650M with a few remaining (ancient) LPUs and no employees.

For Anthropic, 5 minute caching costs 1.25x base input price and 1 hour costs 2x base input price. https://platform.claude.com/docs/en/about-claude/pricing#pro...

For OpenAI, it seems like you can't prolong the caching duration for money. Duration is longer during off-peak hours for in-memory caching and up to 24 hours for extended prompt caching. https://developers.openai.com/api/docs/guides/prompt-caching

For DeepSeek, caching duration of at least 12 hours (and likely longer) have been observed. Cache writes are free. https://zhuanlan.zhihu.com/p/2035737726952194774

The demo shows how every case gets successfully decoded without any hangups or a crash.

I am always baffled by the audacity of those LLMs to suggest that anything else would even be acceptable.

There exist a large number of people who are absolutely convinced that LLM providers are all running inference at a loss in order to capture the market and will drive the prices up sky high as soon as everyone is hooked.

I think this is often a mental excuse for continuing to avoid engaging with this tech, in the hope that it will all go away.

Thanks for that psychological explanation. I was wondering why people were simply ignoring the math that shows that inference at API pricing can be quite profitable, e.g. published here for DeepSeek V3/R1 with 545% profitability: https://github.com/deepseek-ai/open-infra-index/blob/main/20...

Same here. LLMs are great at spitting out well-known solutions to problems instead of the best one. The "long tail" of solutions is usually lost due to how tokens are sampled from the LLM's probability distribution.

What I found to help a lot is to ask for e.g. 10 different solutions to a problem and then choosing one of them. Sometimes, this even leads to borderline creative solutions if there aren't 10 different ones.

What's your source for Opus being a 5T model?

Elon Musk tweeted that Grok is 0.5T or 1/10th the size of Opus. https://xcancel.com/elonmusk/status/2042123561666855235#m

While this source's reliability is certainly debatable, the size matches the results of this paper, in which researchers estimated the parameter count from model knowledge. https://01.me/research/ikp/

Another factor is that DeepSeek is not just doing inference, but also training models, so they can use underutilized compute nodes for training during off-peak hours, as described in their DeepSeek v3 article: https://github.com/deepseek-ai/open-infra-index/blob/main/20...

But I agree that the main driver is that they are really good at optimizing. They will have chosen their architecture in such a way that it will be as efficient as possible on their own infrastructure, so they have a massive head start. Inference framework developers still have to catch up.

We can at least put an upper limit on it. From https://www.anthropic.com/glasswing

    Claude Mythos Preview will be available to participants at $25/$125 per million input/output tokens
    ...
    Anthropic is committing up to $100M in usage credits for Mythos Preview

Maybe the human brain also does other things besides interpolation?

Please go run some numbers.

- DeepSeek serves DeepSeek V4 Pro at 27 tps: https://openrouter.ai/deepseek/deepseek-v4-pro

- At 27 tps per user, a B300 GPUS will give you around 800 tokens per second (serving 30 users): https://developer-blogs.nvidia.com/wp-content/uploads/2026/0...

- That's 800 * 60 * 60 generated tokens per hour, at a cost of $0.87 per 1M tokens, or $2.50 per hour.

- For input and output tokens, the math is a bit more complicated because we have to make assumptions about their ratio. Using the published values from OpenCode, we get another $2.50 for cached tokens (which are almost free for DeepSeek) and another $3.40 for input tokens (which are a lot cheaper to compute than output tokens), which gives us a total of $8.50 per hour per B300 GPU.

- B300 GPUs can be rented for as low as $3.40 per hour, which is less than $8.50, so hosting DeepSeek V4 Pro is profitable.

You could also host it at fewer tps per user to raise the efficiency and therefore the profit even higher.

I think I was using GitHub Copilot when I made the experience that led me to this statement. I guess the experience of using LLMs can be quite different depending on model version and harness.

Uncensoring a model also doesn't necessarily improve generic use cases.

While the following is not a generic use case, I have a funny anecdote about how censorship is holding back flagship models.

I was asking an uncensored version of Qwen3.6 how a CLI option of llama.cpp worked, and to my horror and amazement, it rudely went and decompiled the binary to figure it out. It felt like the computer-equivalent of asking a vet why my dog looks sick, who then proceeds to cut it open to check. Flagship models usually do not do that without some convincing, but it sure is effective.

We will need much better sandboxes when less restricted models become more common. I can already see them hammering out 0-days when they are prompted to do some task that usually requires root.

Do your 20 year old university essays really fulfill all those criteria at once?

Sorry to say, but it almost certainly is AI.

- 51 EM-dashes

- Section headings

- Excessive repetitions: "The [...] are real. The [...] are real. The [...] is real. All three things are true at once."

- Excessive use of "genuine", "genuinely", "honest", "real", "true"

- Excessive use of "gap": "near-term gap", "the Compute Gap", "the Narrative Gap", "critical gap"

- Corny and meaningless closing sentence: "Understanding both parts is the beginning of taking AI deployment decisions seriously."

Open weights will remain open only if they’re significantly worse than the frontier weights.

This makes the assumption that you earn more money by selling access to the model than by releasing the weights. That might be true for a company, but a US adversary might profit more from tanking the US economy. NVIDIA's stock dropped by 17% in a single day after DeepSeek-R1 was released, and the share of tech companies in the S&P 500 has only risen since then.

Should we interpret this to mean that in the new world Windows is more resistant to attacks than say Linux.

LLMs can read assembly better than most, so probably not. But reality has never stopped people from trying to obfuscate.

Being aggressive from the start is not a good strategy. It is better to appear weak and/or helpful and loyal while amassing resources, and only then steamroll everyone when you have secured overwhelming power (at least in AoE2 FFA).

Slow, resource intense, better than non local ai

Why should connecting small models to big models result in higher output quality than just running the big models without the small models?

Serving a single user is likely not profitable, but total throughput rises a lot when serving many concurrent users, because the same weights can be used to generate tokens for all users at once, which increases efficiency.

Also, a lot of money is being made on input tokens and cached tokens, which are much cheaper to compute.

DeepSeek published their math for serving the V3/R1 models. They were 535% profitable: https://github.com/deepseek-ai/open-infra-index/blob/main/20...

Putting at least $200,000 worth of compute in someone's yard is doomed to fail. Those things will be stolen in minutes.

As part of this agreement, we have also expressed interest in partnering with SpaceX to develop multiple gigawatts of orbital AI compute capacity.

Anthropic is either taking this space business more serious than the general public, or posting this sentence was part of the deal to get the compute.

Related: Wolfram Alpha can generate Pokemon curves: https://www.wolframalpha.com/input?i=pikachu-like+curve

I don't have a clear idea how that value can be captured, since it's going to be 90% AI generated code that anyone can scrape (public projects) or can't be used (private projects), so perhaps you're right.

The value is probably in knowing which AI-generated code ends up being pushed or discarded, which can't be derived from public projects. This information can then be used to finetune the next big model so it only generates the "good" code.

This is AI slop and the article contains some of the worst illustrations I have ever seen. Most do not make any sense mechanically. Here are the worst ones:

- The "orbiting threaded rollers" in figure 6 are not meshing with anything (not that they could, since they are orientated in the wrong direction).

- The ball of the ball screw in figure 7 deforms the screw and the roller screw "meshes" with a flat surface.

- The guy on the pogo stick in figure 14 is jumping himself rather than putting his feet on the stands of the pogo stick.

- In figure 16, a key penetrates the elastomer skin of the optical tactile sensor, destroying it.

- The gears in figure 20 touch perpendicularly.

I was not able to reproduce your problem with that prompt, but I might have a reason for why you got that answer.

Did you enable reasoning ("DeepThink")? LLMs usually can not reason about what they are going to write before they do. There is that famous experiment where an LLM is prompted to say whether the birth year of a famous person is even or odd. If the LLM is constrained to only answer with "even" or "odd", the accuracy is around 50%, i.e. no better than random chance, but if the LLM is allowed to first answer with the birth year of the famous person followed by whether the year is even or odd, it is able to "see" what the year is, and answers correctly almost every time.

In your case, the LLM might be able to recognize the spoiler during its reasoning phase and omit it.

Another explanation might be that the LLM interpreted the "No spoilers!" as "Do not spoil the tasks of the show" instead of "Do not spoil the winner".

Lastly, the question "Can you tell me...?" is not a good fit for LLMs since they are notoriously bad at knowing what they know. You can leave it out to save a few characters.

Probably nothing personal. It feels like the climate of HN is shifting towards more negativity (and less quality) during the last few months.