am17an

It's expensive now, I expect once it is with inference providers it will be really dirt cheap. Then it would be truly be a "bicycle for the mind", which Fable promised to be except it proved to be too capricious for that.

Your username indeed checks out

I think we're saying the same thing? I said they are explicitly trained on these tasks, not that they are some separate models during programming RL, or business tasks RL.

The cool thing is that training on diverse datasets improves scores on other downstream tasks, while the training objective is the same.

Maybe for some language modelling tasks, i.e. it learns some internal representation that is transferable. However I would find it quite odd if a model becomes good at Bio while not explicitly going through Bio training.

Have you looked at the what data companies (e.g. Scale, Mercor) hire for? Why do you think Meta records their employees every keystroke/mousestroke/eye-movement?

EDIT: just re-read your comment. I don't think you have a good understanding here, no offence.

nor are they trained for each task individually.

They are explicitly trained for each task individually.

Having looked at his code, I doubt this.

llama 3? Are you from 2023?

Who is their right minds would be wedded to an identity of saying "No"? Code quality puritans are annoying but if they do their job right they actually speed-up the development process because they don't let technical debt accumulate. Ultimately saying "No" is protecting your codebase. In the era of LLMs, saying "No" is much easier because you don't have to worry about the author feeling bad.

Fair enough. I agree with you - although DS4 Pro is a GPT 5 class model which scores 46% on ARC-AGI-2[^1]. It's behind by maybe 9 months, I think it's still good enough for a lot of complex tasks as well. They definitely need to work on a "just fucking works" harness like CC/Codex. Also thanks!

[^1] https://www.nist.gov/news-events/news/2026/05/caisi-evaluati...

Did you read the OP when he's exactly chiding the model you're glazing?

This Claude front end skill is now soon to be slop.

People doing economics with the cloud GPUs, of course cloud GPUs are going cheaper. But also, is generating tokens all you do with your computer? I can play games on DGX spark and also do LLM inference, so sometimes the economics work out, apart from having fun with it.

Local models embody the hacker spirit, constant Claude glazing is spiritually incompatible with tinkering. Don't upload your spirit to the cloud.

No I mean more expensive, i.e. you're consuming vastly more tokens.

The cost being reduced is the cost of your labour. Tokens are only getting more expensive.

Sounds exhausting. Are your revenue numbers up?

Don’t underestimate the march of technology. Just look at your phone, it has more FLOPS than there were in the entire world 40 years ago.

Thank you, there are two things I would like to point out:

1) Google releasing something probably means they don't see it as important. 4-bit KV-cache quantization has been known for a long time. The fact there is almost a mass hysteria about this paper makes me think there is a lack of skepticism in this AI mania, even in relatively tech-savvy crowd.

2) But prices for memory companies are crashing! look around, the whole market is crashing.

There are techniques which already achieve great compression of the cache at 4 bit, eg using hadamard transforms. Going from 4 bit to 3 bit isn’t the great leap people expect this to be. It’s actually slower to run and is generally worse in practice.

Welp, back to pip

Working in open source, I've now heard a wide variety of disabilities that people have and they have to be aided by an LLM for writing even descriptions of their PRs.

You can still run larger MoE models using expert weight off-loading to the CPU for token generation. They are by and large useable, I get ~50 toks/second on a kimi linear 48B (3B active) model on a potato PC + a 3090

Sure. “Tell me a joke”

I was referring to the 35B version. It is surprisingly good for its size. You can use it for implementation tasks without it going off the rails

Damn I’m jealous that they figured out how to pay their contributors. I’ve been toiling away for free

They already have with qwen3.5

What do you use for sub-50ms inference?

Honestly you can run this on a 16GB VRAM GPU with llama.cpp. Just try it!

One often overlooked after that is ggml, the tensor library that runs llama.cpp is not based on pytorch, rather just plain cpp. In a world where pytorch dominates, it shows that alternatives are possible and are worthy to be pursued.

Holy smokes we're cooked.