2028 Could Bring the Most Mind-Bendingly Expensive Apple Product of All Time

https://gizmodo.com/2028-could-bring-the-most-mind-bendingly-expensive-apple-product-of-all-time-2000784460

by megamike • 10 days ago

6 3 10 days ago

Are we going to get 1000 tps local?

andy99• 10 days ago

I’ve been using local models on a Framework desktop. I get > 50 tokens/s on Qwen 3.6 35B A3B, and I find that speed ok for coding (though the model not quite). If machine could do a SOTA model with say 0.5-2T parameters like GLM 5.2 at 100 tok/s, it would be very useable.

In 2 years presumably the SOTA models will be much bigger, but I won’t be surprised if there’s a fable level sub 50B parameter model by that point either*, in which case 1000 tok/s may be reasonable.

* on artificial analysis intelligence index, GPT 4o is 12 and Qwen 35B A3B is 32.

nojvek• 9 days ago

It's just assumed that we need to make models bigger to get more out of them. That is from observation, no one has actually distilled different kinds of intelligences from an LLM to say this is the minimal model size needed for this kind of problem.

Examples: Databases scale with data stored in them. It's a linear relationship. We have databases that load pages on demand as needed and keep hot ones in cache.

There is no 'database page' alternative for LLM's yet. Someday there will be and the paradigm shift will happen.

Local 100 tok/s+ is a gamechanger for sure.

2028 Could Bring the Most Mind-Bendingly Expensive Apple Product of All Time

Related Stories