Looks like 27b models are in reach.
“In the current generation, our density is 8 billion parameters on the hard wired part of the chip., plus the SRAM to allow us to do KV caches, adaptations like fine tuning, and etc. In our next generation, we would have the ability to go up to 20 billion parameters in a chip. Even with trillions of parameters, we’re talking about few tens of chips, which is a very, very small compared to anything else out there on the market today.”
https://www.nextplatform.com/compute/2026/02/19/taalas-etche...
Edit: i do not know how reliable this page is... it has a lot of typos.. for me it does not look like it was written by a LLM