LarsDu88

Certainly there will be demand for Fable7, but that demand is context specific. Frontier labs' profit is dependent on there being sufficient demand for the next layer of capability and whether the premium consumers are willing to pay for that.

The incremental unlock of capability by ever increasing frontier model sizes will eventually reach diminishing returns.

I would argue tnference speed increases would actually unlock a different kind of more meaningful value for a wider audience.

At 9000 tokens/s you could interleave a lot of requests so long as pre-fill is also fast. It really depends on how much you need to keep sessions open to take advantage of KV caching

Currently the setup is paged view in RAM shuttled to HBDRAM (VRAM) on the GPU, which in turn has to get materialized piece by piece onto cache SRAM on the GPU.

Cerebras tries to get around this by keeping everything on cache SRAM as much as possible, which it burns directly to the chip wafer itself and physically places that SRAM directly next to the tiny compute unit that does the actual math.

An ideal setup (not sure how easy this is to achieve in practice), is the burn the weights of the model directly to the chip as a sort of ROM, the actual math operations as actual digital circuits, and have SRAM, or even something akin to naked registers to directly compute off inference batch data. Cuts out 2-3 layers of abstraction and indirection.

Cerebras is not an ASIC. It is a large wafer scale chip that has a load of small SRAM modules paired with tiny compute models. The SRAM is basically a giant cache that is supposed to eliminate the bottleneck between shuttling and materializing big tensors between DRAM and cache, but the actual amount of SRAM is still nowhere near enough to server a frontier model on a single wafer. You still need dozens of wafers, which makes Cerebras rather cost intensive given that Nvidia can get dozens of GPUs out of a single wafer

Honestly, whether you think burning current SOTA to hardware is an overinvestment risk depends on what your definition of intelligence is. If you think intelligence is something that can grow like height such that 18 months from now we will basically be bowing down to machine god giants that are running on B200s, then investing in ASICs is the wrong move. However, if you you subscribe to the (very reasonable view) that intelligence is more like a round ball that we are trying to make as spherical as possible (ala Francois Chollet's writings), then at some point the ball will be smooth enough for most people and many tasks.

It takes about 18 months to go through the design, verification, and manufacturing process if you move at breakneck pace. Design could probably be sped up.

About 18 months ago the top model was GPT-4o. Not great by today's standards, but still good enough for many tasks (certainly a big chunk of chatbot queries). The current SOTA covers far more use cases, but importantly at a level that surpasses many thresholds of utility.

The closest example I've seen is ChatJimmy: https://chatjimmy.ai/ a prototype from Taalas running Llama 8B

Scaling this up to 2.8 Trillion (350X increase), will certainly be challenging.

If I was younger and had the right background, I'd love to dive into attempting somethign like this

They commoditized their compliment, but made sure their chips only worked with their accessory products, particularly in the 70s

cough Ub Iwerks

The open weight, open architecture releases of the past several days has me more convinced that ultimately, the winner will be whoever burns their models to ASICs fastest.

The LLMs themselves are capable of doing some aspects of chip design as evinced by the K3 press release.

Furthermore, the frontier models are "good enough" for a wide swathe of tasks and will soon hit that threshold for a good amount of software engineering (if not already). Does anyone think we need a Mythos level model to plan a road trip, or give someone tips on making a cake recipe?

A Fable 5 model running at 9,000 tokens/s on an ASIC rather than 150 tokens/s on electricity chugging Nvidia GPUs, or even giant SRAM Cerebras or Groq chips could be good enough to meet the majority of demand.

Furthermore, if you're an enterprise the risk of data exfiltration and feeding data to a potential competitor like OpenAI or Anthropic is greatly reduced if you could shift to on-prem ASIC deployments. A handful of chips could cover a wide variety of use cases and cover them more securely. There are a lot of corporate use-cases for LLMs that are not frontier math research or coding.

You mean Jack Kirby right? LoL

I don't get this "build a platform" nonsense. He's a designer. A good one. Who ripped off much of his aethetics from vintage Braun products.

Andry Grove was great, but Intel really was the epitome of "competition is for losers"

In the early days all their products were explicitly designed to only work with each other to create a hardware walled garden.

So build the full model with SRAM and all?

I believes the weights are burned as ROM microcode, but for an effective inference speedup, you do want to burn the architecture (matmuls, activation functions, MoE gates, etc) as well which will differ from model to model.

It's not as simple as a weight swap between identical architectures.

The speed gains are also from not having to route the weights through wiring like with ROM cartridges.

Someone do this right now. I will join!

A really good startup idea right now... Use kimi k3 to reproduce the kimi k3 asic design and start fabbing it immediately. In 12-18 months, start spinning up your own cloud and start competing with the frontier labs ASAP.

Who needs superintelligence with you have 8,700 tokens/s at near Fable levels of performance???

This is like the Bill Gates, Paul Allen moment, but for hardware.

The narrative that superintelligence is imminent is partially at fault here.

There are competing definitions of what intelligence even is, and the one that I find most striking is from Francois Chollet which is that intelligence can be boiled down to skill acquisition efficiency. This type of definition makes intelligence more akin to polishing a ball than growing a watermelon.

The superintelligence doomers warn that the watermelon is going to start growing exponentially and crush everyone. But what might actually be happening is that we are not growing a watermelon but rather polishing the ball until its really smooth and shiny. There's a point where you can get it to micron levels of polish but for most tasks (white collar text domains tasks), it's smooth enough! You will be able to go to the ball store and buy a low cost made in china ball for most tasks.

The real challenge is actually branching out domains and modalities to tackle things like blue collar labor. Over time, white collar work automatable or able to be made hyperefficient by LLMs will see LLM commoditization.

You're confusing Tesla for like 10 other Chinese brands. Tesla too is falling behind, particularly when it comes to price.

BYD may very well be pulling ahead on battery tech and vertical integration right now.

In a few years Tesla's primary moat will be political

To be fair Unreal has rock solid netcode and Doom The Dark Ages has non-existant netcode.

Unity does not make that much money from assets. They make the majority of their revenue through licensing their engine to "whales"... the small percentage of games that make huge revenue.

They also make money through ad services... a market they seriously missed out on (look at AppLovin stock vs Unity).

Asset sales are barely a blip.

Unreal makes the vast majority of it's revenue through microtransactions on its one major whale game Fortnite

MSFT could have opened up idTech completely since they make 0 dollars from licensing the engine anyways.

Microsoft's game divisions make money through making games, so opening up the engine itself would've been conducive to their goals (cultivating an ecosystem of devs and even contractors familiar with the tooling).

idTech rendering is more competitive with Epic's Unreal technology than Unity and Godot.

They should simply open source it if they fire the devs. Else the engine and future support for the games built on it are essentially being tossed into the trash.

Microsoft, one the world's greatest monopolists, bequeaths a game engine monopoly unto Epic Games, in one the biggest corporate blunders of all time.

If they were smarter about this, they would commoditize their compliment and open source the Doom The Dark Ages engine just like John Carmack did with the Quake 3 engine.

Reddit thread with ongoing information regarding the layoffs: https://www.reddit.com/r/Doom/comments/1up5pta/95_reportedly...

Scott Miller (founder of Apogee/3dRealms) stated that id Software and most of programmers (the team behind idTech and Doom: The Dark Ages) will be let go.

Basically one of the last cutting edge engines not built/maintained by Epic Games (Unreal Engine) will basically die.

NO2 and blood pressure is not the same as death

Correction. More sun exposure is CORRELATED with being less likely do die. I don't think there's a single causal connection in this article.

Which unity mcp do you use? I've been playing around with the official one, but was wondering what other folks use.

Ran into a package conflict issue with the popular coplay one

Fun exercise. Type in pokemon or japanese. You can really see the nearest neighbor text in embedding space. Pokemon gives passafes referencing animals and japanese passages referencing foreigners

Pretty effective. You can even type in "child murder" and "genocide" and get relevant passages

John Carmack slaved away writing super optimized ground breaking realtime 3d game engine code that created a multibillion dollar industry.

Adrian Carmack mostly took photographs of clay scultures (and in some cases actual plastic toys, in the case of the chainsaw and pistol used in the original Doom), and digitized them in Corel. Something any half-way decent art student could do (not to discount the iconic visual style of Commander Keen and Doom!)

It was Adrian who walked away with 41% of the >100 million dollars company!