xAI's 100k GPUs data center in Memphis is up and running

https://www.semafor.com/article/09/27/2024/elon-musks-new-memphis-data-center-hits-an-ai-milestone-with-nvidia-chips

by mfiguiere • 2 years ago

33 7 2 years ago

I assume there are others already running 100k GPU training clusters already, but this article claims xAI’s Colossus is the most powerful computer ever built, and that no other company has been able to connect this many GPUs together due to networking limitations. But haven’t there been other companies who’ve purchased more GPUs? Are they simply running them as smaller separate clusters?

xemdetia• 2 years ago

Smaller clusters are common because it depends on what you are trying to accomplish. It's the same problem as any distributed system- going wider with a map only works if you can perform a timely reduce. With checkpointing you run into things like network storage being too slow and so on. There's also the fact you might want to work on/with many models and what you want to do for production (inference) is different than what you try to accomplish in development (training).

michelb2• 2 years ago

I think Meta’s cluster will be slightly larger than 100k next month, scaling to 350k by end of year. And double of that sometime next year. According to their info.

turingfeel• 2 years ago

I haven’t seen any info on those being in a single cluster. They seem to just be referring to total GPUs. It’s currently almost impossible to create a single cluster of GPUs that size.

melodyogonna• 2 years ago

This is interesting, I thought the data center came online weeks ago

_davide_• 2 years ago

I'm waiting for AI bubble to burst to buy some of those at the fraction of the price :)

xAI's 100k GPUs data center in Memphis is up and running

Related Stories