xAI's Colossus: Most Powerful AI Cluster Online in 122 Days

https://twitter.com/elonmusk/status/1830650370336473253

by gfortaine • 2 years ago

27 21 2 years ago

Is this notable due to the lack of HN comments?

Is this really the most powerful “AI cluster” or is this an exaggeration? I would expect that hyperscalers like Google or Amazon would actually have the largest amount of compute power to throw at AI. Or is that no longer true?

EDIT: I see a comment that has a graph of AI compute across different companies. I am not sure if this is trustworthy however, as the graph is itself just generated by an LLM (Claude):

https://x.com/AnthonyEveryWhr/status/1830680977103794177

danpalmer• 2 years ago

I'd say this graph is absolute rubbish. The fact that the numbers are round numbers, decreasing in even steps, with no units, no comparison, incorrect scaling, and incorrect layout for a graph, suggests that this is little more than fanfiction.

The truth is that the hyperscalers are very closely guarding information around things like this. Who has the biggest "cluster" is impossible to answer. Also most hyperscalers will have many clusters, and what's more useful, the SUM or MAX of those clusters? At a small scale probably the MAX, but at a large scale the SUM because you can distribute across clusters without as much loss in overheads and because you have a range of jobs to run.

Havoc3• 2 years ago

No the chart is wrong.

blitzar• 2 years ago

For these numbers to be correct Nvidia would have to be committing securities fraud on a scale never seen before.

MertsA• 2 years ago

The data in that graph is just complete misinformation.

https://engineering.fb.com/2024/03/12/data-center-engineerin...

TkTech• 2 years ago

Is there a reputable source for these claims? Given source has been proven to lie about scale, delivery, and functionality, especially for topics that might boost share prices. https://elonmusk.today/

JumpCrisscross• 2 years ago

“The utility company said that by August, xAI would have access to around 50 megawatts of power, or only enough for around 50,000 chips, and that an upcoming electric substation on the site would give him another 150 megawatts—enough to power 100,000 chips or more. But that wouldn’t happen until 2025, the utility company said” [1].

[1] https://www.theinformation.com/articles/why-musks-ai-rivals-...

summerlight• 2 years ago

This looks like quite a futile PR to downplay other players? MS already had more than 100k H100 before the beginning of this year and the same thing for Meta. Google has its own TPU cluster so it's not even possible to do apple-to-apple comparison.

Havoc3• 2 years ago

Yeah the charts posted in comments on x didn’t look right to me either

mellosouls• 2 years ago

This sounds like a notable achievement but they really ought to have chosen a different name.

https://en.wikipedia.org/wiki/Colossus_computer

XorNot• 2 years ago

Doesn't this literally not matter? We know that something Llama is being trained in about 3 months on Meta's current resources from their current datasets.

If they had less GPU power available, it would take longer but not dramatically so. Building a "more powerful AI cluster" doesn't give you a more powerful AI currently. In fact none of the Transformer architecture's seem to, and their instantiations seem to all be targeted towards "single H100 executions" for inference.

So this really does seem like nothing but hype: buying a lot of commercially available GPUs isn't accomplishing anything.

latentsea• 2 years ago

Right, but that means Meta can train 4 models a year. So, 4 opportunities for feedback. They however, need to release those to the public in order to get feedback on the limitations of the model, so that might limit it to training and releasing 2 models a year.

If you had more GPUs you could iterate faster...

torlok2• 2 years ago

Does xAI do anything interesting or are they just trying to catch up?

Havoc3• 2 years ago

To be fair catching up is quite a feat in this game

torlok2• 2 years ago

That's undeniable, but it would be interesting to see something beyond "you can generate a picture" or "you can ask our model questions, though it may make things up".

Vuizur• 2 years ago

Grok-2 is rank 2 on LLM arena, it's basically as good as the best Gemini model. They already caught up. Only the latest ChatGPT model is a tiny bit better.

ado__dev• 2 years ago

(x) Doubt

jacknews• 2 years ago

Colossus ... didn't I watch a movie about this?

disposition2• 2 years ago

Apparently running on many gas turbines (enough to power 50,000 homes), without any permits for all that pollution [1]. Also, most likely, hoovering up water from the often low Mississippi River

Just your modern day robber baron...

1. https://www.reuters.com/business/environment/musks-xai-opera...

xAI's Colossus: Most Powerful AI Cluster Online in 122 Days

Related Stories