Why is everybody talking about Groq?

https://www.zach.be/p/why-is-everybody-talking-about-groq

by zachbee • 2 years ago

3 6 2 years ago

zach.be

mac3n• 2 years ago

first i've heard of it

declaredapple• 2 years ago

The reason for the SRAM according to them,

If you want low latency you have to be really careful with HBM, not only because of the delay involved, but also the non-determinacy. One of the huge benefits of our LPU architecture is that we can build systems of hundreds of chips with fast interconnect and we know the precise timing of the whole system to within a few parts per million. Once you start integrating non-deterministic components your latency guarantees disappear very quickly.

https://news.ycombinator.com/item?id=39432119

zachbee• 2 years ago

The reason HBM "isn't deterministic" is because the memory controller sometimes stalls reads when doing a refresh operation. It's possible to pause refreshes [1] or schedule refreshes ahead of time to get HBM and DRAM to act deterministic.

To be fair, that's a huge hassle, and I have no clue what level of support commercial HBM controllers have for refresh pausing or manual refresh scheduling. It could very well be impossible to do in practice -- and regardless, their performance numbers speak for themselves :)

[1] https://memlab.ece.gatech.edu/papers/TACO_2014_1.pdf

pg_1234• 2 years ago

nobody seems to be talking about Groq ...

kristianp• 2 years ago

It's pretty confusing, because X (formerly twitter) has a chatbot called Grok. I assume this is a different thing?

tome• 2 years ago

Yes, it's different. See https://wow.groq.com/hey-elon-its-time-to-cease-de-grok/

(I work for Groq)

Why is everybody talking about Groq?

Related Stories