HN user
Somehow I never got the hype around Groq. They served models with fast inference speed - that sounded great in theory - and as a user I was looking forward to use them. But, when I did try, I discovered that they are quantizing the models underneath. And they dont even disclose it. So I stopped using them.
The whole thing never made any sense to me - but I guess AI hype is a thing.
Yes.
related: https://www.nonbios.ai/post/deploying-large-405b-models-in-f...
tldr: uses the latest rocm 6.2 to run full precision inference for llama 405b on a single node 8 x MI300x AMD GPU
How mature do you think Rocm 6.2-AMD stack is compared to Nvidia ?
Seems to me that the only difference between Anthropic and OpenAI is that Anthropic was for-profit from day one and OpenAI is from day yesterday. I pay for both, and pretty sure they will do everything they can to take as much money from me that they can get away with.
This shouldnt be news.