A dive into the AMD driver workflow

https://geohot.github.io//blog/jekyll/update/2023/06/07/a-dive-into-amds-drivers.html

by tikkun • 3 years ago

153 76 3 years ago

geohot.github.io

tikkun• 3 years ago

A possible prescription for AMD regarding AI and CUDA:

1) Open-source driver development as mentioned in this post

2) Set up 24/7 free software tech support on discord. Maybe for all use cases, maybe only AI use cases. Do the tech support via screen sharing, and have a writer join for all calls, so that every issue once solved gets a blog post

3) Have employees run all popular AI tools and get them working on AMD hardware, publish written guides and videos showing how to do it.

zirgs• 3 years ago

4) Release a consumer GPU with 32-80 GB of VRAM.

ciupicri• 3 years ago

What are you planning to play on that thing?

Tuna-Fish• 3 years ago

It sounds silly, but people would endure a lot of pain to fit significantly larger models into ram.

fbdab103• 3 years ago

It could even be 2-3 generations behind and that thing would still sell.

latchkey10• 3 years ago

Or at least make it so that you could rent the larger gpus by the hour.

As far as I know, there isn't a single service that offers bare metal access to MI210/MI250's.

codedokode3• 3 years ago

AMD has CPUs with built-in GPU with hundreds of ALUs and unified memory (GPU uses system memory). Cannot such CPU be used for ML tasks without purchasing expensive graphic cards? 64 Gb of ordinary RAM will be cheaper than 64 Gb of VRAM.

Currently such built-in GPUs seem to be optimized for games only.

hedgehog2• 3 years ago

The problem is not that people within the company don't have good ideas of how to improve deep learning research end user experience, it's just not a priority for AMD. It's annoying as a potential customer but arguably whatever their overall strategy is it's working.

lul_open• 3 years ago

Yes, let's build an open source community on top of a closed platform. It's not like twitter, reddit, Facebook etc have taught us anything.

Open a mailing list like every project that survives more than 5 years. Hell the barrier to entry will ensure you get people who can use a text editor and save you half the question of 'how do I install this on on an Intel a2 Mac???'

branon• 3 years ago

Yes, absolutely do not use Discord for this, Matrix would be better

oldgradstudent• 3 years ago

A note culturally, I do sadly feel like what they responded to was george is upset and saying bad things about us which is bad for our brand and not holy shit we have a broken driver released panicing thousands of people’s kernels and crashing their GPUs. But it’s a start.

LargeTomato• 3 years ago

George is right.

But

He has a history of being an arrogant prick. That will color people's perceptions of you even if it's not relevant to the immediate interaction.

brucethemoose23• 3 years ago

I didn't even know who Hotz was, but that YouTube rant on this very issue made me extremely skeptical.

Should someone who will just drop a GPU vendor if they get flustered really be leading a ML framework?

ysleepy• 3 years ago

I found his video pretty dumb. Everybody is assuming the driver is shitty, but he tried is an officially unsupported[1] GPU with ROCm and it failed. Big whoop.

He is a serial shit stirrer and this is no exception.

Did he try Windows?

As much as I hate that AMD is not really supporting consumer GPUs for compute, presenting his problems as some variant of production drivers breaking is a stretch at best.

[1] https://rocm.docs.amd.com/en/latest/release/gpu_os_support.h...

wmf• 3 years ago

Everybody saying "poor AMD is such an underdog" for years = perma-broken drivers

George being a prick one time = suddenly working driver appears?

My perception is indeed colored but in the opposite direction.

znpy• 3 years ago

And in order to beat them, you must be playing to win, not just playing not to lose. Combine the driver openness with public hardware docs and you have a competitive advantage.

This is so immensely true. I’ve been so enthusiastically loyal to intel gpus in the last ten years or so (avoiding laptops with any kind of discrete gpus) because dealing with closed drivers is so much pain.

I’m still skeptical about amd gpus, even though i hear good stuff about it.

I kist want hardware i can trust, knowing I won’t have dumb driver issues.

WesolyKubeczek• 3 years ago

I knew for years that AMD’s development model leaves a lot to be desired.

They like to throw big balls of source over the wall, and very soon after the bugs that keep haunting the previous generation of hardware just stop getting fixed. You’re SOL unless Dave Airlie himself runs into identical problems on his personal gear and gets angry enough about it to make a fix.

iforgotpassword2• 3 years ago

Yeah, I've ranted about this here in the past. I'm glad someone high-profile enough is finally doing the same, as that might actually lead somewhere.

Their closed development process is so moronic. It means their code is always ahead and out of sync with what is in public. There have been user provided fixes and improvements to ROCm on GitHub with no reaction from AMD. Probably because it wouldn't apply cleanly to whatever they currently have. It's sad to see your customers having to fix your drivers. It's even sadder to see you ignore it.

ethbr0• 3 years ago

This is the greatest travesty of closed/open models, IMHO.

I get exactly what happens -- devs are shipping their sprint pipeline internally, and pull requests or issues from the current open source head never make it onto their radar.

This gatekeeps bugs visibility behind whatever PM is running sprint planning, which always leads to delivering what management wants instead of what users are dealing with.

In complex, diverse environments, you're always going to have a myriad of bugs that are caused by a specific configuration.

Absent a path to actually fix that, you end up with an enterprise-only product that's only stable in extremely specific configurations.

And the greatest evil... you're also ignoring helpful reports of configurations where it's broken!

Roark66• 3 years ago

You can say a lot about nVidia, but for me all their products mostly just work on Linux(I use cuda a lot). I don't understand why AMD is having such trouble doing the same. Likewise with CPUs. It is ironic I have to be using Intel's math libraries to get good performance out of my AMD CPU.

shmerl2• 3 years ago

For gaming and desktop usage, situation is completely the opposite. Nvidia is plagued by having not upstreamed drivers, and AMD just works.

amlib• 3 years ago

really on linux you would be better off having a primary amd or intel gpu and then a secondary nvidia gpu just for compute/cuda tasks. But alas, many new am5 motherboards don't even have a secondary 16x pci-e slot capable of even pcie-4 4x bandwidth anymore. I guess we should thank this cloud compute craze for that...

sosodev2• 3 years ago

This hasn’t been my experience on Linux. The nvidia drivers appear to “just work” but actually caused a lot of instability. My desktop no longer crashes every other day (or more often when gaming) since switching to an AMD GPU.

theresistor• 3 years ago

I always see this comment, but I have used NVidia GPUs for gaming on Linux for a decade and they have always worked perfectly with the proprietary drivers.

jeroenhd• 3 years ago

They work absolutely fine if you stick with what worked five years ago and don't update anything until some Nvidia blog says you should.

Just don't use Wayland, don't use too many screens, don't use a laptop, don't run a recent kernel and don't expect software features like their special sauce screen recorder, or that trick where you can get a free camera inside a game, or anything else packed into their gaming toolkit on Windows. Oh, and accept a very high idle power draw if you choose not to go with Windows.

With all of that, most games work out of the box. CUDA works, video encoding and decoding works (though you're severely limited in terms of the number of simultaneous streams without hex editing the driver).

I do get the occasional Nvidia related crash, but it's been a while. Still, I don't think I'll ever consider buying Nvidia for a Linux device again. I was a fool to think "Nvidia has come a long way, I'm sure a laptop with an Nvidia GPU will work fine with a few tweaks".

Gordonjcp• 3 years ago

I've found them to be rock solid stable, and of course unlike AMD they are accelerated.

NVidia are literally the only game in town for video editing, because AMD won't provide compute acceleration in Linux.

xbeuxhedovksb• 3 years ago

Having a Nvidia Linux gaming setup, from my experience it doesn't work well at all : broken wayland, no hw video enc, etc.

Sure some of those could be solved by sending weeks trying hacks things like VA-API over NVDEC emulation, but at the end of the day this is waste of time, tl:dr everything works on Nvidia Windows, many things broken on Nvidia Linux.

NVidia on Linux is only good for non desktop use : cuda/compute/ml over ssh.

AMD is the exact opposite, desktop things work better except there is no cuda/compute/ml

So if you want something that works over the whole spectrum : desktop, wayland, compute/ml, video enc/dec, screen sharing, etc. it doesn't exist on Linux

aliljet• 3 years ago

I'm so thoroughly confused about why AMD wouldn't be falling over themselves to enable geohot and his followers to build an alternative to CUDA and NVIDIA. This feels like a conversation that geohot is attempting with feckless product and software managers who certainly can't make bold decisions. Has the CEO of AMD effectively spoken about this problem?

izacus• 3 years ago

Why would they spend limited resources on a fight they don't want?

Part of leading a company is knowing which markets to pass up.

fragmede• 3 years ago

You say that like CEOs are infallible. Thorsten Heins and Antonio M. Perez would like to have a word with you.

ddtaylor• 3 years ago

ROCm is a bit of a mess. I have used it successfully on a 6700 XT without issue but it was confusing because I thought I would need a "pro" driver but I did not.

codedokode3• 3 years ago

Knowledgable people, please explain. Vulkan allows executing code on GPU. Can it be used for typical ML tasks (e.g. large matrix multiplication)? Why do we need drivers like CUDA or ROCm then?

raphlinus• 3 years ago

I think we'll get there but it will take years to arrive at a robust solution, because of the complexity of the ecosystem. The cooperative matrix extension, the portable and standard way to do what is usually marketed as "tensor cores," just landed in the spec and drivers a couple months ago. It will take people time to figure out how to use them effectively, partly because you have to query which options are supported at runtime.

There's still a driver and you're still at mercy of it (bugs and all), it's just much more likely to be installed on your Windows or Linux machine than a ROCm stack.

boustrophedon• 3 years ago

To answer your second question directly: CUDA predates Vulkan.

So if your question then becomes: Why did we need Vulkan if we have CUDA? OpenGL, DirectX, Metal, and Vulkan all have lots of graphics-specific concepts built in to their APIs. While you could probably implement most of this in CUDA, at some level you need hardware support for your graphics APIs because in the end the data in your framebuffer goes from the GPU's memory to your monitor via HDMI or DisplayPort.

CUDA has no interface for that graphics-specific stuff (technically there are CUDA-OpenGL interop APIs inside CUDA) because it has to be able to run on AI acceleration hardware in datacenters (although that hardware didn't exist when CUDA was invented either, to be fair)

dur-randir2• 3 years ago

That's why "ROCm support" is pure marketing and FUD. And people rightfully won't buy into it.

A dive into the AMD driver workflow

Related Stories