Alternative(s) to run CUDA on non-Nvidia hardware

https://www.hpcwire.com/2026/07/09/spectral-compute-aims-to-set-cuda-free-will-it-succeed/

by alok-g • 9 days ago

143 75 9 days ago

hpcwire.com

pjmlp12• 9 days ago

Most of these "alternatives" focus on CUDA C++, and overlook what actually makes CUDA interesting.

Already in 2020,

https://developer.nvidia.com/blog/cuda-refresher-the-gpu-com...

msond7• 9 days ago

We're actually targeting all of it, and not just CUDA C++.

pjmlp12• 9 days ago

Including stuff like Fortran, Haskell, Java, .NET via PTX, Python JIT, IDE tooling integration with major IDEs, graphical GPU debugging and profiling, libraries and co?

Then I guess all the best.

embedding-shape• 9 days ago

Ambitious but neat, good luck if nothing else :)

If you were to guess, when do you think your Nsight Compute alternative might be ready with your own toolchain?

ychen306• 8 days ago

How do you deal with target-specific inline asm like tcgen05.mma?

mschuetz6• 9 days ago

Ease of programming and a giant leap in performance is one of the key reasons for the CUDA platform’s widespread adoption

This, so much. Other platforms continue to ignore developer UX, but it's one of the main things that get's new users onboard and keeps old users around.

maxloh• 9 days ago

There is also ZLUDA, which is open source and works on pre-compiled binaries.

https://github.com/vosen/ZLUDA

tuananh• 9 days ago

this is closest thing we have to "cuda on non-nvidia" hardware

msond7• 9 days ago

We have a comparison page: https://docs.scale-lang.com/stable/manual/comparison/#zluda

DiabloD35• 9 days ago

Its easier to just get rid of your legacy code entirely and use Vulkan for compute, or have your compiler emit SPIR-V directly.

No reason to tie yourself to Nvidia's moat.

swerner• 9 days ago

Unfortunately, Vulkan Compute doesn’t to all the things that OpenCL, SYCL, HIP or CUDA do.

binsquare• 9 days ago

Yep, there are inference stacks where it just does not work without cuda in any meaningful performance

mschuetz6• 9 days ago

A couple of years ago I evaluated both Vulkan and Cuda as a choice for future projects. I couldnt get anything done after a week in Vulkan, but had the test prototype project working after just a day in Cuda.

Needless to say, I'd never ever pick Vulkan for any project after that experience. It's just way to needlessly overengineered and bloated.

pjmlp12• 8 days ago

I used to be big into Khronos API camp, even did my project thesis in OpenGL, up to the famous Long Peaks fail.

Vulkan ended up being the same extension spaghetti as its predecessor, and Khronos was only able to come up with something thanks to AMD offering Mantle, C++ bindings and a GLSL successor only came to be thanks to NVidia (Vulkan-hpp and Slang started at NVidia).

The "we build the specification", and then "the community builds the tools", leads to very poor experiences, and if it wasn't for LunarG own interests, there wouldn't even exist any kind of Vulkan SDK.

What they have going is naturally the vendor independence, however we can achieve the same with middleware with the benefit of much better developer experience.

DiabloD35• 8 days ago

Weird, most people have the exact opposite experience.

Having to deal with closed source opaque poorly documented stacks sucks.

sollycb• 9 days ago

Ports are very often incredibly difficult and very time consuming.

One of the biggest complaints we hear from the industry is "we tried to port to X and we could never complete it".

An established codebase can have years of refinement. It will take time to achieve the same with the port.

And with our compiler, just using cuda is no longer putting urself inside the moat :)

DiabloD35• 8 days ago

Ironically, this is what people claim AI can do with a snap of the fingers.

Should be real simple if the HN AI echochamber is right, right?

pjmlp12• 8 days ago

Vulkan tooling is light years behind what CUDA offers in 2026, across programming languages, IDE tooling, graphical debuggers and libraries.

lulzx• 9 days ago

I have been trying for cuda -> metal, to run it on mac, https://github.com/lulzx/cuda-metal

luciana1u• 9 days ago

every CUDA alternative follows the same arc: bold launch, works for 3 operations, then a Discord server where the last message is 'any updates?' from 2024

msond7• 9 days ago

Actually we launched in 2024 and the last message in our discord is definitely not that: https://discord.gg/KNpgGbTc38

woctordho• 9 days ago

There's nothing wrong to run CUDA on non-Nvidia hardware. CUDA has an interface that is reasonably well-designed, well-documented/reverse-engineered, and battle-tested for decades. What we need is not to invent another interface just under the name of 'open standard', but to implement the same interface. ROCm is exactly doing this, and so are other hardware SDKs such as MooreThread and Alibaba T-Head.

rfv6723• 9 days ago

That sounds nice on paper, but you’re assuming Nvidia wants to play fair. Nvidia is never going to share future microarchitecture secrets, so the moment they drop a new chip and update the compiler, everyone playing the compatibility game has to start from scratch.

LogicFailsMe2• 8 days ago

Someone needs to stand up a benchmark suite for ROCM, this, and everyone else attempting it to really get the ball rolling here. SemiAnalysis could have a blast with this.

zdw• 7 days ago

A YouTuber automated ROCm version testing for Strix Halo/R9700 + LLM workloads:

https://kyuz0.github.io/amd-strix-halo-toolboxes/

https://kyuz0.github.io/amd-r9700-ai-toolboxes/

superkuh• 8 days ago

The difference between ROCm and CUDA is that when a consumer GPU is released by nvidia it's supported for CUDA for about a decade (1xxx series cards just dropped last year). When a consumer GPU is released by AMD it's not supported by ROCm till about a year after release and then it's supported for about 3-4 years. With the RX 580 there were only 3.7 years after release before ROCm support was pulled. I bought mine a couple years after release and so only had about a year and a half of ROCm. Never again.

Things might be different in enterprise but for consumer AMD GPU ROCm is a trap. It is a mayfly. Sure, you can try to run the cards unsupported but you're just multiplying the difficulty and maintainence burden. And nothing will just work.

michaelt• 8 days ago

I was in graduate school for robotics when CUDA came out, and the consumer card support was critical to nvidia getting to where they are today.

High Performance Computing option A wants to set up a call with someone with the authority to spend the best part of six figures, which could maybe be part of a funding application within a year or two, if there's a strong enough case for it.

High Performance Computing option B recommends you put in an application for time at the national centre that doles out access in 15 minute increments after you outline your entire project to them.

Then along comes nvidia, with CUDA - they want a one-off payment of $100, and on the day CUDA came out, almost every CS department already had a few dozen of the cards in computers they already owned.

No huge outlay, no ongoing spending commitment, no permission or application process.

3abiton• 8 days ago

Another crucial difference is: ROCm is open source, while cuda isn't. Yes it's tough to port things to newer gpus, but in theory people did it (therock spear heading the ROCm patch for strix halo before the offical support)

tekacs• 8 days ago

So is Spectral, which is mentioned in the headline of the article! As it says there:

SCALE delivers nearly a 6x performance boost on AMD GPUs compared to using HIPIFY to convert CUDA code to AMD’s own ROCm environment

... whilst also running CUDA.

puschkinfr• 9 days ago

In this context AdaptiveCpp should also be mentioned. Started as a SYCL implementation, but recently-ish added a compiler for compiling a CUDA dialect to GPUs and CPUs from basically all vendors

lumrn• 8 days ago

SYCL is probably the most up-to-date CUDA alternative for all intents and purposes, at least if one likes modern C++ style (and lambdas inside lambdas). Expose it as C and get bindings to any other language for relatively little effort as well since it’s just C++. With AdaptiveCpp you can also compile SYCL to CUDA so both ways work with the CUDA dialect (PCUDA).

SYCL, as well as AdaptiveCpp, is a relatively active project though and has been for several years, feeding into the C++ standards committee work and is supported by several large organisations, including US national labs and several European universities. I suppose it’s worth keeping track of for people in related fields.

I suppose it’s just really hard to beat the head start and ecosystem integration NVIDIA has with CUDA.

dachworker• 8 days ago

Why should I not just port my kernel to Triton? What's the appeal of Scale?

noselasd• 8 days ago

You can skip the porting part.

asdaqopqkq• 8 days ago

aren't llms smart enough to directly write custom kernels for custom hardware from cuda code?

u1hcw9nx• 8 days ago

Alternatives exist, but little demand outside hyperscalers and special uses.

Neocloud customers just want plug-and-play CUDA. It works, it's tested, it adapts faster, and has known performance. Alternatives give no significant benefits.

Things can change, but they are not changing now.

cactusplant7374• 8 days ago

Isn't the future of the industry specialized chips like those that Broadcom and Cerebras are making? I don't know how much longer I can tolerate 50 tokens per second. It feels like the dial-up era.

woodrowbarlow2• 8 days ago

i'm also interested in tenstorrent. they're building GPUs with cheap GDDR6 using a fast SRAM cache, and writing their own compiler stack (used instead of CUDA) that pipelines data to the SRAM ahead-of-time so you (in theory) never need to suffer the slow speed of GDDR6 for AI workloads. also they've got built-in SFP cages where the video ports would normally be.

inigyou2• 8 days ago

Is tenstorrent building GPUs now, not just tensor processors?

woodrowbarlow2• 7 days ago

you're right, i misspoke by saying it's a GPU since it doesn't have video output. the just-in-time data pipelining wouldn't work for graphics workloads anyway.

villgax• 8 days ago

@claude add this to the graveyard of wannabees

HarHarVeryFunny• 8 days ago

These efforts to support CUDA on non-Nvidia hardware seem to me misguided. If all you want is to be able to easily use non-NVidia hardware then high level tools like PyTorch already let you do that (and torch.compile uses Triton for target-specific optimization). OTOH if you want to be programming close to the metal to achieve top performance then you are probably not using CUDA in the first place, and using some CUDA translation layer on non-NVidia hardware would be an even worse idea.

msond7• 8 days ago

We actually support NVIDIA hardware, too.

In some benchmarks, SCALE beats nvcc, and we have compiler optimizations in the pipeline that will improve those numbers over time.

If all you want is to be able to easily use non-NVidia hardware then high level tools like PyTorch already let you do that

Somewhat true, but, CUDA is significantly larger than PyTorch and there's more to Accelerated Computing than just those types of applications supported there.

OTOH if you want to be programming close to the metal to achieve top performance then you are probably not using CUDA in the first place, and using some CUDA translation layer on non-NVidia hardware would be an even worse idea.

SOTA mlperf submissions use CUDA to achieve their high levels of performance.

It's not a "translation layer", it's a native, ahead-of-time compiler that makes full use of the native hardware features. Here's an example of a feature (Shuffles) being compiled to take advantage of native hardware instructions, resulting in speedups: https://scale-lang.com/posts/2026-01-19-optimizing-cuda-shuf...

LogicFailsMe2• 8 days ago

I've played with your compiler. It's not very robust yet to production code that leans into all the libraries in the ecosystem. Solve that and you've got a killer app IMO.

mschuetz6• 8 days ago

On the contrary, it's great. Cuda is the single sane compute API and system, so I'll use it even if it means being vendor-locked. If my CUDA programs start running elsewhere without much intervention, that'd be amazing

alightsoul• 8 days ago

I fail to see how scale is not just another form of vendor lock in, given that their compiler is not open source. Every compiler used today except cuda's is open source. And Nvidia can get away with it because no one else cares about development experience

leoh• 8 days ago

Why is London having such a comeback right now in AI?

fragmede• 8 days ago

Probably something to do with Deepmind being headquartered in London.

Alternative(s) to run CUDA on non-Nvidia hardware

Related Stories