Tinygrad and rusticl and aco: why not?

https://airlied.blogspot.com/2023/07/tinygrad-rusticl-aco-why-not.html

by pantalaimon • 3 years ago

46 40 3 years ago

airlied.blogspot.com

rhn_mk13• 3 years ago

Rusticl seems a bit hidden for all the (positive) noise it creates.

I'd like to try it out, but:

- are there any distributions shipping it?

- what hardware does it support?

I can guess that distros are not shipping it due to some stability concerns (which I could not see so far). I'm also guessing that despite being a Gallium driver, some features will depend on the capabilities of the underlying hardware. Can I hope that my AMD APU will finally get some OpenCL support?

brucethemoose27• 3 years ago

Can I hope that my AMD APU will finally get some OpenCL support?

What APU is it? OpenCL support goes back pretty far.

rhn_mk13• 3 years ago

I haven't found anything for the Vega 8 that would run on Linux.

pantalaimon• 3 years ago

what hardware does it support?

That’s the nice thing about it, it supports most hardware supported by Mesa going back even to r600 GPUs [0]. So anything Intel or AMD should work.

My guess for as to why distributions are not enabling it yet might be due to the Rust build dependency that might require a newer Rust version than what’s provided by the distribution.

[0] https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20...

imtringued• 3 years ago

No mention what ACO is.

The benchmark is just a dump of the console output with no explanation what the individual phases mean.

If I am going to be honest, this is a terrible blog post.

sva_• 3 years ago

AMD COmpiler

mathisfun123• 3 years ago

No mention what ACO is.

Griping about initialisms has gotta be hn's favorite pass-time by far. It's as if we don't live in a world where things can be looked up with the flick of wrist. It's as if we're not discussing this on a website entirely premised on making information access/retrieval more efficient.

Mesa ACO compiler

More than enough disambiguation that the first hit tells me exactly what it is.

If I am going to be honest, this is a terrible blog post.

Actually, I was wrong: cheap disparagement is hn's favorite pass-time.

jchw4• 3 years ago

large GPU stacks usually make me want to wipe the machine after I test them

Core value proposition for NixOS/GuixSD right here. That said, the amount of time you have to dedicate to figuring out how to make everything work well for your use cases might negate the benefits a bit :)

fbdab103• 3 years ago

Cannot vouch for it, but the nixified.ai project is precisely geared towards this idea. Having Nix builds to run machine learning environments[0]

[0] https://nixified.ai/

colordrops5• 3 years ago

Once you figure it out you never have to figure it out again.

And there is likely someone else who already figured out something as common as this so you could just import their config.

jchw4• 3 years ago

I love NixOS, but if you're a heavy Linux user the odds that you do something fairly unique in your personal setup gets very high. For me, to give you an example of just one little microcosm, I have probably the only combination of SwayWM, gnome-keyring and NixOS in the world that works the exact way mine does, and it took quite a while to land on it (and sometimes time to maintain it.) It's actually pretty simple and elegant now, but getting to where I am today felt anything but simple.

Back when I first started (5 years ago,) drivers for my label printer were not available, and I stumbled around trying to figure out how to make a derivation for them. Ultimately I succeeded, but for a n00b it did feel pretty ridiculously hard to try to figure it out while not even having much of a background in pure FP languages less the oddity that is Nix. I never got around to adding it to Nixpkgs, but thankfully somebody else did.

There is indeed solutions for a LOT of things. Is home manager worth the time? Not sure, I just use global configs instead of home dir configs for most things. There's multiple solutions for secrets but I have a few different Nix configuration repositories (for different sets of machines, not one for each machine; I only do this when it feels like it makes sense, like to keep servers for some org separate from my home machines) and not all of them have SOPS setup because while it works, bootstrapping it always has me guessing a little to make sure the way I'm setting it up actually makes sense.

Flakes. They're good, but I started before everyone recommended them for everything. The switching cost of flakes was not huge, but nor was it trivial, as I had to adjust a lot of workflows for it. This kind of negates the "you never have to figure it out again" thing. On the contrary, NixOS is a moving and vastly complicated machine, and it indeed feels like things need to be re-figured out quite often. Rust dependency management in Nixpkgs is sure to change in the future: right now it can't handle all of the situations that Cargo can and often requires including a copy of the lockfile in the Nixpkgs tree which seems unideal.

Worth it still? Yes, because I can throw my boot drive out the window and have a machine back up and running in like 10 minutes. All of my desktop machines are very consistent, because any configuration that matters to me goes into declarative and versioned configuration. The concept of "hosing" my OS doesn't exist anymore, and even if a workflow does break, I can trivially bisect it to figure out why. Nix tools are super complicated, but they are super powerful too. NixOS is Linux with superpowers. All of the cool stuff that open source could theoretically enable you to do, NixOS makes practical.

But is it perfect? No. To this day, you will not find a derivation for plenty of very useful closed source software. No VMWare Workstation, IDA Pro, etc. AppImage binaries that run other AppImages don't work with appimage-run; though I have a workaround with Nix-LD that does work, I've never seen anyone else doing it, so I suspect not too many people do this. Most open source software is there, but it can be a bit hit or miss at times. You'll occasionally find places where missing dependencies or hardcoded paths in the upstream sources lead to subtly broken or suboptimal things.

So if you only manage one machine with NixOS or you don't have a lot of time to invest upfront, or you have some use cases that are pathologically difficult to deal with, it's not always practical for everyone.

I think if NixOS immediately sells someone, they're in. They'll get it. If not? Maybe in a couple more years. For me, it was on first sight.

aseipp• 3 years ago

Ehh, the reality is not quite that rose-tinted; in practice you have to put the elbow grease in, still. It's normally not quite as simple as porting someones config IME, because often it's wrapped up with other things they need. I've been a NixOS user and committer for 10 years. Things do in fact change and sometimes you have to learn a new thing, often. I'm dating myself here but I still remember when the Great Haskell Package overhaul happened, in an attempt to start vendoring Haskell packages directly from upstream Hackage; before that there was totally different infrastructure and every package was added manually. That broke a lot of things.

Similarly, I only started using Flakes last year after years and years of development. Rust packaging is still pretty bad and non-granular, unless you use cargo-based generators which all have issues, and none are upstream. If you are using even more marginal languages, your experiences will probably range from bad to worse. NixOS is constant evolving and things do get deprecated and changed. You do have to learn new things.

What you don't have to deal with as often is just, things randomly breaking. And with Flakes you don't have to deal with ambient Nixpkgs changes. I have shell scripts using nix-shell I haven't had to touch in years, despite incompatible changes in the tools used by the script. I still have a walled off NixOS machine running a 15.03pre-release, IIRC. I could probably upgrade it but the software on it isn't going to break. I suspect the disk will fail first (which is fine for this machine.)

I think the best analogy is this: tools like Nix, Guix etc let you move mountains as an individual. That's impressive. It still requires you to get your hands dirty and actually push the thing, which is still work.

brucethemoose27• 3 years ago

I have griped about this before, but I don't really get the hype around tinygrad... GitHub is littered with ML frameworks, some of them (Like MLIR/TVM) with very good working demos right now, and other smaller scale ones (like GGML) are SOTA in very specific niches.

pseg134• 3 years ago

And did you gain any knowledge the last time you “griped” about it? If not then who are these “gripe” sessions even serving?

brucethemoose27• 3 years ago

And did you gain any knowledge

...No.

I am not against tinygrad or anything, but I can't find any projects using it as a backend. And I can't find benchmarks for, say, the stable diffusion or llama demos.

But I am frustrated that it seems to be attracting lots of attention on social media, while other ML frameworks/projects targeting wide hardware support, and already being used as backends in places, are flying under the radar.

And Hotz's very brief spat with AMD that I saw on his YouTube channel also raised my eyebrows.

steeve• 3 years ago

MLIR is really cool imho

mellosouls• 3 years ago

I don't really get the hype around tinygrad

geohot

whether that validates it or not, I think thats the reason.

dist1ll• 3 years ago

Calling a compiler infrastructure project like MLIR an "ML framework" feels a bit disparaging.

brucethemoose27• 3 years ago

No, but its an increasingly popular middle layer for other frameworks.

WinLychee• 3 years ago

It's super easy to install and use, and you can comprehend it in its entirety _because_ it's such a tiny project. Pop open the source code and browse around! The genius is the simplicity.

sitkack• 3 years ago

XDC 2022 | Rusticl: An OpenCL implementation written in Rust | Karol Herbst

https://www.youtube.com/watch?v=Es7UmvNAEvk

Tinygrad and rusticl and aco: why not?

Related Stories