Writing an open source GPU driver without the hardware

https://www.collabora.com/news-and-blog/blog/2022/01/27/writing-an-open-source-gpu-driver-without-the-hardware/

by mfilion • 4 years ago

446 90 4 years ago

wmf2• 4 years ago

And it's already time to start on the Mali-G610/710 for the RK3588.

panny2• 4 years ago

This is going to be great news for new MediaTek chromebooks coming out in June with 2016 MacBook level cpu performance. rk3399 was just barely usable with a lightweight window manager. I may even pick up one of the 8192s in the meantime if a distro like arch linux arm starts supporting it.

Great work from Alyssa and everyone else at Collabora.

pabs32• 4 years ago

Alyssa has also been working on Apple M1 GPU drivers too:

https://rosenzweig.io/blog/asahi-gpu-part-1.html https://rosenzweig.io/blog/asahi-gpu-part-2.html https://rosenzweig.io/blog/asahi-gpu-part-3.html https://rosenzweig.io/blog/asahi-gpu-part-4.html

panny2• 4 years ago

I'm not excited about M1 at all, tbh. I'd rather have a corebooting chromebook, even with the performance differences.

my123• 4 years ago

mt8192 is already buyable for me, that said mt8195 is a significant bump over that: https://www.mediatek.com/products/products/tablets/mediatek-...

Hopefully that reaches retail quite soon.

atty• 4 years ago

As someone who never touches drivers or hardware directly, these sorts of stories are just wild to me, I love reading them. Congrats to Alyssa and the team; as usual, a very interesting write up and impressive work.

GeekyBear• 4 years ago

I really enjoy Alyssa's writing style.

zakki• 4 years ago

Does it mean we can have open source nvidia gpu driver as well?

pabs32• 4 years ago

The open source nouveau driver has existed for years, the problems with it are mainly caused by nvidia; not releasing signed firmware for use by nouveau, not releasing specifications etc.

xvilka2• 4 years ago

What about writing a tool to extract that signed firmware from the proprietary drivers and just using it? I think with some additional reverse engineering it's probably possible to automate this, assuming NVIDIA will not start changing the drivers specifically to fight this.

solarkraft3• 4 years ago

As far as I know nouveau is not limited by reverse engineering respurces as much as actual security measures put in place by Nvidia. IIRC you need some kind of key to clock the card up to a reasonable frequency.

speed_spread• 4 years ago

What an asshole company. I think they have displaced Oracle from the number one spot. I wish the people who thought this idea of locked hardware to have a terrible diarrhea.

stefan_• 4 years ago

GPUs that don't happen to share the silicon with the CPU like on most mobile SoCs are a lot more independent. In the case of NVidia that means integral parts like clocks are locked behind signed firmware and open source drivers are forever stuck running them in their "just enough to show a boot logo" mode.

mschuster914• 4 years ago

Would it not be possible to distribute a program that downloads the proprietary Nvidia driver, extracts the firmware and loads it into the running kernel?

IIRC there was a Debian installer for Sun's JDK/JRE that did precisely that.

izacus• 4 years ago

Opensource nvidia driver is called nouveau and it already exists. What are you really asking?

capableweb• 4 years ago

The nouveau is buggy and has poor performance, sometimes getting blacklisted by various software as well because of this (Example: Chromium - https://news.ycombinator.com/item?id=18834715). It doesn't help that NVIDIA is basically doing nothing to help the nouveau team to do a better job, and that their own driver is also buggy and has poor performance.

wanderer_3• 4 years ago

Linux magic, mucking about in the kernel, M1 compatibility hacking, this writeup has got it all!

Unfortunately, having read this, I feel rather stupid.

pvillano• 4 years ago

you're probably smart at something else

wanderer_3• 4 years ago

probably

solarkraft3• 4 years ago

I clicked the link to check out whether the article was written by Alyssa and - yep! Damn GPU wizard.

Thanks a lot to Alyssa, Collabora and everyone else involved for the amazing work!

miohtama2• 4 years ago

This is amazing work. But what’s the motivation? I understand in 90s Linux was a niche and we had to write our own drivers. But today Linux runs on 2B devices - shouldn’t hw vendors to write their own drivers today? Why try to even support open source for non-co-operative hw companies - just stick with “good” hw companies?

zozbot2346• 4 years ago

By and large, HW vendors are writing hacked-together, barely-functional drivers that only work on a heavily patched "downstream" fork of a single version of the kernel code. This is for proper support that can actually be merged in the mainline kernel and maintained for the foreseeable future.

dj_mc_merlin• 4 years ago

How the fuck?

asciimov• 4 years ago

I've got a CS degree and of all the things that I studied, including OS design, nothing gives me that feeling of black magic like Kernel hacking and driver building. It's like being told your gonna learn how to draw an owl you start with some ovals and then you have a completed owl.

Low level stuff has always been interesting to me, I just have no clue how one would even get started with it, or who would even pay me to work on it.

kupopuffs2• 4 years ago

Well, continue classes and your studying. Have you implemented semaphores?

ensan2• 4 years ago

What does implementing semaphores have to with this?

When it comes to writing firmwares or an OS, nothing can substitute for hands-on experience with actual hardware.

Galanwe• 4 years ago

The easiest way to get started is to just write a driver for whatever simple device you have.

It may be easier if you do some electronics as a hobby since you can create the devices yourself with simple enough protocols. It gives you a good understanding of how the hardware works at a basic level.

There are thousands of tutorials out there on how to get started writing simple drivers, as well as the excellent book "Linux device drivers".

I don't do _complex_ drivers, but I've been able to write simple ones for my DIY electronic devices (keyboards, sensors, ...) and even a reverse engineered one for my BOSS RC5 guitar pedal.

Ballas• 4 years ago

I have written some audio drivers for my own hardware and I still feel lost every time I start with a driver. Admittedly less so than the first time, but if there were decent up-to-date documentation it would really help a lot. (Perhaps my problem is more specific to ALSA documentation)

adamnemecek• 4 years ago

You setup a table of functions that the HW invokes when things happen. These functions add some sort of event onto a queue. At some point in the future, the kernel processes these events. Rinse, repeat.

10000truths• 4 years ago

That, and MMIO. Lots and lots of mapping memory ranges/addresses to peripherals.

ensan2• 4 years ago

IMO the best way to get into this type of low-level tinkering is by writing a simple operating system.

https://github.com/isometimes/rpi4-osdev

There are other courses/projects for other boards. The keyword is usually “baremetal”.

For Linux drivers specifically there are training material from Bootlin etc. They definitely give you a kickstart—the kernel is so complex that most of the device driver knowledge is organizational and cannot be found on YouTube.

tkinom• 4 years ago

Start by tracing exist open source Intel/AMD driver in linux:

   Can be for old laptop (T420, etc with Intel GPU/Driver.) 
   Use FTRACE in kernel. 
   Setup ebpf trace for GPU acitivies.
   Write some doc/blog/medium pages on the process and show off your works.
   Understand, document and improve some opensource GPU API related utilities
   Understand and document interaction between GPU/GUI App and OpenSource driver.

In term of jobs: AMD has 289 opening for intern positions: https://jobs.amd.com/go/Internships-&-Co-op-Opportunities/25... A lot of them are graphic related.

numlock86• 4 years ago

Low level stuff has always been interesting to me, I just have no clue how one would even get started with it, or who would even pay me to work on it.

One keyword is embedded programming. You can start out bare metal basically doing everything in a while(1) loop until you get to a point where different tasks (in the literal way) and peripherals get so much that you will end up doing exactly this, but on a much smaller (and comprehensible) scale.

ckastner• 4 years ago

I also have a CS degree and felt the same black magic during my studies, but then I took some microcontroller classes, and they were eye-opening. In the sense that all of the CS operating system stuff suddenly made so much more sense.

For example, take something like (1) the simplest AVR microcontroller (the type used in Arduinos, hence there is a ton of third-party documentation and code floating around) and (2) your typical Hitachi HD44780 LED display.

To control the display, one needs to pull certain pins high and low, for specific intervals -- this is all in the HD44780's data sheet. Of course, you don't do this every time you send a character; you abstract this away into functions setchar(x, y, c), clearDisplay(), and so on.

Presto, your first driver!

Then you might think: let's print stuff that we receive via a serial connection. So how do you react to that data? Then you learn about interrupts and interrupt handling.

Then you might think: I could be doing stuff while I wait for I/O. Actually, I'd like to do a lot of stuff, in parallel, but the AVRs are single-core (so to speak). So you do what you've heard other OSes do: take available compute time, divide it into slices, and distribute slices to processes. And thus, you have learned about multi-tasking.

This was relatively simple to do on AVRs, and great fun.

chris37879• 4 years ago

Yeah, people don't realize that the operating systems we have today arise from devices about as powerful as some of these microcomputers. Removing the complexities that have been added to the x86 platform itself and being able to see "oh when this pin goes high cause I pressed that momentary button the CPU runs the code at this memory address" things can really start clicking. A great youtube creator I recommend for those types of explanations (he's making his own CPU out of base components) is Ben Eater and specifically his 6502 series, but the series on how USB keyboards work is also quite instructive for people looking at the hardware at that level of detail.

bitwize• 4 years ago

Try running NetBSD. You don't have to marry it, just put it on a spare machine or even in a VM.

The thing about NetBSD (and I can vouch for OpenBSD being much the same) is that it's so stinking well documented, if you go in knowing C you can learn how to write a simple device driver using just the man pages as your guide. Section 9 of the manual in particular documents every kernel API and most if not all internal types.

It's a great OS to practice developing low-level code with. Linux is a bit harder.

jancsika• 4 years ago

So fun to read!

Question wrt:

If Linux doesn’t know a clock or power domain is used by the GPU, it’ll turn off the GPU inadvertently.

If you know the name and ID of the GPU, is there a command from the terminal to tell Linux to turn the thing back on?

csense• 4 years ago

Why is it that closed source drivers are the norm rather than the exception for hardware companies?

People need to buy your hardware to use your drivers.

Meaning driver development is a cost. The revenue comes from hardware sales. You're usually offering the driver to the public for free on your website anyway. Why not simply open source it?

If you open source your driver, community developers could potentially give you free help -- e.g. porting to Linux, getting into the mainline kernel, fixing bugs.

rock_hard• 4 years ago

It’s a mindset issue

Sure, sometimes there is special sauce in the drivers too

But mostly it’s just culture. Hardware companies also don’t like cloud software…because they think on prem is safer or whatever

est31• 4 years ago

Any company that is going to negotiate with a cloud company, or that is going to compete with a cloud company, should avoid using any services from that cloud company. Otherwise you risk giving them access to strategy papers, or even internal communication about strategies or negotiation limts.

Cloud companies are some of the biggest customers of hardware companies like AMD. Eventually they will come to a point where they negotiate.

zeusk• 4 years ago

Because of two reasons

- driver can reveal internal workings of their “proprietary” hardware

- they use driver features to further differentiate their hardware product

But really it’s just old tradition and paranoia.

tlamponi• 4 years ago

- driver can reveal internal workings of their “proprietary” hardware

Every, somewhat solid, HW engineer from the competition will be able to still grasp those inner workings from a proprietary driver and the HW itself. In the end they cannot just copy (most) things over due to facing potential legal issues, no matter if the driver is foss or proprietary.

- they use driver features to further differentiate their hardware product But really it’s just old tradition and paranoia

Agreeing here, albeit AMD and Intel seem to have (somewhat) grasped that it doesn't need to be that way.

tyingq• 4 years ago

Patents you have licensed, but do not own. And so your lawyers advise you not to release any source code.

modeless• 4 years ago

And all the ones you didn't license too, because you don't want any patent trolls to find out you're using a technique vaguely similar to some patent you've never heard of. Aren't patents great?

sobkas3• 4 years ago

Why is it that closed source drivers are the norm rather than the exception for hardware companies?

Because when you want to depreciate your old hardware and want to push your customers to new one, open source driver that is maintained, or even worse, add features that should be exclusive to your new hardware, you have a problem. Planned obsolescence is the name of the game in modern hardware industry. And even "open source" drivers like mwlwifi show similar pattern, good look fixing their ¢¢^√°^¢^° firmware.

sobkas3• 4 years ago

Also when you depend on driver to differentiate between high/low end devices, open source driver is also no. Eg. Nvidia, throttling consumer devices in favour of enterprise grade ones.

ksec• 4 years ago

While the revenue comes from Hardware, Driver is actually the centre of value. Just look at the history of Graphics Card ( even before the term GPU was born ), there are plenty of decent graphics hardware that fail to compete simply because of poor drivers.

Of course this also means there is potential for a competitor to gain competitive advantage when they dont have to rely on GPU for revenue and profits generation. I expect or hope that to be Intel. ( That is ignoring GPU's potential patents issues )

foxfluff• 4 years ago

While the revenue comes from Hardware, Driver is actually the centre of value.

And IME it is the proprietary drivers that usually offer least value and most problems. That sways buying decisions.

franga20002• 4 years ago

Why is it that closed source drivers are the norm

Closed source is the norm, no matter if it's a hardware company's driver/companion app or a service company's client app. It's rarely economical, but usually "makes business sense" as in there's one more proprietary thing that you own and that increases your apparent value to investors. Some reasons I've heard:

- opening it would lower the barrier to entry for competitors (as if software is the big barrier) - it could be licensed to other companies in different industries (that almost never happens) - closed source is more secure and therefore less risky (a classic fallacy)

oaiey• 4 years ago

software is a gigantic barrier even when you put hardware next to it. maybe not for a highly sophisticated hardware like video cards, but generally, yes.

kanaffa12345• 4 years ago

if you're interested in this you might be interested in https://github.com/jbush001/NyuziProcessor

Writing an open source GPU driver without the hardware

Related Stories