Khronos Releases AV1 Decode in Vulkan Video with SDK Support for H.264/H.265

https://www.khronos.org/blog/khronos-releases-vulkan-video-av1-decode-extension-vulkan-sdk-now-supports-h.264-h.265-encode

by doener • 2 years ago

164 66 2 years ago

khronos.org

ekianjo• 2 years ago

AMD not even ready for the Linux support yet? that's disappointing.

jauntywundrkind2• 2 years ago

Not sure why this release is so slanted, but av1 encode on AMD Linux has been available since October. https://www.phoronix.com/news/Mesa-23.3-HQ-AV1-Radeon

How Nvidia managed to emerge looking good on this PR is a colossal surprise & fucked up-ed-ly inappropriate.

shmerl2• 2 years ago

I suppose the above comment was about amdvlk. radv is indeed often ahead of it these days.

If anything, I think AMD should drop amdvlk on Linux (to collaborate on radv there) and instead try to develop radv for Windows if that's doable to replace amdvlk altogether.

aaomidi• 2 years ago

And meanwhile Chrome outright refuses to support vaapi outside of intel (and thats only because of chromebooks).

rasz• 2 years ago

AMD AV1 encoder is really bad, you arent missing much.

https://goughlui.com/2023/12/31/video-codec-round-up-2023-pa...

https://goughlui.com/2024/01/07/video-codec-round-up-2023-pa...

emersion• 2 years ago

https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27...

janandonly• 2 years ago

Too bad there is no mentioning of MacOS support.

maxhille• 2 years ago

AFAIK Apple does not support Vulkan

Tmpod• 2 years ago

Yeah it doesn't. But there is MoltenVK[0], which translates to Apple's Metal.

[0] https://moltengl.com/moltenvk

cm21875• 2 years ago

How useful are hardware decoders? My understanding is that they are not necessarily faster than software decoders (in my experience it is the opposite), so the only benefit I can see is less CPU consumption. I see the benefit when you run on a battery (laptop / smartphone) but on a desktop or server, unless you critically need the CPU at that time, I am not sure how useful it is.

Vogtinator2• 2 years ago

That's exactly what makes them useful. Energy consumption is vastly reduced.

FirmwareBurner3• 2 years ago

I see the benefit when you run on a battery (laptop / smartphone) but on a desktop or server, unless you critically need the CPU at that time, I am not sure how useful it is.

Yes it is. Very useful. The vast majority of people use laptops now, not desktop computers with 120mm tower coolers anymore, and if you want to switch to Linux then not having video decode nuke your battery life and turn your tiny fans into a Pratt & Whitney, then this is another welcome change to closing the gap to Windows.

People, especially on the Linux side, need to stop viewing the world as if everyone still has desktop towers PCs tucked under their desks while disregarding the software issues and deficiencies that only plague laptops, which are the majority of the marketshare and have vastly different constrains than desktops.

Dalewyn5• 2 years ago

Software video decoding isn't some monstrous workload, though. We're not in the era of Pentium 4 and h.264 decoding drowning every CPU out there to a screeching halt anymore, the CPUs of today hardly acknowledge the load even in the low-end tiers.

The benefits of hardware decoding are lower power consumption (read: electricity bills) and offloading the work to dedicated co-processors freeing up resources on the CPU proper, which are indeed far less relevant on desktops and even laptops which can just pack big batteries.

The biggest beneficiaries are by far mobile devices and really low power hardware. Computers which either cannot practically carry more batteries or which use hardware that simply aren't performant for more important reasons.

Desktops and laptops meanwhile are better geared to benefit from the perks of software decoding: Better quality and versatility, and fine-tuning of filters and rendering paths.

jorvi• 2 years ago

People, especially on the Linux side, need to stop viewing the world as if everyone still has desktop towers PCs tucked under their desks while disregarding the software issues and deficiencies that only plague laptops, which are the majority of the marketshare and have vastly different constrains than desktops.

Linux (especially the kernel) isn’t even tuned for towers. It’s tuned for servers.

The defaults make horrible optimizations all in the name of throughput > latency.

ddalex2• 2 years ago

I can tell you from my Chromebook and Macbook experience, HW decoders and encoders are the difference between a productive remote video meetup with good collaboration, and a frustrating lagged glitchy suttering meet with no work done, if CPU only.

Havoc• 2 years ago

They're usually substantially faster. Many low end devices can't do decode on cpu at all well enough to say play YT. Add hw decoder and magic.

Re desktop use - even there the energy efficiency is a win, especially for continuous use. e.g. I've always got YT open

kiririn• 2 years ago

Software decoding 4K x265 requires insane resources. You could live transcode it to 1080p quite reasonably on 1 average core, but simply decoding it requires ~3 average cores

ddalex2• 2 years ago

how come decoding is more work intensive vs transcoding ?

rollcat• 2 years ago

For encoding, it's like riding a bus vs pushing a bus.

AWS sells a $100k 1u encoder box to broadcast companies. They buy that because there's hardly a 2nd best to speak of.

Encoding 15min of 8k source material on i9 9900k (that was 2019 and THE fastest CPU on the market) took me 19h.

Same for decoding, any mobile device will drink battery like it's water on Sahara.

Video is unusable without HW acceleration.

cm21875• 2 years ago

Encoding is very different. Most codecs are designed to be slow to encode, fast to decode. HW encoding is day and night vs software. But I see ffmpeg mentioned in the post. Switching to hardware decoding is likely to slow down your pipeline.

rowanG077• 2 years ago

Asahi linux is a good example. There is no hw acceleration support but video decoding is working pretty flawlessly. Of course at the expense of battery life

BLanen• 2 years ago

Sorry if I'm misunderstanding your comment's implication. But this is not hardware decoding. This is software decoding on the GPU.

NVIDIA for example has specialized (en/de)-coding hardware on their GPUs, which is different. And even that works together with their GPGPU Cuda cores.

bno1• 2 years ago

No, the entire point of those vulkan extensions is to use the hardware-accelerated video processing units. You can use computer shaders for software decoding on the GPU, there is no need for special extensions to do that. AMD and Intel have specialized encoding/decoding hw too.

vetinari• 2 years ago

All three big vendors (Nvidia, AMD, Intel) and also bunch of ARM SoC vendors do have specialized blocks in their graphic solution whose only purpose is to decode and encode specific codec profiles. It is hardware decoding, not decoding on GPU.

fulafel2• 2 years ago

Yes, real-time decoding on a desktop is just nice-to-have, with a couple of exceptions

(1) if your decoder software runs too slow for real-time decoding. I think it historically has happened every once in a while until CPUs and decoder optimizations catch up again. Eg 4k H.265 was too much for many desktops for a while.

(2) you do a lot of it, in which low power decoding->saving electricity is still good

eg Linux web browsers for ages had sw-only decoding at it worked mostly fine, just laptops having battery eaten in browser apps & electron based apps (videoconferencing).

As codecs are historically a top source of remote code execution vulnerabilities that get our device pwned, the poor sandboxability of hw decoders makes them a security problem as well. eg as a pathway to exploit the flaky and high-privileged GPU driver code paths.

cubefox• 2 years ago

Does this mean they decode AV1 using a compute shader as a fallback for GPUs that don't include an AV1 ASIC?

zamadatix2• 2 years ago

That's up to each driver to decide. Support for the extension isn't mandatory, it's just meant to expose the hardware on GPUs.

fulafel2• 2 years ago

There's no shader fallback included, it's just a extension spec for the accelerator interface.

Khronos Releases AV1 Decode in Vulkan Video with SDK Support for H.264/H.265

Related Stories