Raspberry Pi 4: Hardware accelerated video decoding (GPU) in Chromium

https://lemariva.com/blog/rss/raspberry-pi-4-video-acceleration-decode-chromium

by zdw • 6 years ago

128 68 6 years ago

lemariva.com

knaik94• 6 years ago

Do we know what codecs have hardware acceleration support? I know the author mentioned h264

monocasa• 6 years ago

The rpi4 has hardware blocks for h.264, hevc, and vp9 IIRC.

Uehreka• 6 years ago

Ah! So this explains why trying to use a Raspberry Pi 3 to stream a live webcam feed over the network was so slow for me, it was all using software encoding.

CyberRabbi2• 6 years ago

This got me curious about how using xcompmgr causes flickering. My intuition would tell me the opposite would be true.

pengaru• 6 years ago

The core X protocol doesn't have any actual coordination of frames becoming ready to display. There's isn't even a concept of a frame or page at all.

There was an extension added in an attempt to fix this situation, IIRC it's the xdbe extension, but I don't believe it gained much traction.

If you look at the implementation of xcompmgr and the extensions introduced to make compositing possible, namely Composite and DAMAGE, they too don't have any frame-oriented coordination.

So it's just more racy uncoordinated capacitance in the immediate mode rendering pipeline of X. Your X requests logically just throw things on the screen as they occur. There may or may not be a compositor in the middle of that. When the compositor is scheduled and how much of your X requests sitting in the queue become processed as part of the same timeslice is fairly non-deterministic. There's unpredictable scheduling occurring at the kernel level, the compositor process's event handling, and the X display server's client multiplexing level.

There are some good reasons for a replacement like wayland, this area is definitely one of them.

CyberRabbi2• 6 years ago

Got it, so there’s no X client request along the lines of “the frame is fully drawn.” I assumed there was and that the compositor would take advantage of that, doing all the double buffering itself.

igravious• 6 years ago

Anybody have any idea why this isn't enabled out o the box?

cbhl• 6 years ago

I imagine screen tearing, especially under the default compositing window manager, would be unacceptable for the out-of-the-box configuration.

Another possibility is just that nobody has gotten to it yet.

jandrese• 6 years ago

The answer is probably that Broadcom's documentation is dogshit and this required a herculean effort to reverse engineer to get everything working.

908B64B1972• 6 years ago

That's something I've wondered for a long time: Doesn't Broadcom want users to be able to use the full feature range of it's silicon?

rektide3• 6 years ago

chromium has been resistant to enabling Linux acceleration until very recently.

https://bugs.chromium.org/p/chromium/issues/detail?id=463440

from the article, this seems to require setting up an advanced video card config on the pi side & allocating 128mb ram. that used to be a significant portion of ram. rpi3 came with 1gb.

WhaleBasketball• 6 years ago

looking forward to vulkan support on the pi4

squarefoot2• 6 years ago

What about Firefox? Video playback from Youtube on FF is plain terrible; 720p is barely acceptable in full screen for technical videos (forget movies) and 1080p stutters like a kangaroo with parkinson. The same videos play perfectly under LibreElec/Kodi, but due to recent crazy restriction by Google, one is forced to sign up with them to watch videos on YT Kodi addons, while browsers are still ok (for now). Also, Firefox (which I refuse to swap with anything) crashes like 1 in 3 times when I fast forward or backward the video. I keep the crash reporting on so FF devs are likely inundated by reports about the problem, which happens on a stock RPi4 with stock Raspbian and stock Firefox. All constantly updated.

MayeulC• 6 years ago

I haven't tried it on a pi,but mpv (having youtube-dl installed) is my to-go tool for watching youtube videos, since google wants me to sign in or accept its tracking to see them otherwise.

I haven't tried it on a pi, you might have to specify --hwdec or some variation of it to have really smooth playback, otherwise it's as simple as `mpv "url"`.

ComputerGuru• 6 years ago

I was surprised at how much better (more fluid, less stuttering, fewer artifacts) mpv was compared to VLC with hardware rendering enabled on Linux.

SturgeonsLaw• 6 years ago

You can also open Youtube URLs in VLC. Haven't tested the performance on a pi though.

squarefoot2• 6 years ago

Thanks, I'm currently testing Minitube (https://flavio.tordini.org/minitube/) which uses the MPV library and seems not to have some problems experienced with MPV (permissions, unrecognized format, etc). Still have to test it on the RPi though, so I have no idea how fast will do compared to the browser, but at least the crash problems would be solved. I'm afraid it won't last long, however: Google is locking access to more and more resources behind signup walls, which is a turn off for many people, including myself. Hopefully content creators will move elsewhere.

rektide3• 6 years ago

firefox 80 shipped late August & is the first with Linux gpu support under x. wayland was already supported, it seems, since 75.

81 is three weeks old & is the first release that will have Linux video gpu support enabled by default. source: https://9to5linux.com/firefox-81-enters-beta-gpu-acceleratio...

note that this is firefox at large. no idea whether the pi supports the standard va-api acceleration interface or what other potential blockers the pi has.

jancsika2• 6 years ago

Thanks for this comment.

Question-- how exactly does one check that gpu-acceleration is on in FF? I'm running nightly on a NUC with Intel Iris GPU and it seems to be faster than LTS from Debian Bullseye, but I'd like to be sure...

megous• 6 years ago

It just has libva support for video decoding acceleration.

Whether that translates to actual video decoding acceleration on any platform depends on availability of libva backends. And performance (or power consumption) may still suck, if you can't use the frame buffers directly for output to some HW overlay.

Linux kernel APIs are fairly straightforward, and easy to use, if you use them directly from some simple app. But making it work across all the abstractions from browser, to compositor, to display server, via ffmpeg, via libva, etc. is not easy at all.

MaxBarraclough• 6 years ago

How about VLC?

bufferoverflow• 6 years ago

It's 2020, the current year, and we're still having ridiculous problems with GPU not being used for decoding video by default. You know, the #1 use case - watching something on youtube or twitch or instagram.

sgt2• 6 years ago

I partly agree with you, but I doubt the #1 use case for someone buying a Raspberry Pi is to sit and watch YouTube or Twitch. People buy Raspberries to experiment with, to have a Linux "server" at home, or to connect it to something, whether it's a web cam or a sensor using the GPIO pins.

NegativeLatency• 6 years ago

I have a bunch of 3s, but I bought a 4 specifically for it's better specs so I'd have a better computer in my garage for looking up schematics/ordering parts/watching youtube how-to videos.

julianlam• 6 years ago

Chiming in as another person using an RPi 4 as a HTPC[1]. It runs YouTube/Netflix swimmingly and connects to a mythbackend server to watch TV and recorded shows. Essentially it's a home theater thin client.

[1] https://www.hackernoon.com/can-a-pi-4-replace-your-home-thea...

NortySpock2• 6 years ago

Linux "server"

I am slightly miffed at that. I'm running some docker containers and a Sambashare NAS on a Raspberry Pi, are you claiming what I have is not a server?

m4633• 6 years ago

I believe a pi with kodi is a common use case.

spacemanmatt2• 6 years ago

As I cannot trust any multiuser desktop system with sound (it will glitch if I use the computer for much else, I don't care what OS/hardware) I'm mulling a short stack of R.Pi for reliable stream captures. I'm not that voracious but my absolute favorite streams that I really want to time-shift come on at the same time.

intricatedetail• 6 years ago

This is done to trick people into buying more expensive hardware. I also wonder what motives Google has with disabling GPU acceleration on Linux.

deanclatworthy• 6 years ago

Yep. I can watch YouTube no problem on my top of the line 2019 MacBook Pro, presumably it’s hardware accelerated but when I load a Twitch stream it sounds like a jet engine.

kzrdude3• 6 years ago

I just got an rpi4 and it's cool to see what it's about. Probably going to use it as a media center for the tv, and for games (anyone got tips for good games that play with a controller?)

Shared4042• 6 years ago

anyone got tips for good games that play with a controller?

RetroPi + anything from 2D Nintendo consoles, I'm personally a fan of the GBA metroid games. Some other good ones are Fire Emblem, A Link to the Past, Final Fantasy, Mario (1/2/3/World). If you email me, I can help with sourcing ROM's.

As for more modern games, Hollow Knight is a masterpiece. A SNES controller has _almost_ enough buttons, but I had to remap one control to the spacebar.

kzrdude3• 6 years ago

Thanks for the cool tips, I'll check it out. Hollow Knight seems impossible on the raspberry pi, from brief research, but maybe there's a way if I can find one.

lioeters• 6 years ago

If you're into retro/lofi/handmade games, PICO-8 on Raspberry Pi might be fun:

https://www.lexaloffle.com/pico-8.php

kzrdude3• 6 years ago

Thanks a lot for the tip, will check it out!

kusti8• 6 years ago

I was pretty active in the Raspberry Pi community around 4 years ago. Playing YouTube easily was a big problem and I tried making a lot of packages to solve this. I would also build Chromium to add widevine support for DRM content.

I also helped the Raspberry Pi Foundation around 4 years ago building and packaging their patched version of Chromium. What they essentially did was directly modify the Chromium source to add HW decoding specifically for the Pi with MMAL. Then, an extension would be auto installed to enable H264 on YouTube instead of VP9. This was released back when they launched the Pixel desktop.

This obviously had some problems, because the patch needed to be manually changed for every release of Chromium, which come pretty often. It couldn't also be easily upstreamed. Looking at it now, it seems it hasn't been updated in about a year, and I'm curious what the Foundation's roadmap for this is in the future, after having been out of the loop for so long.

CameronNemo• 6 years ago

FWIW Rockchip video decode/encode is not in a much better space.

There is the BSP, id est Rockchip's hacked together fork of the 4.4 kernel, which works. But on the mainline kernel h264 is just now getting added (maybe in 5.10), and userspace is still going to take some time to adapt.

I really hope that one day I can actually use the Pinebook Pro webcam for video chats without my CPU spinning while trying to encode (at least) one stream and decode N streams.

simcop2387• 6 years ago

It also doesn't help that youtube has almost complete dropped h264 these days either, every new video I've seen has been vp9 and av1 only lately.

microcolonel• 6 years ago

youtube has almost complete dropped h264 these days either

Only really true for VOD, and only really true for channels with 300k+ subs.

m4633• 6 years ago

There must be h264 version available somehow for devices that don't do vp9/avi right?

Raspberry Pi 4: Hardware accelerated video decoding (GPU) in Chromium

Related Stories