Qualcomm is adding AV1 support, which could be huge for online video

https://www.protocol.com/newsletters/entertainment/av1-open-video-codec

by clouddrover • 4 years ago

204 91 4 years ago

shmerl5• 4 years ago

Will RDNA3 cards have AV1 encoder? RDNA2 ones got the decoder at least. With encoders in place, video calls can start using AV1 too.

kbumsik2• 4 years ago

AFAIK, none of commercially-available hardware supports AV1 encode yet.

I guess it is quite expensive to implement AV1 encoder.

Edit: I meant no consumer hardware support it.

shmerl5• 4 years ago

Well, RDNA3 GPUs aren't available yet either. But at some point encoders should become viable I suppose.

wmf• 4 years ago

Intel's upcoming DG2 has AV1 encode. https://videocardz.com/newz/intel-confirms-arctic-sound-m-da...

klodolph• 4 years ago

I wonder what the timeline is from “encoder appears in consumer chip” to “people use it for video calls”.

SoapSeller• 4 years ago

Nvidia Orin(the SOC, successor to Xavier) that coming out this Q will have AV1 encoding(up to two streams of 4K60).

One can assume next gen nvidia gpus(rumored for the end of this year) will have it as well.

Dylan168072• 4 years ago

On the desktop you can probably spare a core or two for encoding just fine.

shmerl5• 4 years ago

True, but it can still be pretty CPU intensive. Plus on laptops it's a bigger deal.

LinAGKar3• 4 years ago

A core or two is not enough for high quality real-time encoding.

8K832d7tNmiQ• 4 years ago

There is no additional benefit to use AV1 for video calls since VP9 can do that just enough for the next 5 years.

shmerl5• 4 years ago

Isn't it supposed to have lower bitrate for the same quality? If there wouldn't have been any benefits they wouldn't have developed a whole new codec.

kmfrk• 4 years ago

We only just got AV1 decoding on the latest gen of GPUs, it might be too optimistic to hope for encoding in the next one.

In 2019, Twitch predicted roll-out of AV1 support by 2024, but then the pandemic happened: https://www.streamingmedia.com/Articles/ReadArticle.aspx?Art...:

For head content, it's okay to streaming multiple formats because the viewership is huge, so streaming multiple formats, although increase our cost, it still actually save our traffic cost, so it's still worth while. But for the tail content, it's very different. We can only afford streaming one single format, so our strategy is currently still doing H.264 using hardware, high-density hardware solution, but we're hoping towards 2024, 2025, the AV1 ecosystem is ready, we want to switch to AV1 100%.
Jan Ozer: Did you say 2024 and 2025?
Yueshi Shen: 2024, this is our projection right now. But on the other hand, so as I said, our AV1 release will be, for the head content will be a lot sooner, we are hoping 2022-2023 we are going to release AV1 for the head content. But for the head content, we will continue to stream dual-format, AV1, H.264. But for the tail content, we are hoping towards five years from now to AV1, whole eco, every five-year-old device supports AV1. Then, we will be switching to AV1 100%.

The initial roll-out will probably just involve transcoding at sub-source quality settings to save bandwidth with no encoding to be done by the streamer.

Right now, platforms have only talked about AV1 for transcoding purposes, so there's not the same urgency to add encoding support on consumer-level GPUs. Even thought it'd be amazing to see AV1 streams and YouTube videos, it's mostly on the server-end to cut costs and make things easier on people with data plans and cellular.

The_rationalist• 4 years ago

Energy consumption of HW accelerated AV1 decoding vs h265?

ZeroGravitas4• 4 years ago

Sofware decode is broadly comparable so should be about the same.

Later generations often have more tools, which can lead to bigger die size, but this can also lead to energy gains from those tools depending on content/usage.

See cisco's presentation about adding AV1 to Webex and how it could outperform H264 in this niche due to having specific tools for it.

In Netflix type content the grain emulation for example could make a big difference.

viktorcode• 4 years ago

Note, no support for hardware encoding was announced by Qualcomm. Decoding only. This seems to repeat the trajectory of VP9, where it exists and is supported solely thanks to YouTube, while the whole world is utilising H.265 for everything else, including recording video on smartphones.

It very well may be AV1 and H.266 will play the same roles in the future.

ksec2• 4 years ago

It will be interesting to see if Google will include AV1 encoder in their Pixel Phone and eventually give away hardware AV1 encoder IP.

But Google has never been a coherent company.

aesh2Xa12• 4 years ago

What does it take to integrate an encoder into hardware? Could you explain?

AV1 is an open, royalty-free codec, so I don't understand how Google could help competitors.

Maybe Google is in a place to give away knowledge, although I doubt they know more about a competitor's chip design.

ZeroGravitas4• 4 years ago

Google bought a chip design company and has been releasing codec chip designs for years, probably during the VP8 era but certainly VP9. It's now at the stage when they don't really need to it mostly just happens.

axiolite• 4 years ago

This seems to repeat the trajectory of VP9, where it exists and is supported solely thanks to YouTube, while the whole world is utilising H.265 for everything else, including recording video

VP9 came too close on the heels of VP8. Thanks to WebRTC in particular, VP8 encoding support is everywhere. AV1 has a lot more backing, thanks in no small part to the H.265 licensing mess.

threeseed6• 4 years ago

VP8 encoding support is everywhere

At least for iOS/OSX it is only supported for WebRTC and nothing else.

ZeroGravitas4• 4 years ago

the whole world is utilising H.265 for everything else.

This is very hyerbolic. There's plenty of places that the HEVC licencing catastrophe stopped its use. The release of 3 more MPEG codecs that compete with it is not a good sign for example.

BluSyn• 4 years ago

I'm hoping future Apple chips have AV1 decode support as well. If Intel, Google, Qualcomm, and Apple all add AV1 support in hardware, it would become the gold standard as far as codecs go. Right now it's all a mixed bag.

ChuckNorris89• 4 years ago

Considering Apple's love for proprietary everything and their large market share, I wouldn't be surprised to see them roll out their own codec in SW and HW, same how they developed Metal instead of using the open Vulkan.

threeseed6• 4 years ago

a) Apple has always been an active part of the MPEG standards group for decades so not sure why they would roll their own.

b) CoreGraphics and CoreAnimation are underpinned by Metal. If you were building an OS that was going to be used by every device you make (computer, phone, tablet, monitor, watch, TV streamer, headset) would you rely on an open source project that you don't control and may have different values, wishes, roadmap etc than you.

kalleboo• 4 years ago

Apple wouldn’t do their own video codec, they’d adopt VVC[0], which is what looks like what will become the standard for non-web video (e.g. DVB over the air broadcasts in Europe) [0] https://en.m.wikipedia.org/wiki/Versatile_Video_Coding

trissylegs• 4 years ago

Although apple is a founding member of AOMedia so they might be on board with AV1 https://aomedia.org/membership/members/

bla32• 4 years ago

Isn't Apple ProRes exactly this?

(I don't know codes well, this is an honest question.)

rched2• 4 years ago

I don't think the Vulcan/Metal comparison is useful here. Metal shipped before Vulcan so it was never an option for Apple. The case of AV1 is different.

stingraycharles• 4 years ago

Out of curiosity, what is the big difference of AV1 vs h264 and HEVC / x265 ?

As someone who mainly cares about blu-ray / large movies, I was under the impression that HEVC was becoming the gold standard. Is that not the case, and/or is AV1 the next iteration, and/or do they solve different problems?

LeoPanthera• 4 years ago

The only significant difference is patents and licensing. In terms of actual compression, HEVC has slightly better efficiency.

AV1 is supposedly a "free" format, but I wouldn't bet my business on it. The patent situation is unclear. https://en.wikipedia.org/wiki/AV1#Patent_claims

jug• 4 years ago

I unfortunately expect, given tradition, that the next move for Apple will be to H.266 / VVC in hardware, not AV1.

VVC was finalized in 2020 and there is some influence backing it already where although DVB will evaluate AV1, they've already hopped on VVC.

ZeroGravitas4• 4 years ago

Uptake of the open-video codec AV1 has been slow, with major video providers waiting for broader device support.

The people involved in AV1 had a software decode plan B that has worked better than most of them hoped.

If you combine that with encoding choice based on software decode impact, most big players have started with software rollouts, but still seen gains.

"You need hardware support for a codec" is often the same kind of delaying tactic as "renewables need storage". You do both in parallel, not enter an infinite loop where you don't do either because the other isn't complete yet.

tarsinge• 4 years ago

Hardware support for a codec is a business decision decision, it doesn’t require a tech/physics breakthrough as renewables storage at city and country scale does, it’s not remotely comparable.

asdfasgasdgasdg2• 4 years ago

Renewable storage is also a business decision? Like we know how to build enough storage, it's just very expensive using current technology. In the limit, you can just make a lot of these:

https://e360.yale.edu/features/in-boost-for-renewables-grid-...

pjc50• 4 years ago

renewables storage at city and country scale does

That's also a business decision, just with a bigger number attached. And hitherto it's been far cheaper to use gas. But that's now a "conflict mineral".

phkahler• 4 years ago

This is just lazy and risk-averse management. With the list of companies behind AV1 is certainly looked like the codec of the future several years ago. While predicting the future can be dicey, switching to AV1 should have been fairly obvious. Everyone waiting to see if it will actually take off has simply delayed the takeoff. Software decode has been around for years, so content providers could have encoded and used that for desktop users, or even people on mobile - half of them will blame their devices when the battery doesn't last.

basilgohar• 4 years ago

The entrenched interests are adverse to wide adoption of a royalty-free codec. Even some of the parties involved with AV1 have a conflict of interest as they are patent holders for other technologies that would exclude AV1's wider usage.

GeekyBear• 4 years ago

switching to AV1 should have been fairly obvious

Things are moving, but not quickly.

VLC shipped an optimized x86 software decoder very recently, but Handbrake has yet to find an open source software encoder in good enough shape that they can add it for general consumer use.

https://github.com/HandBrake/HandBrake/issues/457

rdsubhas• 4 years ago

This looks quite big for battery life. Any idea how much this would improve battery life (assuming a mobile device is continuously playing video?)

gjsman-1000• 4 years ago

AV1 has had several issues plaguing its uptake (as a close observer of the technology):

- The reference silicon design for an AV1 decoder was very space-inefficient, and so many chip vendors refused to use it. This, however, necessitated the development of internal or 3rd-party decoding IP which slowed uptake considerably.

- AV1, partially due to it's patent-free design, has an extremely complicated compression process. So complicated, that even in software it is still a few dozen times slower than H.264 encoding. This, plus the first issue mentioned above, means on-chip AV1 encoding is a ways away.

- AV1, despite being "patent-free," has had a patent troll (Sisvel) come along and claim that they have a bunch of patents applying to it. They want 20 cents per device. AV1's founding group claims to have a legal defense fund for members but the terms and coverage are unclear. Sisvel wants 20 cents per device using AV1, so now you need to get the lawyers involved to figure out whether to bet on paying Sisvel or bet on taking the AV1 founder's view of the matter.

- Finally... there's AV2 coming. Though a ways away, it will use recently-expired patents to further help its compression capabilities, new techniques that couldn't be completed on time for AV1, and will feature a more optimized reference silicon implementation that might be actually easily usable by chip designers this time.

chasil• 4 years ago

The wiki mentions that the original reference encoder was not optimized, and did not perform well:

https://en.wikipedia.org/wiki/AV1#History

"While still working on the format, the encoder was not targeted for production use and speed optimizations were not prioritized. Consequently, the early version of AV1 was orders of magnitude slower than existing HEVC encoders. Much of the development effort was consequently shifted towards maturing the reference encoder. In March 2019, it was reported that the speed of the reference encoder had improved greatly and within the same order of magnitude as encoders for other common formats."

Other encoders also exist that are much faster, listed in a different section of the wiki.

https://en.wikipedia.org/wiki/AV1#Software_implementations

I didn't know about Sisvel and the hardware design problems, although Sisvel is in the wiki. The specific license fees that you listed are interesting.

https://en.wikipedia.org/wiki/AV1#Patent_claims

Qualcomm is adding AV1 support, which could be huge for online video

Related Stories