Modern codecs like AV1 can bring better quality video to the open web

https://blog.mozilla.org/blog/2018/07/11/royalty-free-web-video-codecs/

by ALee • 8 years ago

262 127 8 years ago

blog.mozilla.org

orastor• 8 years ago

I hope this takes over HEVC and x264 in the near future!

waivek• 8 years ago

Is AV1 smaller?

matt40772• 8 years ago

Yes, by about 50% and 30%, respectively. See the graph in the article.

BLanen• 8 years ago

x264 is software library. H264 is the spec and format.

gallerdude• 8 years ago

The thing I'm most excited about for AV1 is film grain synthesis. I can't articulate why, but I really love the look of film grain in video. Unfortunately, on most codecs, film grain is a huge hassle which takes up a lot of bandwidth. But AV1 can automatically remove the grain, encode, and add it back in.

tambourine_man2• 8 years ago

That’s amazing. I use grain to disguise compression all the time and always this should be possible.

paavoova2• 8 years ago

Why not keep the bitrate higher? Or are you re-encoding already over-compressed video? In which case re-encoding and adding noise only reduces the amount of detail and bloats the encoded size with, well, noise.

bscphil4• 8 years ago

Curious about this. Does the grain it algorithmically adds back look the same as that it takes away? In other words, is it just a better way to compress existing film grain? Different film sources have extremely different grain profiles, and people who encode movies (both production companies and pirates) will have no interest in the aspects of a codec that remove the real grain in a film to replace it with a generic alternative. I suppose they might adopt AV1 with that feature turned off, but otherwise AV1 will mostly be adopted for web video which currently doesn't have grain due to low bitrate encoding.

tfha4• 8 years ago

Depends how good you are with an encoder. It's not compression, the grain does change, but since it's just random anyway, the human eye can't notice so long as it's the same type of grain.

Encoders generally have a few options to tune the grain that gets added back in to get it as close as possible to the original source.

paavoova2• 8 years ago

Is this similar to DNR schemes? Because that's a lossy process that reduces find detail, and is notorious for having been misused in many bluray releases of analog film remasters.

And adding digital grain "back in" doesn't sound very authentic to the original analog source.

sjwright2• 8 years ago

Film grain is so expensive to encode that in bit-rate-constrained situations you would be forced to choose between obvious ugly compression artefacts, a reasonably good image without grain, or a reasonably good image overlaid with inauthentic grain.

Besides, "authentic" is such an overhyped pile of poo anyway. Everything is a reproduction; what matters should be that the artistic intent remains unaltered.

jacobush• 8 years ago

I am probably kind of crazy, but I love grain and worry when movies lose it. Like the fine old "Predator" movie on Bluray. Nice colors, but the grain is kind of gone. The DVD has more of it, and it looks better.

I feel a ramble coming on, but I believe movies should be available in a version close to how they were viewed on theatrical release.

If you think about it, the first 100 years of this artform is always going to be special.

Veedrac• 8 years ago

Truncated graphs are really misleading; I ended up doing a double-take. There are more honest ways to compress the height of a graph if vertical space is at a premium, like showing %savings.

justinjlynn• 8 years ago

Or using log charts, if you're required to use given units.

ksec3• 8 years ago

I think Mozilla should advocate for patents free Software or get rid of Software Patents completely.

The graph starts at 50%?

And they didn't mention in the Moscow State University test, AV1 was 300x to 1000x slower.

While I like competition from AV1, whatever they are doing in terms of marketing really doesn't resonate with me.

semicolon_storm• 8 years ago

And they didn't mention in the Moscow State University test, AV1 was 300x to 1000x slower.

Reference implementations of codecs are always slower. AV1 hasn't had the benefit of years of optimizations yet, so this slowness is to be expected.

okket• 8 years ago

If the codec can not be implemented efficiently on existing battery powered handheld devices, it is pretty useless. That is the only metric that counts. Even if, it will take many years to a decade to spread, considering the replacement rates of those devices.

CharlesW• 8 years ago

The article also incorrectly calls AV1 "patent free".

It's royalty-free, but what that really means is that contributors are licensing their AV1-connected patents to everyone for free. That's all fine and good as long as AV1 isn't found to infringe on patents owned by non-AOM members.

threeseed6• 8 years ago

AV1 is almost guaranteed to infringe on patents too.

VP8/VP9 both were patent encumbered and required licensing from MPEG-LA. It's going to be interesting to see what happens if AV1 does take off whether the big MPEG-LA patent contributors decide to go for royalties.

LeoPanthera• 8 years ago

Last time I checked (very unscientific test using Handbrake) the AV1 encoder was something like 10x slower than the hevc encoder on the same system. Is this likely to improve?

ysleepy• 8 years ago

It did with H265, the prototype implementations were horribly slow.

threeseed6• 8 years ago

Except that "The AV1 reference encoder is [as of today] a hundred times slower than an HEVC one"

https://www.ibc.org/delivery/codec-wars-the-battle-between-h...

matt40772• 8 years ago

This is dangerously close to becoming a standard talking point nthis topic. It’s dangerous because it’ll be wrong as soon as anyone implements an encoder with speed as an objective, and especially when dedicated hardware starts to appear in devices (about 12 to 18 months from now)

vortico3• 8 years ago

Yes, currently there is no hardware encoder/decoder for AV1, but there is for h264 on many processors. This may change in the future if companies and users adopt it.

bscphil4• 8 years ago

I might be mistaken, but I believe most high quality encodes in both h264 / hevc are not done using dedicated hardware encoders built into processors or graphics cards. While it's much faster there's usually some quality tradeoff. Instead, I believe that newer processors have additional instruction sets that enable much faster software encoding on the same codecs. Someone correct me if I'm wrong about this.

mehrdadn2• 8 years ago

Yes, currently there is no hardware encoder/decoder for AV1, but there is for h264 on many processors. This may change in the future if companies and users adopt it.

Does x86/x64 have any? What's the name of one of the instructions (for example)? I feel like mostly all I see is just bigger vector instructions...

shmerl2• 8 years ago

How is the progress of the encoder? To be able to use it in real time it must be very efficient.

snvzz2• 8 years ago

Mozilla is working on a performance-optimized encoder, and has demonstrated realtime encoding recently on a lots-of-cores cpu.

NelsonMinar4• 8 years ago

I've been using a lot of H.265 (aka x265/HEVC); some pirate scene TV and movie releases come out in it. It's fantastic. About 1/3-1/4 the size of H.264, which makes a huge difference both with download and with archiving. The downside is that there's not a lot of hardware support for encoding or decoding. Doing it on the CPU isn't bad but definitely not as energy efficient.

The problem with H.265 is the IP situation is a mess. AV1 looks to be better in every way. Looking forward to its adoption. So far the pirate scene doesn't seem to be using it at all.

CryoLogic• 8 years ago

The biggest downside is not energy efficiency IMO, it's speed. The speed difference between h.264 and h.265 encoding can be easily 10 fold. More-so on CPU reliant servers without GPUs. There is a big upfront sacrifice to get the small video size.

xxpor• 8 years ago

Even for decoding, you really need hardware acceleration. Trying to decode H.265 on one of these ARM based TV media players is an exercise in futility unless you have HW.

zamadatix3• 8 years ago

Pascal GPUs h265 encoding is 50% or higher than h264 encoding speeds. A single GTX 1080 can nvenc h265 8k@30 10bpp. A 150 dollar GPU will outperform a 3,000 dollar CPU only server in encoding (regardless if it's h264 or 265).

The REAL blockers are licensing and, more importantly, legacy consumer hardware. Whether you want to call the latter a speed or power problem isn't the real issue, it's that not all consumers have the dedicated hardware and the older CPU then has the speed and power problems.

flyGuyOnTheSly• 8 years ago

There is a big upfront sacrifice to get the small video size.

Still, the benefit to the millions of people who might download and watch a video surely outweighs the cost of the one individual computer that spent 10x the amount of CPU cycles encoding it.

skrowl• 8 years ago

The biggest downside is that your phone / tablet / etc can hardware decode h.264, and has to run h.265 or av1 decoding in software. As a result, your battery life gets killed when watching h.265 / av1.

userbinator• 8 years ago

So far the pirate scene doesn't seem to be using it at all.

Pirates, quite understandably, don't care about patents or IP. They can be a good indicator of whether there is any other reason to use AV1 over H.265.

pas• 8 years ago

AV1 is not simply ready. The specification is finished, but the encoder-decoder is not yet optimized. (It's considered experimental by ffmpeg for example. https://trac.ffmpeg.org/wiki/Encode/AV1 )

There's development happening here: https://aomedia.googlesource.com/aom/+log/refs/heads/master/...

puzzlingcaptcha2• 8 years ago

In one of the tests rav1e encoded 1080p source at 0.013fps (i5-7600K @5GHz). That's half a year for a 2 hour movie.

Private torrent communities are already reluctant to use x265 which provides marginal gains compared to x264 at a great increase in encoding time.

A part of the process in pirate movie encoding (p2p, not really scene) is testing optimal encoder parameters (adaptive quantization strength, psychovisual rate-distortion, quantizer curve compression etc) on short clips meant to be representative of the whole movie (e.g. 60 frames every 4000 frames). If encoding a single test clip takes an hour as with x265 that sure puts a damper on things.

About the only place in movie torrent communities where x265 makes sense is 4K encodes as x265 can retain HDR metadata (h264 can too to an extent with the 2017 revision to T-REC H264 but it's hardly supported anywhere).

This may be different for pirate _streaming_ services.

zamadatix3• 8 years ago

I think you're eating into quality loss if you're getting 33%-25% size, most every benchmark puts it at about 50%-60% size for equal quality to h264.

Pirating scene hasn't been using av1 at all because the bitstream hadn't been stabilized (an encode you made last month probably won't work on versions from this month) and, as a result of work still being done on the bitstream, no encoding hardware or software optimizations are available making it on the order of 3 magnitudes slower than existing options. Towards the start of 2019 you should start to see more hardware support pop up though.

vanderZwan• 8 years ago

I think you're eating into quality loss if you're getting 33%-25% size, most every benchmark puts it at about 50%-60% size for equal quality to h264.

Do the benchmarks include animation? Perhaps flat surfaces like one sees in anime compress a lot better than average in x265?

um_ya• 8 years ago

Does AV1 do subtitles? The reason most pirated videos use MKV instead of mp4 is because it does subtitles easily.

NelsonMinar4• 8 years ago

Thanks for correcting me on quality. I'm getting confused because often the H.265 releases have low bitrate 2.0 audio compared to high quality 5.1 in the bigger file releases. They're optimizing for size, not fidelity. (I'm blessed with a tin ear and tin eye.)

baddox• 8 years ago

Hardware support for decoding should be pretty good these days. Any TV or streaming box that supports 4k streaming should support HEVC, since AFAIK all the streaming sources use it (except for YouTube, last I checked). I suppose if you’re streaming on a laptop you might be stuck with software decoding.

I’m also very impressed with HEVC, but mostly from the angle of UHD Blu-rays. The quality is remarkable. For the same size or slightly larger than 1080p Blu-rays (around 50GB, sometimes up to 70), you get 4k resolution (4 times the pixels as 1080p) and HDR.

IntelMiner3• 8 years ago

Where are you finding HEVC encoded pirate content?

I railed against "scene" groups for many years over the quality (or lack thereof) in releases they put out. To the point I simply started buying the media I wanted, ripping it myself and then converting it to HEVC

Doing as close to a 1:1 DVD rip as I can, I've shrunk things like Seinfeld from 152GB of MPEG-2 video to just 31GB in HEVC, and it still looks absolutely fantastic

HD video fares even better. The X-Files on Blu Ray is roughly 1.7 terabytes. HEVC encoded brings it down to 240GB or so

crtasm3• 8 years ago

Quick look at http://m.thepiratebay.org/search/x265/0/0/0 returns a huge number of results.

NelsonMinar4• 8 years ago

PSA Rips is a good place to look if you're curious about how H.265 is used in practice. (Note the 10 bit video.) Their stuff tends to be tagged "hevc-psa". Here's an article they wrote about encoding quality: https://psarips.com/why-psas-releases-are-sometimes-late/

ksec3• 8 years ago

The problem with H.265 is the IP situation is a mess.

I really wish the next gen codec, H.266 VVC could sort this royalty mess out before hand.

Why cant we keep things simple. USD $1 Per devices Royalty on Hardware encode and decode, that is roughly $3B + Royalty every year for the next 20 years.

Software Implementation should be free. So an H.266, VVC based image and video decoder could be immediately rolled out to all PC, and browser vendor could support it.

One of the problem with current MPEG codec is that they are too worry about free software implementation take over their royalty. Well we don't have huge increase in IPC anymore. We have a roadmap from TSMC that shows us the next 10 years of leading edge transistor is going to be more expensive then ever. We don't have any breakthrough / theory on hand they we could make those a lot cheaper. The days of Moore's Law are long gone.

This allow hardware maker to sell "faster" video and picture decoding. Consumers are a lot easier to spend money on physical object, then paying $x for faster software. Once they felt how slow their images and video were, it give them an incentive to upgrade, PC, Tablet, Phone etc.

kuon2• 8 years ago

How does AV1 performs when encoding fast (like realtime for twitch)? Is it possible? If yes, what is the gain over h264?

I really hope an open format will win "the war", but I can't see it work at scale if it doesn't support streaming.

clouddrover5• 8 years ago

Last year Bitmovin demonstrated a live 1080p AV1 stream using 32 cores:

https://bitmovin.com/constantly-evolving-video-landscape-dis...

Twitch wants to use AV1 for their streaming. They want to use the switching frame feature of AV1:

https://www.youtube.com/watch?v=o5sJX6VA34o

And the chroma from luma feature of AV1 works well on video game content:

https://people.xiph.org/~xiphmont/demo/av1/demo1.shtml

https://www.youtube.com/watch?v=yKEDf5-2sT4

kuon2• 8 years ago

That's really promising. I really hope we will see an open video format as standard.

nstart• 8 years ago

I've seen a few mentions of patents in this discussion. Does anyone know how close AV1 is to infringing on any patents or when we might find out? This is the most recent article I found - http://www.streamingmedia.com/Articles/News/Online-Video-New... and it seems that the market doesn't/didn't know enough to know if there are patent infringements.

Yet this article from Mozilla seems like that from June the situation might have changed now that a 1.0 stable spec is out. Given that inspection for infringement has been part of the codec creation process it seems promising that they will be safe. But patents have proven themselves to be funny things.

That said, have there been any updates since June from lawyers/other blogs/etc about the patent situation yet?

cordite• 8 years ago

What happened to webm, which apple doesn’t even support yet?

sparrc• 8 years ago

You might be mixing up containers and codecs. webm is the container format that most av1 video will be delivered in.

With Apple the problem was not so much webm, but the vp9 codec, which is the codec most webm containers use (see youtube).

ie, codecs: av1, vp9 <> h.264, h.265 containers: webm <> mp4

Modern codecs like AV1 can bring better quality video to the open web

Related Stories