Rav1e: An experimental AV1 video encoder, designed to be fast and safe

https://github.com/xiph/rav1e

by adamnemecek • 8 years ago

200 73 8 years ago

github.com

kristofferR• 8 years ago

What does "safest" mean in this context?

AndrewDucker• 8 years ago

"Written in Rust"

blitmap• 8 years ago

I'm trying to get my insurance discounted on my car. I should slap a "written in Rust" sticker on it. Safe driver discount?

jabl• 8 years ago

Presumably that it's implemented in Rust, and thus does not suffer from many of the usual bugs that C code bases suffer from?

There seems to be some asm code as well, which obviously does not enjoy the safety advantages of Rust.

arez• 8 years ago

I always wondered if in practice it really checks out, is rust code really safer? Did the bugs just shift to different ones? Is there anyone who wrote about that already?

zakk2• 8 years ago

~5 fps encoding @ 480p

How does this compare with the reference encoder?

masklinn4• 8 years ago

You'd need to run them on the same machine to make sure you get a proper comparison but https://ffmpeg.zeranoe.com/forum/viewtopic.php?t=5601 has some runs. One of the users downthread ("entac") provides both libaom and libx264 numbers, they get 63fps for libx264 and 0.0924fps for libaom (r9028)

Also this currently does delegates work to libaom.

zakk2• 8 years ago

Many thanks!

derf_3• 8 years ago

Also this currently does delegates work to libaom.

Currently just for the transforms and to initialize the probabilities for the entropy coder.

tatref• 8 years ago

The readme didn't specify the --release flag in the cargo run command, this should make a pretty big difference

steveklabnik• 8 years ago

I sent a PR; as they merged, they mentioned their test scripts use it...

skolemtotem• 8 years ago

Will AV1, or even VP9 for that matter, ever be suitable for realtime encoding, or is that just not their target market?

masklinn4• 8 years ago

AFAIK realtime encoding is a design consideration for AV1.

JustFinishedBSG• 8 years ago

Well eventually we'll get AV1 hardware encoding so I'd say in 5 years smartphones will encode AV1 in real time

matt4077• 8 years ago

Yes, of course. Anything else would be DOA.

On a general note: there really seems to be an extremely inaccurate narrative regarding AV1 and speed taking hold. I can't understand why it isn't easier understood that a reference implementation is about accuracy only, completely ignoring performance considerations. Not in the usual "we'll now try to make it faster", but as in "this is never meant to be used in production, and it's performance is in no way indicative of the performance optimised encoders will see".

As but one example: media encoding is pretty close to being "embarrassingly parallel" in principle, making the first three orders of magnitude easy wins for a straightforward GPU implementation.

Ienuur4i• 8 years ago

media encoding is pretty close to being "embarrassingly parallel" in principle

My understanding is that there are some fairly tight feedback loops in the encoders that make it difficult to offload things to the GPU, at least if you want to maximize the quality per byte metric. If you want to target realtime and don't need optimal compression it probably gets easier.

pas• 8 years ago

People are pragmatic, at least in this regard. They don't really suffer from the bandwidth costs, they want fast encode speeds for offline storage.

And they are simply cautious. They don't really care about the hype x264 is good enough visually, now all visual comparisons are done on ridiculously low bitrate (which is a good thing, but people don't really care).

Jasper_2• 8 years ago

As but one example: media encoding is pretty close to being "embarrassingly parallel" in principle

Which part? 90% of what you're doing is context or inter-frame dependent. Video encoders that live on graphics cards today use dedicated ASIC hardware.

Ace172• 8 years ago

Video compression engineer here.

I can't understand why it isn't easier understood that a reference implementation is about accuracy only, completely ignoring performance considerations.

Because the official codebase conveys another message. Have a look, there are SIMD implementations for almost all supported targets.

https://aomedia.googlesource.com/aom/+/av1-normative/aom_dsp... https://aomedia.googlesource.com/aom/+/av1-normative/aom_dsp... https://aomedia.googlesource.com/aom/+/av1-normative/aom_dsp... ....

What are these files for, if not performance? They've been maintained and kept synchronized with the reference C code during the whole project, long before the codec was frozen (and it was a huge PITA).

This doesn't look like "completely ignoring performance considerations".

As but one example: media encoding is pretty close to being "embarrassingly parallel" in principle,

Almost all video codecs exploit some block-level encoding context, which means the way you encode one block depends on how the previous neighbooring blocks were encoded. This creates a huge dependency between blocks. There are tools like slicing/tiling that allow you to break these dependencies, and thus, encoding in parallel, but at the cost of video quality. Making the problem "embarrassingly parallel" at this point would make the video "embarrasingly ugly".

You could encode multiple frames in parallel ; but then again, being able to encode them independently means you're basically trashing all the compression context (reference frames), and your video quality goes down the tubes.

In an offline encoding scenario (Netflix, Youtube), if you have lots of memory, you can encode multiple independent video sequences from the same movie. Making the problem "embarrassingly parallel" in this case would require an "embarrassingly huge" amount of memory. Also, it's not applicable to a live scenario (think: latency).

p0nce• 8 years ago

Their target market is "negotiating with MPEG for lower royalties"

sp3322• 8 years ago

Yes, it's just that everyone is focusing on size/bandwidth optimization for now. Once they nail down the actual format, projects will start work on making it fast.

masklinn4• 8 years ago

I believe the possibility of "making it fast" is taken in account in the existing design, to avoid designing a format which can't be cheaply optimised & hardware-implemented.

TD-Linux2• 8 years ago

I actually had a realtime rav1e demo running, with some modifications. There's nothing inherently less realtime about AV1 than, say H.264.

derf_3• 8 years ago

There are a number of features that make AV1 structurally more suited to real-time implementations than its predecessor, VP9.

For example, it does adaptive entropy coding instead of explicitly coding probabilities in the header. That means that you don't need to choose between making multiple passes over the frame (one to count symbol occurrences and one to write the bitstream using R-D optimal probabilities) or encoding with sub-optimal probabilities (which can have an overhead upwards of 5% of the bitrate). libaom has always been based on a multi-pass design, as was libvpx before it, but rav1e only needs a single pass per frame (we may add multiple passes for non-realtime use cases later).

In another example, AV1 has explicit dependencies between frames. VP9 maintained multiple banks of probabilities which could be used as a starting point for a new frame. But any frame was allowed to modify any bank. So if you lost a frame, you had no idea if it modified the bank of probabilities used by the next frame. In AV1, probabilities (and all other inter-frame state) propagate via reference frames. So you're guaranteed that if you have all of your references, you can decode a frame correctly. This is important if you want to make a low-latency interactive application that never shows a broken frame.

Some of its tools also become more effective in low-complexity settings. One of the new loop filters, CDEF, gives somewhere around a 2% bitrate savings using objective metrics when tested with libaom running at its highest complexity (although subjective testing suggests the actual improvement is larger). However, when you turn down the complexity, the improvement from CDEF goes up to close to 8%. I.e., using this filter helps you to take shortcuts elsewhere in the encoder.

The real reason the reference encoder is so slow is that it searches a lot of things. You can always make things run faster by searching less. Take a look at http://obe.tv/about-us/obe-blog/item/54-a-look-at-the-bbc-uh... to see how drastically people are limiting HEVC to make it run in real time today (though if you have to go up to 35 Mbps to do so, one might wonder what the point is).

sargun• 8 years ago

Just curious, what's the memory footprint on the encoder in real life?

Do different video encoders, for the same codec, and input produce different outputs, or is the algorithm specified in a way where it produces the same results for two given inputs, no matter what?

TD-Linux2• 8 years ago

Virtually all of the consumed memory is from the reference frames, so it depends on how big your video is. AV1 supports up to 8 reference frames, though if you're memory constrained you could use less.

Different encoders produce different outputs, the algorithm isn't specified.

wolf550e• 8 years ago

For almost all compression algorithms (both lossless and lossy), only the decompression is specified. A compressor can do whatever it wants as long as it produces a bitstream that a compliant decompressor can decode.

For example, you can make a video enocoder that produces a compliant video stream in which every frame is a keyframe and every macroblock is independently fully encoded, thus reducing AV1 (or H265, etc.) to MJPEG. But if the result is decodable by a compliant decoder, your compressor is compliant. It might even be somewhat useful (e.g. output needs to be zero latency or output is intended to be edited).

tzahola3• 8 years ago

And this is how people keep making substantial improvements to ancient formats like JPEG or MP3.

E.g. Guetzli from Google: https://github.com/google/guetzli/blob/master/README.md

Pissompons• 8 years ago

Do different video encoders, for the same codec, and input produce different outputs, or is the algorithm specified in a way where it produces the same results for two given inputs, no matter what?

Someone else answered this but I thought I'd elaborate: A good way to think about a codec is as a toolbox. The specification tells you which tools you can use to build a frame (encoder) and which you must support to turn one into pixels (decoder).

Which tools are used in what way makes a huge difference in the output of the encoder, particularly in terms of compression. Have a look at the results for a few H.264 encoders [0]. For the "video conferencing" use-case, the best encoder (x264) uses ~400kbps to produce the same quality of the worst at ~1000kbps.

And like different tools have different costs (a jackhammer needs a generator, a handheld hammer does not), so do the tools in the codec toolbox. Some tools might make the encode slower or might make the decode drain more battery from a mobile device. Others might take a lot of physical space on a piece of silicon, so they're rarely used in hardware.

So different encoders have very different characteristics, not just in terms of output but in terms of power usage, speed and complexity as well.

[0] http://www.compression.ru/video/codec_comparison/h264_2012/

conradev• 8 years ago

Different video encoders certainly produce different outputs. Even the same video encoders produce different outputs. Here is a good paper that explains why that is, and talks about how to make a deterministic video h264 encoder:

http://www.ndsl.kaist.edu/~kyoungsoo/papers/mmsys14.pdf

Rav1e: An experimental AV1 video encoder, designed to be fast and safe

Related Stories