AVIF for Next-Generation Image Coding

https://netflixtechblog.com/avif-for-next-generation-image-coding-b1d75675fe4

by dedalus • 6 years ago

161 79 6 years ago

gok• 6 years ago

That 420 vs. 440 comparison animation is rather problematic. GIF can only represent 256 colors. Most of the artifacts in those images are due to GIF-specific dithering.

also, previously: https://news.ycombinator.com/item?id=22327480

Dylan168072• 6 years ago

Should have used apng, sure, but the difference is still very visible on those hard color lines.

bmn__• 6 years ago

https://webmasters.stackexchange.com/q/304/true-color-gif

https://enwp.org/GIF#True_color

oconnore• 6 years ago

Shouldn't this be benchmarked against WebP instead of (obviously much worse) JPEG?

LeoPanthera• 6 years ago

Or indeed HEIC.

A lot of so-called "next generation" codecs are really "current-generation" now.

roca5• 6 years ago

Benchmarks against HEVC and WebP are included in the metrics section of the article.

berkut• 6 years ago

Does any browser support HEIC yet? To my knowledge, even Safari doesn't.

daef• 6 years ago

or flif... https://flif.info/

mkl• 6 years ago

It is benchmarked here against WebP, HEVC, and JPEG2000, as well as JPEG. That's what all the graphs are. AVIF wins quite convincingly on the metrics they've used.

rytill• 6 years ago

I don't see compression speed / performance / time mentioned anywhere in the article.

aidenn0• 6 years ago

I'm guessing that Netflix compares not at all about compression time for this; the most used assets are probably downloaded millions of times and change rarely.

Dylan168072• 6 years ago

And honestly if you're not designing a camera then there's extremely little chance you care about still image compression time.

londons_explore2• 6 years ago

As far as I can see, there is no support in any web browsers yet, despite the format being finalized a year ago... Whats the holdup?

niftich4• 6 years ago

AVIF support Firefox issue [1]; Chromium issue [2].

[1] https://bugzilla.mozilla.org/show_bug.cgi?id=1443863 [2] https://bugs.chromium.org/p/chromium/issues/detail?id=960620

qwerty4561273• 6 years ago

For some weird reason browser manufacturers strongly oppose adding support for any new media formats. They will only accept a new format once they decide that's a good political move. If I were Google/Mozilla I'd add support for everything ffmpeg and imagemagick support.

roca5• 6 years ago

It's been a slow process, and hence perhaps underappreciated, but it's very gratifying that over the last 15 years patent-unencumbered media codecs have won. The companies that contributed to this --- Google, Mozilla, Cisco, and others --- and especially Xiph that got the ball rolling --- deserve a lot of credit.

greggman3• 6 years ago

They have? Are we talking images only because in video mp4 and .h264 and .h265 are pretty much it. Apple/Safari still doesn't support vp8/vp9/webm/av1 for video or ogg for audio

jasondclinton2• 6 years ago

Mobile Safari is about 20-35% of all mobile web traffic. Desktop Safari is about 5-15% of all desktop web traffic. Source: https://en.wikipedia.org/wiki/Usage_share_of_web_browsers

est31• 6 years ago

Are we talking images only because in video mp4 and .h264 and .h265 are pretty much it. Apple/Safari still doesn't support vp8/vp9/webm/av1 for video or ogg for audio

h265 is doomed because of the multiple patent pools that have formed and the extreme price hikes compared to h264. It's a risky technology to build on. That's the best ad that AV1 can have, and many members joined the alliance for open media for that reason after it became clear that the patent pool situation wouldn't resolve.

As for your browser support question, yes, Safari has traditionally been anti-ogg, but a few years ago Apple joined the alliance for open media and added opus support to their browsers (although only in the caf container, for some weird unexplainable reason).

Open codecs have won in the lossless audio domain (flac). There is no reason they can't win in the lossy video and lossy audio domains too (not sure about lossless videos but FFV1 seems to have lots of support by archivists who want open technology).

gsich• 6 years ago

Niche browser.

threeseed3• 6 years ago

H.264/H.265 is the standard for video today.

Not sure where you're getting that idea that open codecs have won.

jasondclinton2• 6 years ago

It depends on what you are measuring. If we go purely by web traffic volume, then whatever YouTube is doing is what's "won". AFAICT, that's VP9 to everything that supports it which seems to be all desktops and new Androids but not Apple devices. Android devices of the previous generation fallback to VP8 where there's hardware decode support.

bawolff2• 6 years ago

Personally i dont think that they have "won". However the mere fact they have are seriously considered as an option, is a major victory.

10 years ago only the hard core capital F Free Software people used them or even considered them

tagrun• 6 years ago

I wouldn't lump them together. There is a chasm between the adoption numbers for H.264 and H.265, HEVC failed gain the momentum AVC did --which is exactly the point GP is making.

MPEG-LA is indeed well connected to lobbying parties and Hollywood, which established HEVC as the codec for 4K Blu-ray discs. (I personally don't remember the last time I borrowed an optical disc to watch a movie. I actually don't even own any type of optical disc driver anymore!). However, VP9 dominates over HEVC in streamed media.

herf2• 6 years ago

Isn't HEIC (iPhone and Android P) patent-encumbered by HEVC? $0.20/device, but your photos are probably using it now.

BurningCycles2• 6 years ago

Do new Android phones save photos in HEIC format ?

ksec8• 6 years ago

but it's very gratifying that over the last 15 years patent-unencumbered media codecs have won.

Am I the only one who is not reading this sentence correctly. It seems to suggest patent unencumbered codec have won for the past 15 years.

Regardless of that, AVIF is not patent unencumbered, it is just royalty free.

roca5• 6 years ago

That is a fair clarification, thanks.

reggieband4• 6 years ago

I worked on a team responsible for image assets. I recall the web team asking for webp support from our image server. I suppose nowadays using `srcset` you can get some optimization in the ratio between file size and quality while still providing fallback for unsupported browsers. Given our image service was dynamic (e.g. it generated then cached images at different sizes, qualities, etc from a single high-res source) adding a new format wasn't that difficult. It meant that the image heavy pages would display more quickly which was a concern especially for mobile. So I am a big supporter for more and better formats.

That being said, this AV1 vs. HEVC / AVIF vs. HEIF stuff feels eerily similar to HLS vs. DASH. It's not like these formats have drastically different properties (e.g. GIF vs. JPG vs. PNG) - they are all quite similar. I just want to fast-forward time until whichever one is going to win is standard. It's like VHS vs. Betamax or Blueray vs. HD-dvd. Please just get it over with.

MikusR• 6 years ago

The difference is that HEVC/HEIF has at least three different patent licensing organizations that all want to be paid. While AV1/AVIF is supposed to be "patent free"

reggieband4• 6 years ago

I heard the same about h.264 vs VP9 and it turned out that patent licensing wasn't a long-term issue. For a while you got that license from Flash (if y'all remember when Flash was the way Youtube worked, and I recall mention that it was one of the primary reasons Flash Player was never opened sourced). Then the license for h.264 it was included in every copy of Windows 8 (IIRC). Nowadays no one even mentions the h.264 license.

I'm no expert on the new license but it seems the restrictions on HEVC are relatively light. I'm pretty sure it is free to compress and distribute media but the license affects hardware and software encoders/decoders. So maybe your HEVC enabled video card with built-in decoder will be a dollar more expensive. I know that every Apple device shipped comes with HEVC hardware standard for the last few years. I would wager most Android devices too.

niftich4• 6 years ago

I am not a lawyer, and I realize there's a fine line between genuine concern and speading FUD, but in the spring of 2019 I looked into the patent situation around the HEIF container itself [1] -- the container upon which AVIF builds -- and skimmed through the 5 US patents I found, which cover some techniques that can be used in the format.

Most of them can probably be avoided for the purposes of an AVIF file, but patent US20160232939A1 [2] in my reading seems to be about in-container signalling to express relationships between a "static media item" and "one or more entities" that together "form a group", and "indicating, in the file, a grouping type for the group". The patent appears to be written in a way to allow this definition to encompass, say, a thumbnail and a bunch of frames thereafter, or, say a master image and a set of pictures derived from it, or alternate camera angles of the same thing. Some of these techniques sound like stuff we've seen before, but as is common in patents, the precise wording of claims is often key, and this is where patent lawyers come in.

A thorough look of the AVIF specification [3] and the patents registered with the MPEG LA about this format [1] is likely wise before any widespread deployment that makes use of advanced features of the HEIF container; using it to hold exactly 1 'one-layer' still-image is probably fine.

Additionally, in my reading [5], the HEIF reference software released by Nokia [4] includes a patent grant for non-commercial evaluation, testing and academic research only.

[1] https://news.ycombinator.com/item?id=19874321 [2] https://patents.google.com/patent/US20160232939A1/ [3] https://aomediacodec.github.io/av1-avif/ [4] https://github.com/nokiatech/heif/blob/master/LICENSE.TXT [5] https://news.ycombinator.com/item?id=19874368

ksec8• 6 years ago

While AV1/AVIF is supposed to be "patent free"

Thanks to the propaganda spread by Google and Mozilla. AV1 is "Royalty free". Not Patent Free.

Not to mention Google patents other people's work into AV1. [1]

[1] https://arstechnica.com/tech-policy/2018/06/inventor-says-go...

shmerl3• 6 years ago

Why use HEIF container for AVIF, instead of something derived from Matroska for example, like WebP does? Is HEIF free to use?

giantrobot3• 6 years ago

The HEIF and HEVC container are the same. So if they use the HEIF container they already have libraries on devices that can handle the format, metadata, etc. I don't believe Netflix is delivering any Matroska content so they have no media pipeline or tooling for the format. It's the same reason they didn't base the container format on RIFF or ASN.1.

shmerl3• 6 years ago

Since they are proposing it as a common standard, I don't think it should matter what they already use, in comparison with something that's actually free to use.

If HEIF is free, then fine. But MPEG stuff is not created free by design, so it has to be stated somewhere.

JyrkiAlakuijala3• 6 years ago

In the brotli data format (rfc7932) we left the concept of a container away altogether.

In gzip there is a container format that allows specifying things that are mostly meaningless (like having a reserved value for 'Amiga').

In JPEG XL there are two stream forms, one with and one without the container. Both work for normal use.

jiggawatts3• 6 years ago

I'm going to put my cynical hat on for a bit and notice that the company behind this format, Netflix, has a walled garden. They have a streaming service that is consumed only through a handful of apps that they are in full control of. They're not doing this for the community, for web standards, or the greater good. They're making this format simply to save some bandwidth for themselves, in a way that won't translate to the general community. The second picture in the linked blog article is literally a diagram of this garden. (Titled: "Compressed image assets destined for various client devices...")

I had a similar comment about the JPEG XL format (mostly developed by Google) and the CSS "Display P3" colour space extension (Apple), both recently featured on YCombinator. These mega-corporations are building an ecosystem where in 2020, the future, it's impossible to send a wide-gamut, 10-bit, or HDR still image to anyone via any of the following: web standards, chat, email, or document exchange formats. The best you can do is send an 8-bit SDR sRGB image and hope for the best. PC monitors, tablets, and most phones have no consistent support for colour management, 10-bit, or HDR. Televisions are leaving the entire PC and Mobile ecosystem in the dust. The closest approximation we PC peasants have is to upload a HDR YouTube video, send a link to it, and hope the viewer uses an newish iPhone. That's just sad, isn't it?

The stewardship of web standards by Microsoft (#2 biggest company), Apple (#3), and Alphabet Group (#4) have led to this. Now Netflix (#50) wants to throw their unnecessary format into the fray, almost completely ignoring the presence of JPEG XL and HEIC. They mention these alternatives in passing, and then notably don't compare image quality, features, or the compression ratio of those to their own format. You see, JXL is a Google's thing, HEIC is an Apple thing, and AVIF is a Netflix thing. So we're going to end up with as many image formats as there are walled gardens. I bet you too can't wait for whatever image format Facebook comes up with specifically to reduce their CDN bandwidth utilisation of Instagram pictures... only.

Notice also that their "idea" of making an image format readily available is a docker container, which is the most insane thing I've ever seen. Where's the Photoshop plugin? The Lightroom plugin? The Windows 10 image codec? Oh wait.. those are Adobe and Microsoft things, so... nowhere to be seen. Netflix Pty Ltd is not in this game to help someone else's walled garden.

No still image interchange for you peasant! Sit down, stay in the garden, and stream that content...

ksec8• 6 years ago

First,

Jpeg XL is not a Google thing, and neither is HEIC an Apple thing. Nor the AVIF part.

AVIF is from Open Media which is indeed a Google Thing. Netflix just happen to take side with OM. JPEG XL is done by a Google Employees as well as other contributors put forward to the JPEG committee. i.e It is not a Google only format.

HEIC is really just HEVC Codec in an image format developed by Nokia and submitted to MPEG ( Not to be confused with MPEG-LA ).

So really JPEG XL isn't a walled garden. And support everything you described including HDR. The problem is Jpeg XL doesn't do so well in small image size as compared to HEIF or AVIF.

jiggawatts3• 6 years ago

Jpeg XL is not a Google thing, and neither is HEIC an Apple thing...<snip>...Employees as well as other contributors put forward to the JPEG committee.

"Technically open" doesn't help much if each of those formats is primarily pushed by one mega-corp, optimised for their specific use-case, and no industry-wide effort materialises to establish widespread interoperability.

Just last week I tried to convert raw photos to HEIF images on Windows so I could share my images in maximum quality with a bunch of iPhone users. It's basically impossible. The only tool I found was Gimp, but it doesn't properly understand colour spaces and the output looked totally wrong. I'd need OSX or IOS to author files in this format, making HEIC an "Apple thing" for all practical purposes.

It's a safe bet that Google will embed JPEG XL support in Chromium, right? Will I be able to email a .JXL file to an IOS user? Will a Windows PC user with Outlook on their desktop be able to see the image embedded in their email without having to save it and open it again in Chrome? Can I paste a .JXL image into a Word document and have its various features be correctly preserved, or will this be restricted to Google Docs viewed exclusively via Chrome?

Look, here's the thing: All of these post-JPEG formats are nice and all, but are worthless to me and a billion other people unless there's proper effort put into mass adoption. Real industry-wide cooperation, not just token openness.

As I said, it is 2020, and nobody has a viable option for distributing still images in anything other than a handful of 20-30 year old formats.

The problem is Jpeg XL doesn't do so well in small image size as compared to HEIF or AVIF.

So then why isn't NetFlix working with the JPEG XL group to add the AV1 codec as one of the encoding options? Why are they instead creating yet another standard with no adoption likely outside of their walled garden?

Conversely, why are all the post-JPEG formats such utter failures? Take a look at the list: https://jpeg.org/jpeg/index.html

    JPEG         - The "popular one"
    JPEG XT      - Never heard of it
    JPEG-LS      - Never heard of it
    JPEG 2000    - Heard of it, never used it
    JPEG XR      - Never heard of it
    AIC          - Heard of it, never used it
    JPEG Systems - Never heard of it
    JPEG XS      - Never heard of it
    JPEG Pleno   - Never heard of it
    JPEG XL      - Not yet useful, will likely never use it

Clearly, whatever they JPEG group is doing, it's not working. They got lucky once, and in the 27 years since have failed to reproduce that success.

This is the core point of what I'm saying: The reason that GIF, PNG, and JPG got widespread support has more to do with luck rather than the standardisation approach. The laundry list of failures and the sad current state are evidence that luck is not a strategy and something different must be tried.

I put this in the same category of failure as IPv6. The IETF got lucky with IPv4 and figured that "more of the same" will work. They didn't get lucky the second time.

I wish I had the "political clout" to bang some heads together at these megacorps, make them sit down together and fix this.

Realistically, I foresee another decade of 8-bit, SDR, sRGB images stretched incorrectly to whatever random gamut the $150 monitor of the recipient is capable of displaying.

JyrkiAlakuijala3• 6 years ago

JPEG XL does better for small images if we talk about small as 32x32 pixels. It's header overhead is far less than anything else.

JPEG XL's current encoder is focused on qualities 0.5 BPP and upwards. Current internet image usage averages at 2 to 3 BPP based on Chrome stats from June 2019. See https://www.spiedigitallibrary.org/conference-proceedings-of... Figure 1.

When you go to 0.1 BPP, every codec will look unacceptable, but the current encoder of JPEG XL looks worse than video codecs.

We will likely be able to extend the scale of handling bad quality down to 0.1 to 0.25 BPP, but it definitely has not been a priority since we don't anticipate that users actually would use it for photographs.

(When an image has a low noise level or less features, then we naturally can compress it down to 0.00x BPP with JPEG XL, too. The 0.5 BPP rule of thumb is for normal vacation photographs and selfies :-)

colbyn• 6 years ago

Interesting. Personally, I’ve been using VMAF extensively in my imager.io project. With regards to only testing the luma channel, someone submitted a PR for such some time ago yet it’s never been merged; I was under the impression that the UV channel isn’t significant (in practice). Personally, I think VMAF is great for images. For me, the biggest downside of VMAF (implementation wise) is that it’s not currently thread-safe.

pornel• 6 years ago

This is a major improvement. It's better than WebP by a bigger margin than WebP was better than JPEG.

Lossy WebP couldn't even match JPEG in some aspects, like full-resolution color (4:2:0 only, not even full 8-bit depth). In terms of quality or features AVIF doesn't have such "buts". It can support HDR and high depth we're likely to want in the near future. AV1 has palette blocks, so it's no longer all-or-nothing choice of lossy vs lossless formats.

I am worried that the AVIF spec is complex (like HEIF, it's a bag of image-related features, more like a PowerPoint file than an image), but probably only commonly-used features will survive out of it. Early JPEG also used to have many more modes than are usable today.

If you like questionable hacks, you can use AV1 codec today in Firefox and Chrome by embedding <video> element with a 1-frame AV1 "video".

chungy• 6 years ago

It's better than WebP by a bigger margin than WebP was better than JPEG.

Depending on certain definitions of better. Image quality is certainly superior in JPEG than in (lossy) WebP.

ec109685• 6 years ago

The JPEG images in the article are much worse than what you can achieve with JPEG export in Preview.app. While they don't look like AVIF, they aren't nearly as trash as the article makes JPEG's out to be.

herf2• 6 years ago

I'm no fan of 4:2:2 (2:1 chroma subsampling), but the comparison looks a lot better for JPEG when you use 4:2:2 (which is the usual JPEG default). Using 4:4:4 instead drives the blocking artifacts way up, and isn't fair to JPEG.

chubs• 6 years ago

I spent some time searching for a JS shim that enables AVIF files to be used in the browser, is anyone aware of one?

niftich4• 6 years ago

The official avif github [1] references Kagami/avif.js [2]. It's intended as a server-side component to be installed, and uses service workers to on-the-fly repack AVIF images as AV1 videos. The code is licensed CC0, and is easy to read, so you can tweak it to your needs.

If the browser can't natively decode AV1 videos, it calls dav1d.js [3] to decode, which is a webassembly port of dav1d [4].

[1] https://github.com/AOMediaCodec/av1-avif/wiki [2] https://github.com/Kagami/avif.js [3] https://github.com/Kagami/dav1d.js [4] https://code.videolan.org/videolan/dav1d

JyrkiAlakuijala3• 6 years ago

encode.su forum posts compare image quality of JPEG XL, AVIF, WebP, MozJPEG and HEVC:

https://encode.su/threads/3108-Google-s-compression-proje%D1...

The most comprehensive comparison:

https://medium.com/@scopeburst/mozjpeg-comparison-44035c42ab...

"... according to my personal visual tests, starting from 0.4–0.5 bpp it [JPEG XL] usually wins most other formats."

AVIF for Next-Generation Image Coding

Related Stories