JPEG XL and the Pareto Front

https://cloudinary.com/blog/jpeg-xl-and-the-pareto-front

by botanical • 2 years ago

497 310 2 years ago

cloudinary.com

mikae12• 2 years ago

If only Google could be convinced to adopt this marvelous codec... Not looking super positive at the moment:

https://issues.chromium.org/issues/40270698

https://bugs.chromium.org/p/chromium/issues/detail?id=145180...

pgeorgi8• 2 years ago

All those requests to revert the removal are funny: you want Chrome to re-add jxl behind a feature flag? Doesn't seem very useful.

Also, all those Chrome offshoots (Edge, Brave, Opera, etc) could easily add and enable it to distinguish themselves from Chrome ("faster page load", "less network use") and don't. Makes me wonder what's going on...

silisili• 2 years ago

Simply put these offshoots don't really seem to do browser code, and realize how expensive it would be for them to diverge at the core.

eviks4• 2 years ago

No, obviously to re-add jxl without a flag

lonjil27• 2 years ago

you want Chrome to re-add jxl behind a feature flag? Doesn't seem very useful.

Chrome has a neat feature where some flags can be enabled by websites, so that websites can choose to cooperate in testing. They never did this for JXL, but if they re-added JXL behind a flag, they could do so but with such testing enabled. Then they could get real data from websites actually using it, without committing to supporting it if it isn't useful.

Also, all those Chrome offshoots (Edge, Brave, Opera, etc) could easily add and enable it to distinguish themselves from Chrome ("faster page load", "less network use") and don't. Makes me wonder what's going on...

Edge doesn't use Chrome's own codec support. It uses Windows's media framework. JXL is being added to it next year.

sergioisidoro• 2 years ago

It's so frustrating how the chromium team is ending up as a gatekeeper of the Internet by pick and choosing what gets developed or not.

I recently come across another issue pertaining to the chromium team not budging on their decisions, despite pressure from the community and an RFC backing it up - in my case custom headers in WebSocket handshakes, that are supported by other Javascript runtimes like node and bun, but the chromium maintainer just disagrees with it - https://github.com/whatwg/websockets/issues/16#issuecomment-...

hwbunny• 2 years ago

Question is for how long. Time to slam the hammer on them.

madeofpalk• 2 years ago

Where's Firefox's and Webkit's position on the proposal?

pgeorgi8• 2 years ago

It's so frustrating how the chromium team is ending up as a gatekeeper of the Internet by pick and choosing what gets developed or not.

https://github.com/niutech/jxl.js is based on Chromium tech (Squoosh from GoogleChromeLabs) and provides an opportunity to use JXL with no practical way for Chromium folks to intervene.

Even if that's a suboptimal solution, JXL's benefits supposedly should outweight the cost of integrating that, and yet I haven't seen actual JXL users running to that in droves.

So JXL might not be a good support for your theory: where people could do they still don't. Maybe the format isn't actually that important, it's just a popular meme to rehash.

jillesvangurp3• 2 years ago

They didn't "lose interest", their lawyers pulled the emergency brakes. Blame patent holders, not Google. Like Microsoft: https://www.theregister.com/2022/02/17/microsoft_ans_patent/. Microsoft could probably be convinced to be reasonable. But there may be a few others. Google actually also holds some patents over this but they've done the right thing and license those patents along with their implementation.

To fix this, you'd need to convince Google, and other large companies that would be exposed to law suits related to these patents (Apple, Adobe, etc.), that these patent holders are not going to insist on being compensated.

Other formats are less risky; especially the older ones. Jpeg is fine because it's been out there for so long that any patents applicable to it have long expired. Same with GIF, which once was held up by patents. Png is at this point also fine. If any patents applied at all they will soon have expired as the PNG standard dates back to 1997 and work on it depended on research from the seventies and eighties.

lifthrasiir12• 2 years ago

[...] other large companies that would be exposed to law suits related to these patents (Apple, Adobe, etc.) [...]

Adobe included JPEG XL support to their products and also the DNG specification. So that argument is pretty much dead, no?

zokier• 2 years ago

their lawyers pulled the emergency brakes

Do you have source for that claim?

peppermint_gum2• 2 years ago

To fix this, you'd need to convince Google, and other large companies that would be exposed to law suits related to these patents (Apple, Adobe, etc.), that these patent holders are not going to insist on being compensated.

Apple has implemented JPEG XL support in macOS and iOS. Adobe has also implemented support for JPEG XL in their products.

Also, if patents were the reason Google removed JXL from Chrome, why would they make up technical reasons for doing so?

Please don't present unsourced conspiracy theories as if they were confirmed facts.

JyrkiAlakuijala47• 2 years ago

That ANS patent supposedly relates to refining the coding tables based on symbols being decided.

It is slower for decoding and Jpeg xl does not do that for decoding speed reasons.

The specification doesn't allow it. All coding tables need to be in final form.

bmicraft• 2 years ago

Safari supports jxl since version 17

jonsneyers13• 2 years ago

There are no royalties to be paid on JPEG XL. Nobody but Cloudinary and Google is claiming to hold relevant patents, and Cloudinary and Google have provided a royalty free license. Of course the way the patent system works, anything less than 20 years old is theoretically risky. But so far, there is nobody claiming royalties need to be paid on JPEG XL, so it is similar to WebP in that regard.

lonjil27• 2 years ago

The Microsoft patent doesn't apply to JXL, and in any case, Microsoft has literally already affirmed that they will not use it to go after any open codec.

Pikamander2• 2 years ago

Mozilla effectively gave up on it before Google did.

https://bugzilla.mozilla.org/show_bug.cgi?id=1539075

It's a real shame, because this is one of those few areas where Firefox could have lead the charge instead of following in Chrome's footsteps. I remember when they first added APNG support and it took Chrome years to catch up, but I guess those days are gone.

Oddly enough, Safari is the only major browser that currently supports it despite regularly falling behind on tons of other cutting-edge web standards.

https://caniuse.com/jpegxl

JyrkiAlakuijala47• 2 years ago

I followed Mozilla/Firefox integration closely. I was able to observe enthusiasm from their junior to staff level engineers (linkedin-assisted analysis of the related bugs ;-). However, an engineering director stepped in and locked the discussions because they were in "no new information" stage, and their position has been neutral on JPEG XL, and the integration has not progressed from the nightly builds to the next stage.

Ten years ago Mozilla used to have the most prominent image and video compression effort called Daala. They posted inspiring blog posts about their experiments. Some of their work was integrated with Cisco's Thor and On2's/Chrome's VP8/9/10, leading to AV1 and AVIF. Today, I believe, Mozilla has focused away from this research and the ex-Daala researchers have found new roles.

miragecraft• 2 years ago

It feels like nowadays Mozilla is extremely shorthanded.

They probably gave up because they simply don’t have the money/resources to pursue this.

Zamicol• 2 years ago

The new version of libjxl brings a very substantial reduction in memory consumption, by an order of magnitude, for both lossy and lossless compression. Also the speed is improved, especially for multi-threaded lossless encoding where the default effort setting is now an order of magnitude faster.

Very impressive! The article too is well written. Great work all around.

Modified3019• 2 years ago

At the very low quality settings, it's kinda remarkable how jpeg manages to to keep a sharper approximation of detail that preserves the holistic quality of the image better in spite of the obvious artifacts making it look like a mess of cubism when examined close. It's basically converting the image into some kind of abstract art style.

Whereas jxl and avif just become blurry.

porker• 2 years ago

Yes, that was my takeaway from this that JPEG keeps edge sharpness really well (e.g. the eyelashes) while the jxl and avif smooth all detail out of the image.

mort968• 2 years ago

No, JXL and AVIF keep keep the same level of edge sharpness but without all the blocking artifacts when given the same amount of bits per pixel as the lowest-quality jpeg.

JyrkiAlakuijala47• 2 years ago

It is because JPEG is given 0.5 bits per pixel, where JPEG XL and AVIF are given around 0.22 and 0.2.

These images attempt to be at equal level of distortion, not at equal compression.

Bpps are reported beside the images.

In practice, use of quality 65 is rare in the internet and only used at the lowest quality tier sites. Quality 75 seems to be usual poor quality and quality 85 the average. I use quality 94 yuv444 or better when I need to compress.

bmacho2• 2 years ago

You refer to this? https://res.cloudinary.com/jon/qp-low.png

Bitrates are in the left column, jpg low quality is the same size as jxl/avif med-low quality (0.4bpp), so you should compare the bottom left picture to the top mid and right pictures.

mrob• 2 years ago

JPEG bitrates are higher, so all it means is that SSIMULACRA2 is the wrong metric for this test. It seems that SSIMULACRA2 heavily penalizes blocking artifacts but doesn't much care about blur. I agree that the JPEG versions look better at the same SSIMULACRA2 score.

JyrkiAlakuijala47• 2 years ago

Ideally one would use human ratings.

The author of the blog post did exactly that in a previous blog post:

https://cloudinary.com/labs/cid22/plots

Human ratings are expensive and clumsy so people often use computed aka objective metrics, too.

The best OSS metrics today are butteraugli, dssim and simulacra. The author is using one of them. None of the codecs was optimized for that metrics except jpegli partially.

jonsneyers13• 2 years ago

Humans generally tend to prefer smoothing over visible blocking artifacts. This is especially true when a direct comparison to the original image is not possible. Of course different humans have different tastes, and some do prefer blocking over blur. SSIMULACRA2 is based on the aggregated opinions of many thousands of people. It does care more about blur than metrics like PSNR, but maybe not as much as you do.

izacus7• 2 years ago

Well, that's because JPEG is still using about twice as many bits per pixel, making the output size significantly larger.

Don't get swept away by false comparisons, JXL and AVIF look significantly better if you give them twice as much filesize to work with as well.

aidenn02• 2 years ago

One does wonder how much of JXL's awesomeness is the encoder vs. the format. Its ability to make high quality, compact images just with "-d 1.0" is uncanny. With other codecs, I had to pass different quality settings depending on the image type to get similar results.

kasabali8• 2 years ago

That's a very good point. At this rate of development I wouldn't be surprised if libjxl becomes x264 of image encoders.

On the other hand, libvpx has always been a mediocre encoder which I think might be the reason for disappointing performance (I mean in general, not just speed) of vp8/vp9 formats, which inevitably also affected performance of lossy WebP. Dark Shikari even did a comparison of still image performances of x264 vs vp8 [0].

[0] https://web.archive.org/web/20150419071902/http://x264dev.mu...

JyrkiAlakuijala47• 2 years ago

While WebP lossy still has image quality issues it has improved a lot over the years. One should not consider a comparison done with 2010-2015 implementations indicative of quality performance today.

edflsafoiewq6• 2 years ago

They've also made a JPEG encoder, cjpegli, with the same "-d 1.0" interface.

dingdingdang2• 2 years ago

Excellent run-through of jpegli encoder here: https://giannirosato.com/blog/post/jpegli/ - wish I could find a pre-compiled terminal utility for cjpegli!

JyrkiAlakuijala47• 2 years ago

Pik was designed initially without quality options only to do the best there is to achieve distance 1.0.

We kept a lot of focus on visually lossless and I didn't want to add format features which would add complexity but not help at high quality settings.

In addition to modeling features, the context modeling and efficiency of entropy coding is critical at high quality. I consider AVIFs entropy coding ill-suited for high quality or lossless photography.

anewhnaccount2• 2 years ago

Should the Pareto front not be drawn with line perpendicular to the axes rather than diagonal lines?

penteract3• 2 years ago

Yes, it should, but it looks like they just added a line to the jxl 0.10 series of data on whatever they used to make the graph, and labelled it the Pareto front. Looking closely at the graphs, they actually miss some points where version 0.9 should be included in the frontier.

lifthrasiir12• 2 years ago

I think it can be understood as an expected Pareto frontier if enough options are added to make it continuous, which is often implied in this kind of discussions.

jamesthurley• 2 years ago

Perpendicular to which axis?

penteract3• 2 years ago

both - staircase style.

JyrkiAlakuijala47• 2 years ago

Often with this kind of pareto it can be argued that even when continuous decisions are not available, a compression system could keep choosing every second at effort 7 and every second at effort 6 (or any ratio), leading, on the average interpolated results. Naturally such interpolation does not produce straight lines in log space.

kasabali8• 2 years ago

I'm surprised mozjpeg performed worse than libjpeg-turbo at high quality settings. I thought its aim was having better pq than libjpeg-turbo at the expense of speed.

JyrkiAlakuijala47• 2 years ago

It is consistent to what I have seen. Both in metrics and in eyeballing. Mozjpeg gives good results around quality 75, but less good at 90+++

AceJohnny2• 2 years ago

I do not understand why this article focuses so much on encode speed, but for decode, which I believe represents 99% of usage in this web-connected world, give a cursory...

Decode speed is not really a significant problem on modern computers, but it is interesting to take a quick look at the numbers.

lifthrasiir12• 2 years ago

Anything more than 100 MB/s is considered "enough" for the internet because at that point your bottleneck is no longer decoding. Most modern compression algorithms are asymmetric, that is, you can spend much more time on compression without significantly affecting the decompression performance, so it is indeed less significant once the base performance is achieved.

oynqr2• 2 years ago

When you actually want good latency, using the throughput as a metric is a bit misguided.

JyrkiAlakuijala47• 2 years ago

During the design process of pik/jpeg xl I experimented on decode speed as a personal experience to have an opinion about this. I tried a special version of chrome that artificially throttled the image decoding. Once the decoding speed gets into the 20 megapixels per second the feeling coming from the additional speed was difficult to notice. I tried 2, 20 and 200 megapixels per second throttlings. This naturally depends on image sizes and uses too.

There was a much more easy to notice impact from progressive images and even sequential images displayed in a streaming manner during the download. As a rule of thumb, sequential top-to-bottom streaming feels 2x faster as a waiting rendering, and progressive feels 2x faster than sequential streaming.

silvestrov• 2 years ago

Decoding speed is important for battery time.

If a new format drains battery twice as fast, users don't want it.

jsheard• 2 years ago

Is it practical to use hardware video decoders to decode the image formats derived from video formats, like AVIF/AV1 and HEIC/H264? If so that could be a compelling reason to prefer them over a format like JPEG XL which has to be decoded in software on all of today's hardware. Everything has H264 decode and AV1 decode is steadily becoming a standard feature as well.

jonsneyers13• 2 years ago

No browser bothers with hardware decode of WebP or AVIF even if it is available. It is not worth the trouble for still images. Software decode is fast enough, and can have advantages over hw decode, such as streaming/progressive decoding. So this is not really a big issue.

izacus7• 2 years ago

No, not really - mostly because setup time and concurrent decode limitations of HW decoders across platforms tend so undermine any performance or battery gains from that approach. As far as I know, not even mobile platforms bother with it with native decoders for any format.

PetahNZ• 2 years ago

My server will encode 1,000,000 images itself, but each client will only decode like 10.

okamiueru• 2 years ago

That isn't saying much or anything.

bombcar6• 2 years ago

But you may have fifty million clients, so the total "CPU hours" spend on decoding will outlast encoding.

But the person encoding is picking the format, not the decoder.

lonjil27• 2 years ago

Real-time encoding is pretty popular, for which encoding speed is pretty important.

stusmall• 2 years ago

In some use cases the company is paying for the encoding, but the client is doing the decoding. As long as the client can decode the handful of images on the page fast enough for the human to not notice, its fine. Meanwhile any percentage improvement for encoding can save real money.

JyrkiAlakuijala47• 2 years ago

Companies and economies who optimize the cost of clients as if their own can see diffuse benefits from it. Even if they consider 0.01 % of that cost it can lead to better decisions. If that will make the website load faster or allow for more realistic product pictures, it can lead to faster growth or less returns etc.

youngtaff• 2 years ago

Because that’s Cloudinary’s use case… they literally spend millions of dollars encoding images

mips_r4300i• 2 years ago

This is really impressive even compared to WebP. And unlike WebP, it's backwards compatible.

I have forever associated Webp with macroblocky, poor colors, and a general ungraceful degradation that doesn't really happen the same way even with old JPEG.

I am gonna go look at the complexity of the JXL decoder vs WebP. Curious if it's even practical to decode on embedded. JPEG is easily decodable, and you can do it in small pieces at a time to work within memory constraints.

bombcar6• 2 years ago

Everyone hates WebP because when you save it, nothing can open it.

That's improved somewhat, but the formats that will have an easy time winning are the ones that people can use, even if that means a browser should "save JPGXL as JPEG" for awhile or something.

ComputerGuru4• 2 years ago

Everyone hates webp for a different reason. I hate it because it can only do 4:2:0 chroma, except in lossless mode. Lossless WebP is better than PNG, but I will take the peace of mind of knowing PNG is always lossless over having a WebP and not knowing what was done to it.

lonjil27• 2 years ago

I think JXL has been seeing adoption by apps faster than Webp or AVIF.

mfkp• 2 years ago

I've noticed in chrome-based browsers, you can right click on a webp file and "edit image". When you save it, it defaults to png download, which makes a simple conversion.

Mobile browsers seem to default to downloading in png as well.

CharlesW• 2 years ago

And unlike WebP, it's backwards compatible.

No, JPEG XL files can't be viewed/decoded by software or devices that don't have a JPEG XL decoder.

Zardoz84• 2 years ago

JPEG XL can be converted to/from JPEG without any loss of quality. See another commenter where shows a example where doing JPEG -> JPEG XL -> JPEG generates a binary exact copy of the original JPEG.

Yeah, this not means what usually we call backwards compatibility, but allows usage like storing the images as JPEG XL and, on the fly, send a JPEG to clients that can't use it, without any loss of information. WebP can't do that.

throwaway81523• 2 years ago

Does JPEG XL have patent issues? I half remember something about that. Regular JPG seems fine to me. Better compression isn't going to help anyone since they will find other ways to waste any bandwidth available.

lifthrasiir12• 2 years ago

The main innovation claimed by Microsoft's rANS patent is about the adaptive probability distribution, that is, you should be able to efficiently correct the distribution so that you can use less bits. While that alone is an absurd claim (that's a benefit shared with arithmetic coding and its variants!) and there is a very clear prior art, JPEG XL doesn't dynamically vary the distribution so is thought to be not related to the patent anyway.

jonsneyers13• 2 years ago

No it doesn't.

And yes, regular JPEG is still a fine format. That's part of the point of the article. But for many use cases, better compression is always welcome. Also having features like alpha transparency, lossless, HDR etc can be quite desirable, and those things are not really possible in JPEG.

taylorius• 2 years ago

The article mentions encoding speed as something to consider, alongside compression ratio. I would argue that decoding speed is also important. A lot of the more modern formats (webp, avif etc) can take significantly more CPU cycles to decode than a plain old jpg. This can slow things down noticeably,especially on mobile.

oynqr2• 2 years ago

JPEG and JXL have the benefit of (optional) progressive decoding, so even if the image is a little larger than AVIF, you may still see content faster.

lifthrasiir12• 2 years ago

Note that JPEG XL always supports progressive decoding, because the top-level format is structured in that way. The optional part is a finer-grained adjustment to make the output more suitable for specific cases.

izacus7• 2 years ago

That's great, are there any comparison graphs and benchmarks showing that in real life (similarly to this article)?

lifthrasiir12• 2 years ago

Any computation-intensive media format on mobile is likely using a hardware decoder module anyway, and that most frequently includes JPEG. So that comparison is not adaquate.

kasabali8• 2 years ago

"computation-intensive media" = videos

Seriously, when is the last time mobile phones used hardware decoding for showing images? Flip phones in 2005?

I know camera apps use hardware encoding but I doubt gallery apps or browsers bother with going through the hardware decoding pipeline for hundreds of JPEG images you scroll through in seconds. And when it comes to showing a single image they'll still opt to software decoding because it's more flexible when it comes to integration, implementation, customization and format limits. So not surprisingly I'm not convinced when I repeatedly see this claim that mobile phones commonly use hardware decoding for image formats and software decoding speed doesn't matter.

izacus7• 2 years ago

No, not a single mobile platform uses hardware decode modules for still image decoding as of 2024.

At best, the camera processors output encoded JPEG/HEIF for taken pictures, but that's about it.

bmacho2• 2 years ago

How is lossless webp 0.6th of the size of lossless avif? I find it hard to believe that.

1497655• 2 years ago

Lossless webp is actually quite good, especially on text heavy images, e.g. screenshots of a terminal with `cwebp -z9` are usually smaller than `jxl -d 0 -e 9` in my experience.

lonjil27• 2 years ago

Lossless AVIF is just really quite bad. Notice that how for photographic content, it is barely better than PNG, and for non-photographic content, it is far worse than PNG.

edflsafoiewq6• 2 years ago

It's so bad you wonder why AV1 even has a lossless mode. Maybe lossy mode has some subimages it uses lossless mode on?

jug4• 2 years ago

WebP is awesome at lossless and way better than even PNG.

It's because WebP has a special encoding pipeline for lossless pictures (just like PNG) while AVIF is basically just asking a lossy encoder originally designed for video content to stop losing detail. Since it's not designed for that it's terrible for the job, taking lots of time and resources to produce a worse result.

kllrnohj8• 2 years ago

and way better than even PNG.

I mean it's kinda hard to be worse than just shoving a bitmap through zlib...

account426• 2 years ago

On the other hand, animated webp is surprisingly bad considering that the format comes from a video codec.

derf_3• 2 years ago

Usually the issue is not using the YCgCo-R colorspace. I do not see enough details in the article to know if that is the case here. There are politics around getting the codepoint included: https://github.com/AOMediaCodec/av1-avif/issues/129

eviks4• 2 years ago

Welcome efficiency improvements

And in general, Jon's posts provide a pretty good overview on the topic of codec comparison

Pity such a great format is being held back by the much less rigorous reviews

a-french-anon4• 2 years ago

Pretty good article, though I would have used oxipng instead of optipng in the lossless comparisons, it's the new standard, there.

jonsneyers13• 2 years ago

Thanks for the suggestion, oxipng is indeed a better choice. Next time I will add it to the plots!

gaazoh2• 2 years ago

The inclusion of QOI in the lossless benchmarks made me smile. It's a basically irrelevant format, that isn't supported by default by any general-public software, that aims to be just OK, not even good, yet it has a spot on one of these charts (non-photographic encoding). Neat.

lifthrasiir12• 2 years ago

And yet didn't reach the Pareto frontier! It's quite obvious in hindsight though---QOI decoding is inherently sequential and can't be easily parallelized.

gaazoh2• 2 years ago

Of course it didn't, it wasn't designed to be either the fastest nor the best. Just OK and simple. Yet in some cases it's not completely overtaken by competition, and I think that's cool.

I don't believe QOI will ever have any sort of real-world practical use, but that's quite OK and I love it for it has made me and plenty of others look into binary file formats and compression and demystify it, and look further into it. I wrote a fully functional streaming codec for QOI, and it has taught me many things, and started me on other projects, either working with more complex file formats or thinking about how to improve upon QOI. I would probably never have gotten to this point if I tried the same thing starting with any other format, as they are at least an order of magnitude more complex, even for the simple ones.

phoboslab2• 2 years ago

As far a I understand this benchmark JXL was using 8 CPU cores, while QOI naturally only used one. If you were to plot the graph with compute used (watts?) instead of Mpx/s, QOI would compare much better.

Also, curious that they only benchmarked QOI for "non-photographic images (manga)", where QOI fares quite badly because it doesn't have palleted mode. QOI does much better with photos.

shdon• 2 years ago

GameMaker Studio has actually rather quickly jumped onto the QOI bandwagon, having 2 years ago replaced PNG textures with QOI (and added BZ2 compression on top) and found a 20% average reduction in size. So GameMaker Studio and all the games produced with it in the past 2 years or so do actually use QOI internally.

Not something a consumer knowingly uses, but also not quite irrelevant either.

pornel2• 2 years ago

Oof that looks like a confused mess of a format.

bz2 is obsolete. It’s very slow, and not that good at compressing. zstd and lzma beat it on both compression and speed at the same time.

QOI’s only selling point is simplicity of implementation that doesn’t require a complex decompressor. Addition of bz2 completely defeats that. QOI’s poorly compressed data inside another compressor may even make overall compression worse. It could heve been a raw bitmap or a PNG with gzip replaced with zstd.

JyrkiAlakuijala47• 2 years ago

That feels a little sad. If qoi had anything good (fast single-threaded decoding speed for photographic content) adding bz2 most certainly removed it. They could have just used WebP lossless and it would have been faster and smaller.

btdmaster2• 2 years ago

Missing from the article is rav1e, which encodes AV1, and hence AVIF, a lot faster than the reference implementation aom. I've had cases where aom would not finish converting an image in a minute of waiting what rav1e would do in less than 10 seconds.

JyrkiAlakuijala47• 2 years ago

Is rav1e pareto-curve ahead of libaom pareto-curve?

Does fast rav1e look better than jpegli at high encode speeds?

btdmaster2• 2 years ago

Difficult to know without reproduction steps from the article, but I would think it behaves better than libaom for the same quality setting.

Edit: found https://github.com/xiph/rav1e/issues/2759

pornel2• 2 years ago

rav1e is generally head to head with libaom on static images, and which one wins on the speed/quality/size frontier depends a lot on the image and settings, as much as +/- 20%. I suspect rav1e has an inefficient block size selection algorithm, so the particular shape of blocks is a make or break for it.

I’ve only compared rav1e to mozjpeg and libwebp, and at fastest speeds it’s only barely ahead.

jonsneyers13• 2 years ago

Both rav1e and libaom have a speed setting. At similar speeds, I have not observed huge differences in compression performance between the two.

jug4• 2 years ago

Pay attention to just how good WebP is at _lossless_ comparison though!

I've always thought that one as flying under the radar. Most get stuck on WebP not offering tangible enough benefits (or even worse) over MozJPEG encoding, but WebP _lossless_ is absolutely fantastic for performance/speed! PNG or even OptiPNG is far worse. And very well supported online now, and leaving the horrible lossless AVIF in the dust too of course.

jonsneyers13• 2 years ago

Lossless WebP is very good indeed. The main problem is that it is not very future-proof since it only supports 8-bit. For SDR images that's fine, but for HDR this is a fundamental limitation that is about as bad as GIF's limitation to 256 colors.

jug4• 2 years ago

Ah, I didn't know this and I agree this is a fairly big issue and increasingly so over time. I think smartphones in particular hastened the demand for HDR quite a bit, what was once a premium/enthusiast feature you only had to explicitly buy into.

omoikane• 2 years ago

I haven't ran across websites that serves up HDR images, I am not sure I would notice the difference. WebP seems appropriately named and optimized for image delivery on the web.

Maybe you are thinking of high bit depth for archival use? I can see some use cases there where 8-bit is not sufficient, though personally I store high bit depth images in whatever raw format was produced by my camera (which is usually some variant of TIFF).

chungy3• 2 years ago

Lossless WebP is also stuck with a low axis limit of 16383.

It is a good format when you can use it, but JPEG XL almost always compresses better anyway, and lacks color space and dimension limits.

kllrnohj8• 2 years ago

For things where you actually care about lossless you probably also don't care about HDR.

HDR is (or can be) good for video & photography, but it's absolutely ass for UI.

Besides, you can just throw a gainmap approach at it if you really care. Works great with jpeg, and gainmaps are being added to heif & avif as well, no reason jpegxl couldn't get the same treatment. The lack of "true 10-bit" is significantly less impactful at that point

JyrkiAlakuijala47• 2 years ago

Thank you! <3

WebP also has a near-lossless encoding mode based on lossless WebP specification that is mostly unadvertised, but should be preferred over real lossless in almost every use case. Often you can half the size without additional visible loss.

netol• 2 years ago

Is this mode picked automatically in "mixed" mode?

Unfortunately, that option doesn't seem to be available in gif2webp (I mostly use WebP for GIF images - as animated AVIF support is poor on browsers and that has an impact in interoperability)

kurtextrem• 2 years ago

do you know why Jon didn't compare near-lossless in the "visually lossless" part?

a-french-anon4• 2 years ago

An issue with lossless WebP is that it only supports (A)RGB and encodes grayscale via hacks that aren't as good as simply supporting monochrome.

If you compress a whole manga, PNG (via oxipng, optipng is basically deprecated) is still the way to go.

Another something not mentioned in here is that lossless JPEG2000 can be surprisingly good and fast on photographic content.

edflsafoiewq6• 2 years ago

IIRC the way you encode grayscale in WebP is a SUBTRACT_GREEN transform that makes the red and blue channel 0 everywhere, and then use a 1-element prefix code for R and B, so the R and B for each pixel take zero bits. Same idea with A for opaque images. Do you know why that's not good enough?

out_of_protocol• 2 years ago

Just tried it on random manga page

- OxiPNG - 730k

- webp lossless max effort - 702k

- avif lossless max effort - 2.54MB (yay!)

- jpegxl lossless max effort - 506k (winner!)

Akronymus• 2 years ago

I really like webp. Sadly there's still a lot of applications that dont work with it (looking at discord)

ocdtrekkie• 2 years ago

It is ironic you said this because when I disabled webp in my browser because it had a huge security vulnerability, Discord was the only site which broke and didn't immediately just serving me more reasonable image formats.

kodabbb3• 2 years ago

Looks like there are more savings coming on lossless AVIF: https://www.reddit.com/r/AV1/comments/1b3lh08/comment/kstmbr...

JyrkiAlakuijala47• 2 years ago

Also more savings will come for JPEG XL soon.

Possibly mostly focused on medium and low quality.

ummonk• 2 years ago

I'm actually shocked by how poorly the lossless versions of new image formats (AVIF and HEIC) perform compared to the venerable png.

KingOfCoders3• 2 years ago

But all compression in the graph is lossless. The whole article is basically about JPEG XL lossless. Am I missing something?

artemisart• 2 years ago

About 3/4 of the article? There is a lot of analysis on lossy compression too after the lossless part.

account426• 2 years ago

Last time I tested cwebp it did not handle (PNG) color spaces correctly so the result of a supposedly lossless conversion actually looked different from the original. What good is better lossless compression if it is not actually lossless visually?

jug4• 2 years ago

Wow, that new jpegli encoder. Just wow. Look at those results. Haha, JPEG has many years left still.

kasabali8• 2 years ago

JPEG has many years left still

Such a shame arithmetic coding (which is already in the standard) isn't widely supported in the real world. Because converting Huffman coded images losslessly to arithmetic coding provides an easy 5-10% size advantage in my tests.

Alien technology from the future indeed.

mort968• 2 years ago

The benefits of JPEG kind of go away if you start adopting more recent changes to the format, no? JPEG is nice because everything has supported it for 20+ years. JPEG-with-arithmetic-coding is essentially a new, incompatible format, why not use JXL or AVIF instead?

dancemethis• 2 years ago

"Pareto" being used outside the context of Brazil's best prank call ever (Telerj Prank) will always confuse me. I keep thinking, "what does the 'thin-voiced lawyer' have to do with statistics?"...

MikeCapone• 2 years ago

I really hope this can become a new standard and be available everywhere (image tools, browsers, etc).

While in practice it won't change my life much, I like the elegance of using a modern standard with this level of performance an efficiency.

TacticalCoder• 2 years ago

Without taking into account whether JPEG XL shines on its own or not (which it may or may not), JPEG XL completely rocks for sure because it does this:

    .. $  ls -l a.jpg && shasum a.jpg
    ... 615504 ...  a.jpg
    716744d950ecf9e5757c565041143775a810e10f  a.jpg

    .. $  cjxl a.jpg a.jxl
    Read JPEG image with 615504 bytes.
    Compressed to 537339 bytes including container

    .. $  ls -l a.jxl
    ... 537339 ... a.jxl

But, wait for it:

    .. $  djxl a.jxl b.jpg
    Read 537339 compressed bytes.
    Reconstructed to JPEG.

    .. $  ls -l b.jpg && shasum b.jpg
    ... 615504 ... b.jpg
    716744d950ecf9e5757c565041143775a810e10f  b.jpg

Do you realize how many billions of JPEG files there are out there which people want to keep? If you recompress your old JPEG files using a lossy format, you lower its quality.

But with JPEG XL, you can save 15% to 30% and still, if you want, get your original JPG 100% identical, bit for bit.

That's wonderful.

P.S: I'm sadly on Debian stable (12 / Bookworm) which is on ImageMagick 6.9 and my Emacs uses (AFAIK) ImageMagick to display pictures. And JPEG XL support was only added in ImageMagick 7. I haven't looked more into that yet.

izacus7• 2 years ago

I'm sure that will be hugely cherished by users which take screenshots of JPEGs so they can resend them on WhatsApp :P

F3nd03• 2 years ago

This particular feature might not, but if said screenshots are often compressed with JPEG XL, they will be spared the generation loss that becomes blatantly visible in some other formats: https://invidious.protokolla.fi/watch?v=w7UDJUCMTng

JyrkiAlakuijala47• 2 years ago

I managed to add that requirement to jpeg xl. I think it will be helpful to preserve our digital legacy intact without lossy re-encodings.

kodabbb3• 2 years ago

Looks like there are more savings coming on lossless AVIF: https://www.reddit.com/r/AV1/comments/1b3lh08/comment/kstmbr...

JyrkiAlakuijala47• 2 years ago

I plan to add 15–25 % more quality in the ugly lowest end quality in JPEG XL in the coming two months.

JyrkiAlakuijala47• 2 years ago

It is worth noting that the JPEG XL effort produced a nice new parallelism library called Highway. This library is powering not only JPEG XL but also Google's latest Gemma AI models.

jhalstead• 2 years ago

[0] for those interested in Highway.

It's also mentioned in [1], which starts off

Today we're sharing open source code that can sort arrays of numbers about ten times as fast as the C++ std::sort, and outperforms state of the art architecture-specific algorithms, while being portable across all modern CPU architectures. Below we discuss how we achieved this.

[0] https://github.com/google/highway

[1] https://opensource.googleblog.com/2022/06/Vectorized%20and%2..., which has an associated paper at https://arxiv.org/pdf/2205.05982.pdf.

janwas• 2 years ago

:) Thanks for the mention. Highway/vqsort TL here, happy to discuss.

PS: I used to work on JPEG XL. It is great to see these outstanding improvements, congrats to the team!

a-french-anon4• 2 years ago

And VIPS! It seems like the best way to get portable SIMD in C++, to me.

sandstrom• 2 years ago

JPEG XL is awesome!

One thing I think would help with its adoption, is if they would work with e.g. the libvips team to better implement it.

For example, streaming encoder and streaming decoder would be the preferred integration method in libvips.

ImageXav2• 2 years ago

Maybe someone here will know of a website that describes each step of the jpeg xl format in detail? Unlike for traditional jpeg, I have found it hard to find a document providing clear instructions on the relevant steps, which is a shame as there are clearly tons of interesting innovations that have been compiled together to make this happen, and I'm sure the individual components are useful in their own right!

adgjlsfhk15• 2 years ago

https://github.com/libjxl/libjxl/blob/main/doc/format_overvi... is a pretty detailed but good overview. The highlights are variable size DCT (up to 128x128), ANS entropy prediction, and chroma from luminance prediction. https://github.com/libjxl/libjxl/blob/main/doc/encode_effort... also gives a good breakdown of features by effort level.

ImageXav2• 2 years ago

Thank you for sharing. This gives me a good idea of where to start looking.

jeffbee• 2 years ago

The choice to represent the speed based on multithreaded encoding strikes me as somewhat arbitrary. If your software has a critical path dependent on minimal latency of a single image, then it makes some sense, but you still may have more or fewer than 8 cores. On the other hand if you have another source of parallelism, for example you are encoding a library of images, then it is quite irrelevant. I think the fine data in the article would be even more useful if the single threaded speed and the scalability of the codec were treated separately.

redder23• 2 years ago

AVIF looks better here: JPEG XL looks very blurred out on the bottom with high compression. AVIF preserves much more detail and sharpness.

https://res.cloudinary.com/jon/qp-low.png

jiggawatts3• 2 years ago

The problem with JPEG XL is that it is written in an unsafe language and has already had several memory safety vulnerabilities found in it.

Image codecs are used in a wide range of attacker-controlled scenarios and need to be completely safe.

I know Rust advocates sound like a broken record, but this is the poster child for a library that should never have been even started in C++ in the first place.

It’s absolute insanity that we write codecs — pure functions — in an unsafe language that has a compiler that defaults to “anything goes” as an optimisation technique.

lonjil27• 2 years ago

Pretty much every codec in every browser is written in an unsafe language, unfortunately. I don't see why JXL should be singled out. On the other hand, there is a JXL decoder in Rust called jxl-oxide [1] which works quite well, and has been confirmed by JPEG as conformant. Hopefully it will be adopted for decode-only usecases.

[1] https://github.com/tirr-c/jxl-oxide/pull/267

It’s absolute insanity that we write codecs — pure functions — in an unsafe language that has a compiler that defaults to “anything goes” as an optimisation technique.

Rust and C++ are exactly the same in how they optimize, compilers for both assume that your code has zero UB. The difference is that Rust makes it much harder to accidentally have UB.

jiggawatts3• 2 years ago

"We've never had to wear helmets before, why start now?"

There are only a handful of image codecs that are widely accepted. Essentially just GIF, PNG, and JPG. There's a smattering of support for more modern formats, but those three dominate.

Adding a fourth image format is increasing this attack surface by a substantial margin across a huge range of software. Not just web browsers, but chat apps, server software (thumbnail generators), editors, etc...

This is the kind of thing that gets baked into standard libraries, operating systems, and frameworks. It's up there with JSON or XML.

You had better be damned sure what you're doing is not going to cause a long list of CVEs!

JPEG XL is a complex codec, with a lot of code. This increases the chance of bugs and increases the attack surface.

A (surprisingly!) good metric for complexity is the size of the zip file of the code. Libjpeg is something like 360 kB, libpng is 350 kB, and giflib is 90 kB.

The JXL source is 1.4 MB zipped, making more than twice the size of the above three combined!

The other libraries use C/C++ not because that's a better choice, but because it was the only choice back in the ... checks Wikipedia ... 1980s and 90s!

We live in the future. We have memory-safe languages now. We're allowed to use them. You won't get in trouble from anyone, I promise.

account426• 2 years ago

an unsafe language

No such thing.

a compiler that defaults to “anything goes” as an optimisation technique

That's just FUD.

jshier2• 2 years ago

I have an existing workflow where I take JPEGs (giant PNGs) from designers and reencode them using mozjpeg. However, I can't find a way to invoke jpegli tool in the same way, especially since it seems to just be part of the jpeg-xl tool? Is that right? Are there any sample invocations anywhere?

chungy3• 2 years ago

You should be able to use the `cjpegli` command the same way you'd use cjxl. The simplest invocation would be `cjpegli input.png output.jpg`

JyrkiAlakuijala47• 2 years ago

Or replace/ldconfig the jpeg library with the jpegli library. It is API/ABI compatible with libjpeg-turbo and mozjpeg.

jshier2• 2 years ago

Thanks. It seems the macOS release is a bit behind on brew (0.9.1) and doesn't include the cjpegli tool yet.

ksec• 2 years ago

It is nice 0.10 finally landed those memory and speed optimisation.

But the King remains HALIC. In terms of MT encoder it still uses 3.5x more memory than HALIC, and 6x encoding time compared to HALIC. While offering the same or smaller files size in Lossless. Hopefully JPEG XL could narrow those gaps some days.

JPEG XL and the Pareto Front

Related Stories