Why Apple Uses JPEG XL in the iPhone 16 and What It Means for Your Photos

https://petapixel.com/2024/09/18/why-apple-uses-jpeg-xl-in-the-iphone-16-and-what-it-means-for-your-photos/

by alwillis • 2 years ago

209 156 2 years ago

Reading the whole piece a few days ago, it's a pretty good overview of the promises of JPEG XL.

Apart from that, Apple's POV and PR bits being given such a central role felt a bit weird, especially as petapixel already spotted Samsung adopting JPEG XL months before Apple.

Aside from the petty "who was first" bickering, it's a completely different move to adopt a common standard already accepted by rival companies on the android side, and it means we can really expect a larger adoption of JPEG XL than the other standards Apple just pitched on its own.

That was the biggest beacon of hope IMHO, it would have benefited from more prominence.

simondotau• 2 years ago

Attaching any significance to being “first” on open standards is a game Apple rarely plays, but which others impose upon them because Apple’s adoption is (rightly or wrongly) seen as the most consequential and/or most newsworthy inflection point.

makeitdouble2• 2 years ago

In the specific case of JPEG XL I think Google support will be the real inflection point: no chrome and default android support is a deal breaker for wide audience content publishers.

The ironic part being of course that Chrome used to support it way too early, but support was dropped as nobody followed. So yes, Apple support is a big deal, but not more consequential than the other actors of the pretty vast ecosystems.

ksec5• 2 years ago

In terms of JPEG XL Lossless, Camera makers are already or has plan to adopt the format as RAW. In that space I think the adoption roadmap is quite good.

alwillis3• 2 years ago

… especially as petapixel already spotted Samsung adopting JPEG XL months before Apple.

JPEG XL was added to iOS 17 and macOS 14 last year [1].

[1] https://developer.apple.com/wwdc23/10122

macinjosh• 2 years ago

The article states that because the file sizes are smaller the format is more environmentally friendly because, they state, "All that stuff lives somewhere, and wherever it is, it requires energy to operate."

Cold storage exists, as well as different tiers of storage. The hard drive on my shelf isn't using any energy. Are they storing all of their images in RAM? Maybe you could say this would lead to less use of storage space so less use of raw materials for storage devices.

I would posit that it is possible this format is less environmentally friendly because it takes more compute cycles to produce the output from the compressed data, but I have no real insight into this just intuition.

r00fus• 2 years ago

On the other hand, we have industry defined standards to compress TCP headers [1] - incurring a cost to compress/decompress but resulting in lower packet sizes.

[1] https://en.wikipedia.org/wiki/Robust_Header_Compression

threeseed• 2 years ago

Most iPhone users would be storing photos in iCloud.

And most professionals are storing their photos in Creative Cloud.

And in both cases the photo data would be replicated on multiple hard drives for redundancy. So it would definitely be more environmentally friendly to have smaller photos.

The more important reason is that cloud storage costs significantly more than local.

k3102• 2 years ago

What is environmentally (OK, mentally) unfriendly to me is the creation of new formats that various apps, and notably, websites where I post images, can't handle, so I swear a bit, convert the image to jpeg or png, and post.

People are part of the environment, too!

But lo! men have become the tools of their tools. ... Thoreau.

astrange4• 2 years ago

You wouldn't like a website that accepted raw phone photos, because they're in HDR, and you'd get blinded by looking at them.

Instagram already does this to me when I look at it at night, because it accepts HDR video.

furyofantares• 2 years ago

The hard drive on my shelf isn't using any energy.

When it filled up you put it on a shelf and bought a new one.

kybernetyk• 2 years ago

That's essentially what I do with my RAW files. One hard drive per year. It's still less expensive than every cloud storage I looked at.

xxs• 2 years ago

Are they storing all of their images in RAM?

That part would not matter regardless. RAM is powered at all times, kept being refreshed too. Unless, of course you mean, they'd have to keep buying new machines to store more pictures.

jessekv• 2 years ago

Are they storing all of their images in RAM?

Consider the millions of iPhone users uploading new photos and scrolling their iCloud-hosted photo library. A substantial amount of photo data is in RAM at any given moment, spread across many network devices.

I would posit that it is possible this format is less environmentally friendly because it takes more compute cycles

I'm curious how much energy it takes to send bits over cellular or wifi, my intuition is that this is orders of magnitude higher than compression or encoding.

astrange4• 2 years ago

I'm curious how much energy it takes to send bits over cellular or wifi, my intuition is that this is orders of magnitude higher than compression or encoding.

Correct, but the computational cost of decoding a codec is generally correlated with the number of bits. So JXL is not necessarily more expensive to decode than older codecs, if it compresses better.

the_gorilla• 2 years ago

If people are going to take credit for crunching something down from 10 MB to 8 MB, I wish they'd also take the blame for massively bloating something into a modern monstrosity out of sheer laziness or betting that profit margins will be higher if they waste user time, storage, and cpu for the sake of releasing the project faster.

semi-extrinsic• 2 years ago

This whataboutism we are hearing now about "the energy needed to store your pictures" is just corporate whataboutism, trying to push blame onto consumers.

Let's say you have 2 TB of photos stored. If that's consuming more than 2 watts of electricity on average, the cloud providers have to be quite incompetent (they are mostly not).

2 watts is 1 kWh in 20 days, so 18 kWh in one year. The emissions from 18 kWh in the US is around 6 kg of CO2. This is the equivalent of driving a car for about 20 minutes on the highway, one time per year. Or spending 5 minutes longer in the shower 4 times per year.

teddyh• 2 years ago

The hard drive on my shelf isn't using any energy

But it is losing its data: <https://news.ycombinator.com/item?id=41244143>

illiac7866• 2 years ago

You encode the image once, but transmit it many, many times, in general. It it send from phone to iCloud, then from iCloud to your other apple device and so and so on. God forbid you share it with someone. Then iCloud has internal resiliencies and redundancies, etc. etc.

Short point: I do think it’s environmentally better (or less worse). it’s a drop in the ocean though.

scosman2• 2 years ago

Thank god they went with a standard this time. When they launched HEIC, there wasn’t a single workable open source decoder. Hell, there wasn’t even a single non-Apple decoder.

XL color depth looks amazing.

rgovostes• 2 years ago

An annoying oversight is that while my Fujifilm camera is modern enough to shoot HEIF+RAW, Apple Photos only knows to group JPEG+RAW as a single photo. Because Apple did not spend a day of engineering time bringing feature parity for the file format they themselves promoted, it has turned into a bigger feature to match and merge the HEIF and RAW assets after the fact. After several years, I'm growing doubtful they'll ever accomplish it.

I have yet to see whether they did it right with JXL+RAW (or is it DNG+RAW?) but hopefully they will before it becomes available in mainstream cameras.

lonjil11• 2 years ago

Apple's ProRAW format is Linear DNG, that is to say, already debayered, not what most people would call "raw". Previously they've used Lossless JPEG for compression inside these DNG files, but now they use JXL.

The DNG spec also allows for using JXL to compress proper raws, but I don't know if anyone is doing this yet. I know Samsung uses JXL compression in DNGs produced by their phones, but I haven't checked if those are proper raws or debayered.

masklinn2• 2 years ago

Thank god they went with a standard because when they went with a standard it wasn’t widespread?

What?

happyopossum• 2 years ago

HEIC is a standard too - it wasn’t a secret internal Apple project…

zenexer• 2 years ago

It might be a standard, but for a long time the licensing costs were exorbitant, and that likely stifled adoption. While licensing costs have come down, the pushback against HEIC’s pricing led to the development of better, royalty-free alternatives—including JPEG XL. Thank god they went with an unencumbered standard this time.

npteljes2• 2 years ago

Yes, OP worded this a bit incorrectly. Open source, royalty-free standard would be better.

scosman2• 2 years ago

Not really. It was a 2 company colab with a pseudo spec. But their implementation had quirks that weren’t in the spec, and weren’t documented anywhere.

We spend a week building a decoder and kept finding new undocumented bits.

I wouldn’t call it a standard when zero folks outside Apple had access a reliable decoder at launch.

weiliddat2• 2 years ago

Is part of the reason patents / licensing issues with HEVC that makes it harder to adopt?

masklinn2• 2 years ago

They already adopted it 7 years ago so no.

weiliddat2• 2 years ago

I don’t mean Apple but other parties

b15h0p• 2 years ago

To increase adoption they should not have limited this to the latest iPhone models. Why on earth can a one year old iPhone 15’s CPU not handle encoding JXL? It can encode 4K video in real time, so this should be no problem at all, right?

NinoScript• 2 years ago

I’d guess hardware acceleration could have something to do with it

JyrkiAlakuijala5• 2 years ago

I don't know, but my guesswork is that the DNG/ProRAW/JXL support comes with compatibility challenges. Limiting the size of the launch to well-informed photography prosumers and professionals will help to iron out the compatibility challenges — rather than make all confused consumers face these challenges at once.

I don't think that hardware support plays a role here. The fastest encoding modes of JPEG XL are ridiculously fast on software, and Apple's CPUs seem powerful enough.

the4anoni• 2 years ago

This, I just don't understand why it seems only latest iPhones got this.

rob74• 2 years ago

I also don't know, but I suspect the fact that Apple not only develops the OS, but also sells the devices, might have something to do with it...

brigade5• 2 years ago

The CPU isn’t used for encoding video

LtdJorge• 2 years ago

Encoding a single image with the CPU takes nothing compared to modern video codecs.

lonjil11• 2 years ago

There's literally no way Apple developed a hardware accelerator for JXL encoding. So they're definitely using some kind of a CPU or generic DSP.

cchi_co• 2 years ago

It does seem odd to restrict new features to the latest models, especially when older ones still have powerful capabilities.

Asmod4n2• 2 years ago

It’s good in a corporate setting, you don’t want to suddenly have to deal with a new file format for which you won’t have an app installed on your company PCs to view them.

alwillis3• 2 years ago

Apple announced support for JPEG XL last year at WWDC; Apple's 1st party apps supported JPEG XL when iOS 17 shipped: https://developer.apple.com/wwdc23/10122

Regarding older hardware, I addressed that: https://news.ycombinator.com/item?id=41611709

WhyNotHugo• 2 years ago

This is how they sell new phones. The grand majority of new features don't require the latest hardware, but artificially restricting it increases sales of new phones. None technical people can't usually tell the difference between hardware and software clearly enough to understand this nasty trick.

illiac7866• 2 years ago

To be fair it’s less bad than what some other vendors are doing, which is stopping to provide updates. Not saying apple is an angel, they intentionally generate ewaste by withholding such features…

marcellus23• 2 years ago

I don't think anyone is deciding to buy a new phone or not based on JPEG XL support.

jiggawatts• 2 years ago

One of the tricks for achieving the target battery life is that photo and video formats are offloaded to dedicated and very power-efficient hardware in all mobile devices. The iPhone 16 is the first to get hardware offload for AV1 and JXL, which is why it'll support these formats.

It's not just software, unlike in the PC world where going from 5W hardware decode to 50W software decode basically doesn't matter.

illiac7866• 2 years ago

Is that documented somewhere?

account423• 2 years ago

The "CPU" (that is, the generic compute part) cannot encode 4K video in real time, that part is handle by codec-specific HW.

lonjil11• 2 years ago

They aren't doing JXL encoding in hardware though. Zero chance that the new iPhone chip has hardware acceleration for JXL. Definitely just plain old CPU encoding.

And we're not talking about video anyway, this is about ProRAW, a still image format.

FollowingTheDao4• 2 years ago

This makes me think: If JPEG-XL needs more computing power to decompress, does that out weigh the ecological benefit benefit of the smaller file size?

pornel5• 2 years ago

JPEG XL is pretty cheap to decompress.

Advancements in compression algorithms also came with advancements in decompression speed. New algorithms like tANS are both compressing well, and have very fast implementations.

And generally smaller files decompress faster, because there's just less data to process.

alwillis3• 2 years ago

From the article:

    As Apple explains on the new iPhone models, JPEG XL files are supported
    on iOS 17 and later and macOS 14 and later.

JPEG XL isn't limited to the latest phones; just a phone that can run iOS 17 or later. I have used JPEG XL on my iPhone 13 mini with no issues. iOS 17 runs on the iPhone XS (2018) or newer.

The difference is JPEG XL is now part of the Apple's image pipeline for the camera in iPhone 16.

Any 3rd party photo app developer can support JPEG XL if they wish.

sho• 2 years ago

Some more recent developments around browser support: https://www.phoronix.com/news/Mozilla-Interest-JPEG-XL-Rust

MrAlex94• 2 years ago

Weirdly enough, JPEG-XL support is actually fairly decent in Firefox already and there were patches developed by the community that work well for things such as color profiles, animation support etc. I’ve had them in Waterfox for a few years now - it was a purely “political” decision, if you want to call it that, to stop any more progress until recently.

iforgotpassword• 2 years ago

Man this back and forth is really frustrating.

So Google killed XL a while ago already and I feel like either Microsoft or Mozilla at least considered following suit. After Apple has done heic for a while now I assumed it might go that way regarding a jpeg replacement, but now they did a 180 and switched to xl. I mean good, it's not as patent encumbered, but wtf am I supposed to expect for the future now? Will Google add XL back to chrome? I guess it will take another decade or five until we have a jpeg replacement that's being universally agreed upon, because come on, it would be too easy if we don't get another plot twist where a major player jumps onto something else again for a while.

lonjil11• 2 years ago

So Google killed XL a while ago already and I feel like either Microsoft or Mozilla at least considered following suit.

There's been JXL stuff visible in Windows 11 preview builds for months now. They might roll out support next year.

JyrkiAlakuijala5• 2 years ago

Use Jpegli for now. You can get half the benefits of JPEG XL with it.

Washuu• 2 years ago

Firefox Nightly is listed as supported on the jpegxl.info web site. My understanding is that it does not fully support the specification yet which is why it is not released to production.

https://jpegxl.info/resources/supported-software.html

lencastre• 2 years ago

Cries in iPhone11Pro … also WTF!? Why not make QOK format and x266 available and exclusive to iPhone17

TiredOfLife• 2 years ago

For one you can't use Chrome to view those photos.

note: the format was co-developed by Google who also makes Chome.

swiftcoder2• 2 years ago

Indeed, but Chrome removed support for the format some time ago.

lonjil11• 2 years ago

I don't think you can use Chrome to view any raw image files...

0x69420• 2 years ago

please tell me this means chromium will un-drop jxl and we can just stick them on the web like png/jpg/gif

modeless2• 2 years ago

these .jxl files are wrapped in a DNG container, so you can’t just fire off .jxl files from the iPhone 16 Pro.

Any move toward JPEG XL support is good, but this is lame. Even if the Chrome team comes to its senses and restores jxl support you won't be able to view these files.

brigade5• 2 years ago

Of course it’s .dng; .jxl doesn’t carry the metadata needed to process a RAW image because it’s not intended for that.

USiBqidmOOkAqRb2• 2 years ago

Which metadata exactly? Is the actual concern that someone unaware might accidentally stumble on the unprocessed image?

illiac7866• 2 years ago

Hmmm, is the jpeg xl data inside the dng not demosaicized?

pornel5• 2 years ago

You wouldn't want 24MB files with GPS and face detection metadata served raw on the Web anyway.

contravariant• 2 years ago

I wouldn't want them on my phone either, but maybe that's just me.

modeless2• 2 years ago

Serving images from CDNs is not the only use case for browser JXL support. Browsers can perform client side processing of images before upload. They even host fully featured image editors like https://photopea.com. And sometimes you do want to see an original image from your camera in your browser, like say on the web version of Google Photos.

lonjil11• 2 years ago

Adobe added JXL as a compression option for the DNG format last year. Previous options were JPEG, Lossless JPEG, Deflate, and nothing. Apple used Lossless JPEG. Apple engineers have decided to take advantage of the option Adobe added last year to make their DNG files smaller.

nesk_• 2 years ago

That’s good news. I’ve watched a really good video in the last weeks about JPEG XL advantages, if you want to learn a bit more: https://youtube.com/watch?v=FlWjf8asI4Y

hajhatten• 2 years ago

Was just about to see if someone mentioned this video. Really good explanation comparing JPEG XL vs AVIF.

trompetenaccoun2• 2 years ago

This may be a naive question but can't we have the features of JPEG XL combined with the compression algorithm of AVIF? Why does it have to be one or the other? Size clearly matters, especially for the majority of the world's population who do not have access to super fast internet and TBs of storage. It's definitely a luxury not having to care about file sizes.

pif• 2 years ago

Buying Apple is a luxury in itself: don't expect them thinking about the non rich.

account423• 2 years ago

Because there is no point, AVIF does not have inherently superior compression to JPEG XL.

lonjil11• 2 years ago

AVIF is worse than JXL both for high fidelity compression and for lossless compression, which is what Apple is using JXL for here.

larrysalibra• 2 years ago

How does JPEG XL compare to Apple’s current default HEIC? Is HEIC eventually going away in favor of JPEG XL?

lonjil11• 2 years ago

They aren't switching from HEIC to JXL. Apple uses HEIC for regular photos, but they use DNG with Lossless JPEG for "ProRAW". They are switching the latter to use JXL inside DNG instead of L-JPEG.

dgreensp• 2 years ago

Ok wow, that was not clear from the article. It also seems kind of confusing and unfortunate that Apple is using so many different image formats.

praseodym• 2 years ago

JPEG XL also supports re-encoding existing JPEG files to decrease file size while keeping the original file quality. That really seems like useful feature but so far I haven’t seen any tooling (in macOS) to re-encode my existing photo library.

sureIy• 2 years ago

It would be safe to assume that Apple will eventually add a way to recompress your photo library to JXL… if they weren’t in the business of selling storage and cloud storage. They have in the past released tools to optimize storage so it wouldn’t be completely out of the ordinary, but… I wouldn’t hold my breath.

appendix-rock3• 2 years ago

Sigh. I’ll happily hold my breath. Apple has done plenty to reduce use of storage. They even give you free iCloud storage to back your phone up when transferring to a new device. A very clear attractive source of penny pinching that they’ve put effort into to leaving on the table. This is tiring.

CorrectHorseBat• 2 years ago

Not just the original file quality, it's exactly the same image but (losslessly) compressed better.

TacticalCoder• 2 years ago

but so far I haven’t seen any tooling (in macOS) to re-encode my existing photo library

On Linux you can re-encode every single .jpg to .jxl and they'll decode bit-for-bit, to the original .jpg.

On Debian and derivatives it's the libjxl-tools: the cjxl executable converts to .jxl and the djxl decompresses.

Works flawlessly.

P.S: as a sidenote there's zero reason anymore to serve a .jpg file to a browser when the browser supports .jxl files. That's just a waste of bandwith. (and if your stack cannot serve different files depending on the users' browsers' capacities, it's not much of a stack)

illiac7866• 2 years ago

It’s actually even better, it’s supposed to be reversible. You want you jpeg file back? There you go.

compressedgas• 2 years ago

This is why I consider JPEG XL to be the standardized successor to Dropbox's Lepton https://github.com/dropbox/lepton

mihaaly• 2 years ago

Yes, yes, but 48mP only, when will they finally have 240MP on a sensor that can be mounted on the ass of an ant. But double the protrusion 6 times I guess for the lens supporting the new tech and whatnot (tele-macro at night in a panoramic sport event!), it may even take picture for you without thinking taking a picture, you purchased a smartphone didn't you, let it be smart then, it will know better than you what you need, how you need it, and when you need it, all is necessary to fix it to your forehead so it can see what you see, tiny bitsy inconvenience beyond storing and managing the billboard sized but polaroid quality stream of pics vomited out by the device - never to see the most.

This good photo = phone deception marketers pushed on idiotic customers who actually pay the premium for the marketing material is pathetic.

cherioo• 2 years ago

I don’t have iphone 16, and this article puzzles me.

Is apple only using jxl for their “raw” camera capture, but not regular camera capture?? The non-raw use case seems to be the one that would have more impact to regular folks.

Why? Is jxl inferior to HEIC?

oktoberpaard• 2 years ago

ProRAW files are large, so there's more potential to save space, making ProRAW a more attractive feature to use for a wider audience.

brigade5• 2 years ago

DNG spec added JPEG-XL as their modern codec. Both lossy and lossless modes are significantly better than the other 3 allowed codecs (JPEG, lossless JPEG, and zip.)

7e• 2 years ago

I can’t wait for animated JPEG XL to replace animated GIF.

edflsafoiewq2• 2 years ago

You may be waiting a while. APNG and animated WebP are already widely rolled out, but haven't replaced GIF.

account423• 2 years ago

Neither of these are well supported at all. APNG has a whole history of non-support by major browsers and animated WebP doesn't even have its own mime type so you can't fall back to gif/apng if support is not there - and support for webp does not guarantee support for animated webp. Both are also decidedly mediocre improvements over gif (which is quite the accomplishment considering how shit gif compression is).

Really, animated WebP has little reason to exist. At least the lossy version should really just be a video file with no audio track and a convention for the loop metadata. Too bad that browsers decided to create yet another format (with intentionally crippled compression compared to webm) rather than allow silent videos in <img> tags.

pornel5• 2 years ago

Animated AVIF is widely supported, and can represent GIFs losslessly.

BTW, Chrome vetoed the idea of supporting muted video files in `<img>` like Safari does, so we've got this silly animated AVIF that is a video format that has been turned into a still image format that has been turned back into a video format, which takes regular AV1 video data but packages it in a slightly different way, enough to break video tooling.

lonjil11• 2 years ago

Animated AVIF is widely supported, and can represent GIFs losslessly.

Doesn't lossless AVIF have terrible compression ratios?

bmicraft3• 2 years ago

You could already use apng and the file size isn't too bad.

bmicraft3• 2 years ago

Well, most "GIFs" already really aren't that. Most image hosters already convert to and serve a silent looping h264 in it's place.

thecosmicfrog• 2 years ago

Compared to a standard JPEG, a JPEG XL file is up to 55% smaller

I still find JPEG "XL" to be such a bizarre name. I would intuitively think it would result in larger file sizes.

oneeyedpigeon• 2 years ago

Although it might seem confusing at first glance, having your selling point as "our file sizes are larger!" is so counterintuitive, that I think it's obviously not that!

hootz• 2 years ago

Sometimes it can be like "our file sizes are larger, but look at this big list of cool features".

Iulioh• 2 years ago

Bigger is better, it is known.

1000100_1000101• 2 years ago

Lets just call it JPEG 40 to avoid any confusion.

bmicraft3• 2 years ago

The XL probably relates to the fact that the relative savings increase with higher res pictures. Whereas original jpeg's file size might have scaled linearly, jxl could do better than that.

illiac7866• 2 years ago

Even weirder, L stands for legacy.

amy-petrik-214• 2 years ago

JPEG XL is a great name from a marketing perspective. The Apple roadmap has several successor compressors with AI elements in the planning sheet - JPEG XL PLUS, JPEG XL MAX. These formats will have small nominal yearly licensing fees to use. Finally, the most advanced format will be JPEG XXXL which they are developing in collaboration with pornhub.

zuhsetaqi• 2 years ago

What exactly has Apple to do with the name of JPEG XL?

astrange4• 2 years ago

Isn't it a little early to write this article? They haven't tried the implementation yet.

dsego• 2 years ago

Compared to JPEG XL, HEIC — an implementation of HEIF — is just not good.

I would've loved an explanation with this statement.

EasyMark• 2 years ago

Isn’t Samsung generally ahead of apple on most technical fronts though? Especially on mobile, I think people usually just assume apple just spends more effort on refining it, which is not a bad rule of thumb, but certainly not always the truth.

Why Apple Uses JPEG XL in the iPhone 16 and What It Means for Your Photos

Related Stories