Bringing Wide Color to Instagram

https://engineering.instagram.com/bringing-wide-color-to-instagram-5a5481802d7d#.44soiw7he

by natthub • 10 years ago

375 117 10 years ago

engineering.instagram.com

mikeyk10• 10 years ago

Here if anyone has questions on the implementation or such.

sosuke• 10 years ago

I this why iPhone photos of red roses, or other vibrant roses look poor? I love the idea of a photo room is it hand painted?

therealmarv6• 10 years ago

Yes they look poor if you watch them on not color managed software environments (should not happen on Apple devices). See here a sample image https://code.google.com/p/android/issues/detail?id=225281

realschnabler• 10 years ago

Not necessarily on topic - but given your code samples are written in Objective-C, how much of your codebase is still in Objective-C and what's your opinion on porting it over to Swift?

perfmode• 10 years ago

The swift compiler is probably too slow for projects as large as instagram.

mikeyk10• 10 years ago

Good q. All of it is still Objective C and C++; there's a few blockers to starting to move to Swift, including the relatively large amount of custom build + development tooling that we and FB have in place.

vanderZwan2• 10 years ago

Bit of a jump in topic, but I'm kind of curious: I don't use instagram myself, bugt I'm sure it resizes images for online viewing, saving bandwidth ans such. Does it do so with correct gamma[0]? Since that's a thing many image-related applications get wrong.

[0] http://blog.johnnovak.net/2016/09/21/what-every-coder-should...

mark-r• 10 years ago

If it's using Pillow for the resizing, probably not. I've not looked at Pillow specifically, but PIL certainly wasn't very good.

ec109685• 10 years ago

They don't store full resolution images: Square Image: 1080px in width by 1080px in height

Vertical Image: 1080px in width by 1350px in height

Horizontal Image: 1080px in width by 566px in height

aus_• 10 years ago

Is Wide Color preserved when processing video?

mikeyk10• 10 years ago

Photos only. Apple's APIs only capture in Wide Color when shooting in photo mode, and their documentation only recommends using wide color/Display P3 for images.

colanderman• 10 years ago

Do you test with Adobe RGB (which many prosumer cameras can output) or only P3 (which AFAIK only Apple devices output)?

mikeyk10• 10 years ago

Just P3, though the wider gamut available in our graphics operations should benefit photos brought in using Adobe RGB too since iOS is fully color managed.

terhechte• 10 years ago

I know there's not much love for the API these days, but is / will there be a way to access wide color images from the API? iPads and Macbook Pros also have wide color screens nowadays, so it would make sense to display them for specific use-cases in third party clients.

Nav_Panel• 10 years ago

Great article. Question not directly related to your writeup: in your casual usage of a phone/photo app with the new, wider color space, do you notice a difference in your experience of using the app/looking at images? Or, in other words, does the wider color space feel at all different?

dTal• 10 years ago

"When we started this project, none of us at IG were deep experts in color."

This is pretty astonishing to me. I always thought applying smartly designed color filters to pictures was basically Instagram's entire value proposition. In particular, I would have thought that designing filters to emulate the look and feel of various old films would have taken some fairly involved imaging knowledge. How did Instagram get so far without color experts?

argonaut• 10 years ago

I was never under this impression. I was always under the impression the filters are just created by a designer playing around with various effects until it looked nice.

elastic_church2• 10 years ago

Because it isn't complicated or novel to make compressed 8-bit jpegs have color filters. There are tools for the job and they've been around for a long time.

Working in a different color space than standard requires a little bit of familiarity and finesse that modifying 8-bit jpegs for consumption on the internet did not require.

Many photographers and printers are familiar with this dilemma in a variety of circumstances, where the cameras create images in a different color space and higher bit depth that can't be perceived at all with any technology or the human eye.

rwc• 10 years ago

In the "How I Built This" podcast for Instagram, Kevin Systrom specifically says the filters were created to take the relatively low-quality photos early smartphone cameras were capable of and making them look like photographer-quality photos. Filters were about taking average photos and making them "pop" but not necessarily by virtue of having deep domain knowledge of color.

trjordan• 10 years ago

Instagram's value prop was, and is, mobile.

ape4• 10 years ago

Instagram value proposition is that other mostly young people use it.

vladdanilov• 10 years ago

It's great you listen. So I'll try.

1. Scaling down in linear colorspace is essential. One example is [1], where [2] is sRGB and [3] is linear. There are some canary images too [4].

2. Plain bicubic filtering is not good anymore. EWA (Elliptical Weighted Averaging) filtering by Nicolas Robidoux produces much better results [5].

3. Using default JPEG quantization tables at quality 75 is not good anymore. That's what people referring as horrible compression. MozJPEG [6] is a much better alternative. With edge detection and quality assessment, it's even better.

4. You have to realize that 8-bit wide-gamut photographs will show noticeable banding on sRGB devices. Here's my attempt [7] to reveal the issue using sRGB as a wider gamut colorspace.

[1] https://unsplash.com/photos/UyUvM0xcqMA

[2] https://cloud.githubusercontent.com/assets/107935/13997633/a...

[3] https://cloud.githubusercontent.com/assets/107935/13997660/b...

[4] https://cloud.githubusercontent.com/assets/72159/11488537/3d...

[5] http://www.imagemagick.org/Usage/filter/nicolas/

[6] https://github.com/mozilla/mozjpeg

[7] https://twitter.com/vmdanilov/status/745321798309412865

unhammer• 10 years ago

I thought there was something familiar about that name Robidoux … I see he's the same one that got some crowdfounded work on GIMP's new scaling methods: http://libregraphicsworld.org/blog/entry/advanced-samplers-f...

bronxbomber92• 10 years ago

What is the pixel format of your OpenGL surfaces?

therealmarv6• 10 years ago

Maybe a stupid question. But how to save an image as Display P3 JPEG from Photoshop? I want to play with this color standard.

buss_jan• 10 years ago

Our iOS dev hunkered down today and may have come up with a decent solution to your EAGLView issue. You can find the write-up on medium: https://medium.com/imgly/bringing-wide-color-to-photoeditor-....

rubberbandage• 10 years ago

Your link somehow got truncated. Here’s the full thing: https://medium.com/imgly/bringing-wide-color-to-photoeditor-...

grenoire• 10 years ago

I did not know that Instagram was using OpenGL for processing. That's pretty neat, given the capabilities of OpenGL. I'm looking forward to seeing more filters with more lifelike Polaroid effects.

But before that, could they not convert images to horribly encoded JPEGs? I get it, bandwidth and costs, but it's an image service that still drowns in its own... when it gets an image with strong reds and blues.

mikeyk10• 10 years ago

We started using OpenGL in 2011. Our CPU-based image filters used to take 4+ seconds with pixel-by-pixel manipulation, and now can render 30+ times per second.

If you have some sample images where the current image pipeline is going wrong let me know and we can look into improving.

osi• 10 years ago

I think that's back when Gotham died :/

http://randsinrepose.com/archives/rip-gotham/

nrjdhsbsid• 10 years ago

Instead of fixing the horrible JPEG encoding can you please add support for webp? It's quite a bit smaller and well supported with polyfills since it's just a single vp8 frame

DinosaurArms• 10 years ago

I also went through that process with my app Picfx. Using OpenGL for filters is much quicker, the only downside I've found is being limited by the texture size, I did set up a way to process images in tiles but ultimately decided to just limit images to the texture size. Great info on the colour space, I'm sure it will be useful.

andresgottlieb• 10 years ago

Note that the red test image is only valid for iOS devices, since in Android you can see the Instagram logo even though it doesn't support wide color

PeanutNore• 10 years ago

I can't see the Instagram logo in the red test image on Windows 7

dgant• 10 years ago

On Windows, I can see the logo on one of my screens, but not the other.

kibwen• 10 years ago

Yes, was a bit confused when I opened the image on my ancient work computer (Windows, old Dell monitor) and could see the logo clearly. Interestingly, Chrome refused to display the image despite it nominally being a PNG, and I had to open it via an external program. Dragging the window to my second monitor (different model) causes the logo to vanish (though I do feel like I see it for a split second on the second monitor... optical illusion?).

bentpins• 10 years ago

Chrome should be able to display it fine. The dl=1 param on the Dropbox URL means the request is served with a response header indicating it's a download. Change the link to dl=0 and it won't force the download.

web007• 10 years ago

You are probably seeing the color management of monitor 1 in effect on monitor 2 while the window is less than halfway. Once it goes above half then monitor 2's color management takes over fully and it disappears.

jessriedel• 10 years ago

Can't see the logo on OSX Sierra on my 2014 MacBook Pro Retina 13in.

tibbon• 10 years ago

Can see on my 2016 MBP Touchbar 15" w/Sierra

jeffamcgee• 10 years ago

I have a 2016 MBP w/ touchbar, and depending on which color display profile I select in System Preferences, sometimes I can see it, and other times I can't.

tunaoftheland• 10 years ago

I couldn't either on my MBP late 2013 until I changed the Display Profile to "Display P3". There are other profiles that will show the logo, too, like "Adobe RGB (1998)"

eridius2• 10 years ago

I don't think that computer has a wide-color display. The 2016 MBP should (at least, the one with the touchbar, not sure about the one without), and the retina iMac supports it as well.

yefim• 10 years ago

Do you know why that is the case? Do Android devices project the Display P3 color space into the sRGB color space?

yincrash2• 10 years ago

https://www.reddit.com/r/Android/comments/52ultq/does_you_an...

This comment explains it (at least for Webkit's test image, which is probably similar to Instagram's).

No, it is a bad test. It is an image with an ICC tag that indicates it uses a color space larger than sRGB. The image data has the logo using color that should be outside the sRGB color space, but it still uses 8 or 16 bits to store that data. Android doesn't have color management. Android basically assumes all images are sRGB, so you see the logo. iOS does have color management. iOS sees the ICC profile and interprets the image data so that if you do not have a display that could show you the different reds in the image, it doesn't display them. So we have everyone in this thread on Android thinking they have a wide color display. Most of their displays aren't even 100% sRGB. My Nexus 4 shows the logo. It is very much not a wide-color display.

therealmarv6• 10 years ago

No. There is absolutely no color management in Android. https://code.google.com/p/android/issues/detail?id=225281

acegopher• 10 years ago

Why can you see the Instagram logo on Android if it doesn't support wide color?

mikeyk10• 10 years ago

Good to know--I didn't run it through my Pixel. Some devices will do a relative projection from Display P3 to sRGB, which means that it will look "relatively" like it would in Display P3 but projected onto the sRGB color space.

Edited to add: and some other ones are doing something even less fancy, which is just to ignore the color profile entirely and just assume sRGB and display it incorrectly, taking for example what would have been the maximum red point for Display-P3 and making it the maximum red point in sRGB.

bostand2• 10 years ago

Since you brought up your Pixel: what is the point of adding something 1% of your customers maybe can see instead fixing that horrible horrible compression that makes uploads from android (80% world wide market share) look like crap?

(Not an android user, I just want to figure out how a company of your size prioritises between bugs and features.)

ksherlock• 10 years ago

On OS X you can adjust your color display profile settings to make it magically appear and disappear.

ankka• 10 years ago

In wrong color though.

artursapek• 10 years ago

Ubuntu checking in - I too see a red square

mikeytown2• 10 years ago

Windows I can see it if using XnView but I can't see it if I use the built in image viewers. It's an older computer so it's XnView converting to sRGB most likely.

gojomo2• 10 years ago

Link for viewing 'canary image' in browser, without triggering download/dropbox:

https://imgur.com/a/v82GI

(Indeed, I can see a faint Instagram logo on my iPhone7, but not my older rMBP.)

phinnaeus• 10 years ago

Imgur seems to be changing the profile or somesuch because I can see the logo when I downloaded the image but not on the imgur image here.

NSAID• 10 years ago

Same.

Here's the link from the article with the download flag removed:

https://www.dropbox.com/s/tgarynpj65ouafd/insta-logo.png

SloopJon• 10 years ago

Last I checked, the browsers vary in their support for color management. Try a quick test at a page like this before drawing any conclusions from the canary image:

http://cameratico.com/tools/web-browser-color-management-tes...

crymer11• 10 years ago

If you have a Macbook and can't see the logo on the canary image, try changing the color profile for your display. On my mid-2014 rMBP, the default color profile was "Color LCD", but I could change it to "Display P3" and see the logo.

To make the change: System Preferences -> Displays -> Color

IshKebab• 10 years ago

But unless the display actually supports wide colour gamuts that will just make everything look worse.

mschuster91• 10 years ago

Thanks for the hint. On my 2015 Retina MBP, when opening the IG image in Preview, I see the IG logo. However, when dragging the image into Chrome, I do not see it.

How do I get Chrome to do proper rendering?

zer0t3ch3• 10 years ago

Have you tried restarted Chrome since you changed the display profile?

matt40772• 10 years ago

Chrome does something strange with colors on macOS. Colors always render visibly different than Safari.

That being said, the logo is visible in Chrome for (using a Dell display).

cletus• 10 years ago

Ok, so now that's done. How about the ability to zoom?

arnarbi• 10 years ago

Update the app? I can zoom in Instagram, both iOS and Android. Apparently since 18 weeks ago: https://www.instagram.com/p/BJxv5WthYas/?hl=en

yladiz2• 10 years ago

I'm not sure if this is different in the iPhone 7, but the 6S is pretty terrible at color representation in photos, specifically when dealing with neon lights. The iPhone tends to overexpose the neon light to make up for the surrounding being darker, so to get a 'decent' shot, I have to down the exposure by about 2/3 on the 'brightness' in the normal Camera app. But in general, it has a hard time showing as good of color as you would be able to capture in a better camera. For example, I've taken some photos of museum paintings during a visit, and the colors tends to be a little darker and not truly representative (yellows appear like mustard rather than a brighter yellow, for example). I'd love to be able to take more color accurate photos, and it would be more worth it to get the iPhone 7 if that's the case.

therealmarv6• 10 years ago

That's more a problem of the camera and ISP and not color management and display.

therealmarv6• 10 years ago

meaning DSP (easy confused).

yladiz2• 10 years ago

That's what I figured. I hope that the sensor becomes better and better, so that one day I can take great neon light photos (as well as see the correct color of red).

ramzyo2• 10 years ago

Awesome, would be great to see in-app support for 2x optical zoom on the iPhone 7+ next

mikeyk10• 10 years ago

We built this in already! We don't have a "1x" or "2x" indicator, but the dual lens camera is fully used in Instagram now and will do the smart transition between 1x>2x optical and 2x+ digital zoom.

ramzyo2• 10 years ago

Oh no way! That's awesome, apologies for not knowing this and thanks!

eslaught• 10 years ago

Are browsers generally color-correct? I was going through the Webkit wide color examples and was getting different visual results on different browsers[1]:

https://webkit.org/blog-files/color-gamut/

[1]: Compared Safari, Firefox and Chrome on a 2016 MacBook Pro w/ Touchbar running macOS 10.12.

mikeyk10• 10 years ago

Browsers are all over the place, unfortunately. It's part of why sRGB because the only reasonable color profile for Web use. I think we'll see wide color become common in apps before the Web.

gsnedders• 10 years ago

All over the place in what way? Support for different color profiles? Actually handling color spaces at all? The fact that there's no consistency when it comes to untagged images? The mess which is plugins? The ability to specific CSS colors in a specific color space?

TheRealPomax2• 10 years ago

So... what is "wide color"? Are we talking https://en.wikipedia.org/wiki/Wide-gamut_RGB_color_space or something else that has no documentation anywhere on the internet?

Not trolling, I care an incredible amount about color spaces, and had expected support for something like the UHDTV Rec.2020 color space with D65 (cool/blue white), not an obscure Adobe standard with D50 (warm/yellow white). It's wider than Rec.2020 though, so if this is what you're talking about: awesome, please update your article so people can find out more about this color model and the rest of the world can catch up!

cmelbye• 10 years ago

It was in the fourth paragraph:

The color space that Apple chose for its devices going forward is Display P3.

https://en.wikipedia.org/wiki/DCI-P3

TheRealPomax2• 10 years ago

How extremely disappointing.

planteen• 10 years ago

Cool article.

Another things worth mentioning is that lots of professional photo & graphics people have been using the Adobe RGB color space for almost 20 years which is "wider" than sRGB.

therealmarv6• 10 years ago

When on Android? I also opened a bug on Android Bugtracker: https://code.google.com/p/android/issues/detail?id=225281 and it is marked as a small issue. On Android we are lacking color management and are only restricted to sRGB. Original Wide Gamut and P3 images look dull on Android devices. I also want good wide P3 colors and color management without buying an iPhone.

simonjgreen• 10 years ago

The canary image in the article displays fine on my S7

petters• 10 years ago

"What every coder should know about gamma" may be of interest here. http://blog.johnnovak.net/2016/09/21/what-every-coder-should...

Bringing Wide Color to Instagram

Related Stories