Understanding the iPhone 6 Plus Screen

http://oleb.net/blog/2014/11/iphone-6-plus-screen/

by elo • 12 years ago

170 54 12 years ago

0x04• 12 years ago

Would it take a jailbreak to trick uikit into rendering at native resolution?

What would be the downside? Larger UI elements and less available space on the screen? Could a @2x instead of @3x mode work or would that result in super tiny "bad hidpi" UI?

wingerlang• 12 years ago

There are at least two released jailbreak tweaks that does something related to this [0][1]. Although I don't know exactly what they actually do, technically, as I have not had the interest in these specific screen details.

[0] http://www.idownloadblog.com/2014/11/07/littlebrother-iphone... [1] http://www.idownloadblog.com/2014/11/06/upscale-change-resol...

0x04• 12 years ago

Hm, those two tweaks appear to be for iphone5/5s only. I'm more interested in the capabilities of the 3x screen on the 6+.

I wonder if this whole downsampling business is happening because the 6+ didn't get a "full resolution" screen in time and so they had to go with off-the-shelf 1080p screens? Related to the GT scandal maybe?

TillE• 12 years ago

The scaling makes sense for legacy apps, but I can't understand why they don't present a 1920x1080 screen for everything else.

Hasn't iOS had tools for building resolution-independent apps ever since the iPhone 5 was released?

0x04• 12 years ago

Probably because 1080p as a @3x screen gives a smaller effective viewport than the lesser models, which would make it look weaker?

zw• 12 years ago

No, iOS has had tools for building apps with flexible canvases, that has very little to do with the resolution of the screen these days.

userbinator2• 12 years ago

Does anyone wonder what's with the odd 414x736 resolution? I can't say I've ever heard of that resolution before; 360x640 at 3x would fit the 1080x1920 native resolution perfectly, or 2x the 540x960 of the iPhone 4. Apple's "Think Different" mentality at play here?

refulgentis• 12 years ago

Word on the Apple rumor street was that the display was supposed to be a full 3x 414x736 (1242x2208), but the display yield was too poor.

ChuckMcM• 12 years ago

That would be my guess as well, a compromise, no sapphire and a fudged screen. That said, my take away from this article was that once your screen resolutions gets above 300 ppi the resolution available is not nearly as important as it is otherwise.

twoodfin• 12 years ago

I didn't hear any rumors like that other than Gruber's very-late-in-the-game speculation (on the potential resolution, not the yield). If there were any, they were more than outpaced by rumors that the larger iPhone would feature a 1080p (or lower!) resolution.

Here's one from all the way back in October '13 that predicted the 1080p screen on the "5.7 inch" phone:

http://www.displaysearchblog.com/2013/10/in-2014-apple-will-...

I think they have made a conscious decision to bet on display scaling technology at high PPI's, so they have the flexibility to always buy the best displays they can get manufactured at the price point they need to hit.

quarterto• 12 years ago

360x640 at 3x would fit the 1080x1920 native resolution perfectly

But then you'd end up with a comically large UI at 6" with no space for any content because everything's so huge on the screen. The entire point of the larger phone is to have more content area.

Dylan16807• 12 years ago

A change of 15% or less each direction is hardly the difference between 'good' and 'comical'.

If the entire point of this phone is to fit more on the screen it's doing a pretty bad job of it.

jordanthoms5• 12 years ago

iOS still doesn't have proper resolution independence, hence hacks like this. There's no reason to be doing post scaling on a rendered image unless you have poor software which can't render at the proper resolution in the first place.

If anyone else did this they'd be pilloried for it being the total battery wasting font-destroying hack that it is, but Apple gets away with it.

This is true on the Macbook Retinas also - when I set mine to looks like 1680 or 1920, fonts get blurry due to this scaling, when if the software was better it'd be rendered at the native resolution and they would look as good as they do at the 2x resolution. (where the numbers work out that you can scale without a loss of fidelity)

TazeTSchnitzel• 12 years ago

Are you sure? I think iOS is resolution-independent, they just chose to do this since a 3x scale factor is more convenient.

EDIT: It also sounds like it was originally going to be the correct resolution, they just couldn't get the yields. If that's the case, perhaps they made it downscale at the last minute and used a 1080p panel?

leoc• 12 years ago

It also sounds like it was originally going to be the correct resolution, they just couldn't get the yields.

I strongly assume that the lower resolution was indeed Plan B, and that it's the result of another twist in the saga of Apple's relationships with the mobile component suppliers.

On a not-really-unrelated note, the Samsung Galaxy Note 4's screen is apparently pretty stellar in all respects, not just resolution: http://www.displaymate.com/Galaxy_Note4_ShootOut_1.htm .

jordanthoms5• 12 years ago

Why should 3x be any more convenient than 2.5x or 2.2x in a properly designed system? It's not any more difficult in Android (although images are prescaled in size buckets for efficiency and IQ reasons).

terhechte• 12 years ago

In Apple's defense, they've been working on full resolution independence since 2005 or so, with a setting for developers available from Leopard (2007) on, where one could set the resolution in between 1x to 3x. However, due to the complexity of the issue, Carbon apps, and probably a lot of other things, that never worked really well. I've been playing around with it with each new OSX release, until they removed it (I think Lion or Mountain Lion).

Basically, I think they never got it working good enough and just fixed it to 1x and 2x.

Here's a screenshot from the utility they had for this. http://i.stack.imgur.com/0NToe.png

ridiculous_fish5• 12 years ago

I worked on resolution independence on OS X over a period of years. I think the above comment is not made with full appreciation of the difficulty of the problem. Supporting arbitrary resolutions well is very difficult.

The underlying frameworks (e.g. CoreGraphics) are absolutely capable of rendering at arbitrary resolutions, and have been for a long time. It's not particularly hard: you interpose an affine transform which converts from local to device coordinates, apply it to all your geometry, and it falls out.

But "rendering at arbitrary resolutions" is not the same as "looks good at arbitrary resolutions." For example, a line 1 px wide at 1x will be 1.5 pixels wide at 1.5x. That means at least one edge must be aligned on a partial pixel, and there's the potential for weird antialiasing effects. There's also a behavior change: you cannot redraw half a pixel, so now "dirtying" a pixel requires redrawing more components at 1.5x than at 1x, which can cause performance problems and other bugs. And there's also a question of which pixels get the partial alignment: do you round in a direction? If so, which one? Or maybe you don't round and you have two pixels that are a quarter covered?

Which leads to the problem of centering. I wish to center a bitmap image within a button's border. The image is 101 logical points high, and the border is 200 logical points high. With a 2x scale factor, I can center absolutely, and still be aligned to device pixels. With a 1x, 1.25, 1.33, etc. scale factor, centering will align me on a partial pixel, which looks like crap. So I have to round. Which way? If the goal is "make it look good," then the answer is "whichever way looks good," which depends on the visual style of the bezel, i.e. whether the top or bottom has more visual weight. So now we need hinting.

And that's where things start to get really nasty. In order to make things look good at arbitrary resolutions, we want to round to device pixels. But the rounding direction is not a local question! Consider what happens if we have two visual elements abutting in logical coordinates, and they round in opposite directions: now there's a device pixel between them. That's very visible: you get a pixel crack! So you have to coordinate rounding.

WPF is a good example of a framework that attempted resolution independence and encountered this problem. Initially it has the "SnapsToDevicePixels" property, which triggers rounding behavior at draw time. But draw time is too late, because of the "abutting elements rounding in opposite directions" problem. So they introduced the "UseLayoutRounding" property, which does...something. And the guidance is basically "turn it on and see if it helps, if not, disable it." Great.

The web also has this problem in spades. Websites break in all sorts of fun ways when you zoom in or out. We tolerate this because, frankly, the bar is super-low for websites.

As I see it, the two options are:

1. Make everything vectors. You'll have to choose between weird antialiasing artifacts and potential pixel cracks; either way things will look bad. And you'll encounter bitmaps eventually, and have to deal with the necessities of resampling and pixel aligning at that point. 2. Scale only to integral sizes, and resample. You'll avoid antialiasing and pixel alignment issues, but pay a performance penalty, and things may look slightly blurry.

So which option is better? #1 has the potential for the highest-quality output, but at a significant price: developers must test their apps at more scale factors, and the failure mode is ugly drawing artifacts or outright bugs. #2 is more utilitarian: the output is not as nice, and you incur a perf penalty, but that's borne by the system instead of the apps, and the overall system is more consistent. #2 is also more forward looking: if you expect that pixel densities will continue to increase, then resampling artifacts will eventually be indiscernible, but a pixel crack will always be visible.

Apple took the practical and forward-looking approach to this problem. I can't fault them: to my knowledge, nobody has successfully implemented true resolution independence in a framework with wide adoption. If they have, I'd love to know how!

jordanthoms5• 12 years ago

What about Android? The layout managers there attempt to get good placing of items, dealing with these issues.

Agreed it's a hard problem and that comment was certainly reductionist, but at the same time it's depressing to have this be the issue that makes fonts blurry - the hardware is fine but the software side is letting it down.

0x04• 12 years ago

I think the OSX and iOS approach with @2x and @3x is the only sane way to go and is genius in its simplicity. It really works out well even for old apps that aren't updated (just double), while Windows' crazy DPI slider never worked properly. And you mention WPF, which for the longest had crazy blurring artifacts even in its text rendering.

People often say "just use SVG/PDF/vectors for all the things" but look online and you'll find many good arguments against it, not least because icons and text at lower pixel sizes really needs to be "pixeled" manually to look good.

acqq3• 12 years ago

Yes, I always believed Apple programmers were aware of all the potential problems.

There are also much older implementations: Postscript has "only" 595 x 842 "logical" pixels for a whole A4 page but it's just a first step in working with any resolution of the printer.

And NeXT started with http://en.wikipedia.org/wiki/Display_PostScript and Apple has http://en.wikipedia.org/wiki/Quartz_2D

To competently argue the topic it's good to understand more than "this number is different than that number."

mpweiher• 12 years ago

While I only looked at the performance implications back when, here are my 2¢:

As far as I can tell, all the problems you describe boil down to "In order to make things look good at arbitrary resolutions, we want to round to device pixels."

As TFA shows, with iPhone 6+, you can't. Round to device pixels, that is. I wrote about this when the 6+ came out[1], it's good to see it confirmed empirically (awesome job by Ole, incidentally!).

So there no longer is a choice between your options 1 and 2, because #2 is out the window, no integral scaling available.

Of course, to make #1 work properly, what they need to do is remove all the places where snapping to device pixels is done, because otherwise you get pixel cracks, and those are noticeable.

The anti-aliasing effects are apparently theoretical at this point: yes you can create test cases to show them, but in practice the resolution is high enough that it no longer matters. From TFA:

"I am actually surprised how little of an issue the automatic downsampling is in practice. As I mentioned, I simply don’t notice anything of the effects I have illustrated here in real life."

Or as Jon Gruber put it: "Its 401 PPI display is the first display I’ve ever used on which, no matter how close I hold it to my eyes, I can’t perceive the pixels. "[2]

I do think it makes more sense to do the scaling directly in CoreGraphics, disabling all the features that make snapping to the pixel grid possible and letting the pixels fall where they may. It would certainly be less work overall and use less memory. I can only speculate as to why it wasn't implemented this way, my guess would be that getting all the pixel-snapping out is not that easy, and possibly it was a rush job (as suggested elsewhere) due to last minute unavailability of the higher resolution display.

In the future, I would expect either (a) an actual 3x display or (b) the display stack adapted to your (1) choice.

[1] http://blog.metaobject.com/2014/09/iphone-6-plus-and-end-of-...

[2] http://daringfireball.net/2014/09/the_iphones_6

grey-area4• 12 years ago

Make everything vectors.

IMHO this is what Apple should have done a long time ago, certainly on ios. Yes it has issues, but those issues are mostly faced by the OS vendor once, and have been successfully dealt with in the case of font rendering for example with antialiasing, hinting etc. You don't hear anyone decrying fonts being vector formats and asking to go back to pixel fonts nowadays, and resolutions are only going to increase.

The OS should be handling rendering, hinting, caching, raster representations based on vector data. As you point out this is a hard problem and Apple are uniquely qualified to handle it. Instead Apple have punted on this and developers are actively encouraged to produce raster representations of each asset which break for every new device which comes out (as you point out this doesn't just apply to fonts).

Controls like buttons or custom ones should be drawn in code, not included as raster assets, and the iconography for things like buttons, launch screens, branding or other app-unique assets could be stored as the original vector, as it should be. If you want to put a raster image on a button, well, sometimes that's not going to look good.

On top of the problems caused by resamping, what Apple have done instead of choosing the vector route is create an ever-growing headache for developers and themselves by focussing on raster assets and requiring pre-rendered bitmap reprentations of assets like icons, which usually start as vector anyway. How many different pre-rendered sizes of ios icon are we up to now, 20? Where does it end? This is not practical or forward looking, it's a hack which imposes real costs on developers, customers and Apple - there is a huge amount of work in creating and maintaining all the different launch screen, icon and other assets now required to be rendered at 3 or more different sizes for Apple devices, and this is bloating every single ios app with multiple megabytes of assets which could be generated on the fly from their vector representation, over the entire app store that alone is a huge waste of bandwidth for Apple and customers.

So there are lots of reasons to aim for real resolution independence, and from the point of view of developers, Apple chose a quick hack (which has devolved into a hacky mess over time), over a more difficult, forward-looking solution which anticipates that screen sizes and formats are going to multiply until it is impossible to cater to all of them with raster designs.

ecloud• 12 years ago

I agree 100% with grey-area. But the trouble with vector artwork is the lack of an efficient widely-accepted storage format. SVG is full-featured but not particularly compact, and it's difficult to write a rendering engine which handles all its features correctly. There is such a big impedance mismatch between what SVG specifies and what GPUs can render without a lot of CPU-side preprocessing.

Since font technology is so old and so efficiently implemented, the obvious thing to do is to extend the same technology to rendering color fonts AND color vector artwork, such as icons. But Apple is going the wrong way on that too: they added emoji support to their fonts by embedding PNG images. Yuck. This article has links which explain more about the MS proposal, which is much better: https://en.wikipedia.org/wiki/OpenType#Color The glyphs can be built with layers, each of which has a color and a vector path. This is what we need. And then we need the Khoros Group, and then the GPU vendors, to focus on accelerating exactly the kinds of vector paths which are found in fonts. This will give us a better solution for rendering fonts and also a way of rendering resolution-independent icons and other types of application artwork at the same time. And the font format will constrain the icon designers to create icons that are efficient for the GPU to render: as few layers as possible, and without all the SVG features that make rendering so difficult.

In the meantime, it's a nice coincidence that the vogue of the moment is going towards monochrome icons. That means that the use of icon fonts (such as Font Awesome, and others) can go on for a few years even without adding color. But we can depend on there being a backlash eventually: we have color screens and are using them to show monochrome UIs? So the technology needs to be ready by the time the designers get back around to that way of thinking again.

kllrnohj4• 12 years ago

For example, a line 1 px wide at 1x will be 1.5 pixels wide at 1.5x. That means at least one edge must be aligned on a partial pixel, and there's the potential for weird antialiasing effects.

True, BUT you still have that problem on the 6+ with everything being scaled by 15%. In fact, the problem is vastly worse. Instead of the occasional line being 2px instead of 3px, you now have every line being blurry. And you can solve the 2px vs. 3px line thing quite easily by letting devs choose between specifying in pixels vs. points, which is what Android does. The problems you talk about are all problems that only exist if you exclusively work in points. They all go away when you can also use pixels.

And the density is not at all high enough to mask this. It's very visible as soon as you pick up the device, and the dividing lines in things like the calculator app are hilariously uneven and awful.

jdub• 12 years ago

Thanks for the great post. Despite your background and very complete explanation, you're going to get a lot of, "Couldn't you just...?" responses. :-)

joakleaf• 12 years ago

Regarding blurry fonts on you Macbook Retina: It only makes sense to set the resolution to match the number of hardware pixels. You cannot change the resolution of the hardware display, so any resolution other than the native resolution will look blurry as the system has to resample to match the hardware. This has nothing to do with OSX -- OSX can render fonts in any resolution.

... You can easily connect your Macbook Retina to an external monitor of any resolution. OSX is not limited to your Macbook Retina resolution.

On the iPhone6+ Apple could easily draw everything in 2.6x resolution to match the display, but I guess they feel using 3x+downsampling simplifies the UI-design process. From a technical point of view it is an unnecessary hack.

jordanthoms5• 12 years ago

The problem here is that you are conflating resolution with how big things are. They aren't the same.

I want the resolution on my Macbook to always be 2880x1800. But I want to be able to make things bigger or smaller, while still rendering at that resolution so things don't have to be resampled. (and in the case of the iPhone 6+, the default/only setting requires this resampling).

The problem is that when I set it to 'looks like 1680', rather than just rendering things natively at 2880x1800 but a little bit smaller, we have to get into all these scaling hacks (rendering at 3360 x 2100 and then scaling down to 2880x1800), which means you lose the accurate font rendering and clarity you got, everything has a slight blur to it because it's been scaled down.

saturdaysaint• 12 years ago

"If anyone else did this they'd be pilloried for it being the total battery wasting font-destroying hack that it is, but Apple gets away with it."

Except we're talking about a phone that sits at or near the top of every battery life shoot out chart. And the pickiest screen-specific shootouts fail to mention any difference in font detail between iPhone models. Some hacks work.

archagon• 12 years ago

It looks like the plan for resolution-independence on Apple platforms is "get the DPI so high that you can set the screen size to whatever you want". And I'm OK with that. Not everything can be represented with vector graphics.

Understanding the iPhone 6 Plus Screen

Related Stories