The Care and Feeding of the Android GPU

http://www.satine.org/archives/2011/01/01/the-care-and-feeding-of-the-android-gpu/

by buymorechuck • 16 years ago

57 55 16 years ago

wccrawford• 16 years ago

This isn't the first thing I've seen that Google devs really just didn't get. I think it's from the rapid expansion that Google has done. They get people working on things that aren't really their area of expertise because they wanted to try something in their 20% time.

In this case, it's obvious that we've got server people working on mobile code. Future hardware will resolve these inefficiencies? UGH. That's definitely someone from the server world speaking.

tgflynn2• 16 years ago

I'm talking out of my hat here a bit because I haven't actually worked with Android yet but it sounds to me that these problems might be related to the choice of Java as the core programming environment. I doubt that was a decision made by somebody working on a 20% project.

buymorechuck4• 16 years ago

Although Java has its problems, I think the core of Android's issue is a bad utilization of computing resources. One can only hope that Google will eventually fix this problem.

Palm's WebOS put much of its core logic in JavaScript, which had a slower implementation (at the time) than Dalvik VM does. However, Palm carefully made sure to take advantage of the GPU to achieve many of the effects in the UI.

jrockway3• 16 years ago

But the thing is, nobody actually cares about this. I have used an Android phone since the beginning. Touch responsiveness is not something I have ever noticed. Any delay in rendering web pages is due to the network.

I think people that write articles about touch responsiveness or animation latency are suffering from the "benchmarking hello world" problem. Yeah, you could speed that stuff up. But 99% of the time, the phone is waiting for data on a socket, which is not going to get fast any time soon.

buymorechuck4• 16 years ago

Perhaps, but for me, it's absolutely about usability. Whenever I see new Android phones or builds, the first thing I try out is how responsive the scrolling or zooming is. Often times, I'm waiting for Android to respond to my touch, other times my touches are missed completely. These issues cause unpredictability for me and make the phone harder to use.

As phones got faster, the situation improved, until a larger screen was added, which brought things back down again as the CPU burns to keep up. The milliseconds latency here and the stuttering FPS there decreases the usability significantly in my opinion.

I think that there is a group of users who don't notice these latency and animation issues. But I do, and I think many users still unconsciously attribute their dislike for Android to these issues. (example: my brother in law, used several Android Froyo phones extensively before settling on a BlackBerry because it felt "faster".)

pkulak• 16 years ago

People care, they just don't know what to call it. People who use iPhones try out an Android device and say things like "it's just not as polished", or just notice that it doesn't feel right. That "not feeling right" is because when you scroll on an iPhone it really feels like there's a piece of paper under the screen that your physically moving around with the tip of you finger. With Android it just feels like your finger is telling a computer what to do, which is in turn doing a reasonably good job of doing it.

rektide5• 16 years ago

Any delay in rendering web pages is due to the network.

many many webapps do not use the network, and any latency experienced comes from the page's own processing, the browser & the os.

rektide5• 16 years ago

Samsung cared, and they spend X million dollars building new devices that made use of the hardware they had onboard because they cared. As a result they can run a browser at 720p smoothly on their phone.

Innovation doesnt come from asking people what they want. It comes from finding new markets and making new means.

gubatron• 16 years ago

if you're dealing with lots of data from the Media Providers right on the phone, and you've worked for months on optimizing how to retrieve, cache, massage and display this data on lists, and you're still not getting a decent frame rate when scrolling through List Views you care about posts like these.

Never ocurred to me that GPU would help. I also come from a server programming background and hearing that GPU acceleration on the UI could help is like breathing fresh air.

There's several other more complex things you can deal with if you're writing apps that bring some actual value to the user, other than doing stupid web clients on android. It's not all waiting for data on a socket.

pohl• 16 years ago

DF regarding this thread: http://daringfireball.net/linked/2011/01/04/hacker-news-ying

nikdudnik• 16 years ago

The first thing I've noticed about Android was the overall slugishness of UI animation. And the guy who was showing it to me was amazed by the smoothness of my iPhone 3G UI. BUT YES OF COURSE NOBODY CARES.

rektide5• 16 years ago

i'd throw ANDROID-1147 and ANDROID-1376 in the Android technical management "absolutely really just doesn't get it" bucket. otoh, the issue Charles Ying of Satine.org rages aobut, ANDROID-6914, seems much more questionable to me because the issue itself is really complex.

the two tickets, "Support for external keyboard, mouse" and "Browser doesn't render SVG images" both strike me as absolutely essential to making smartphones competitive with PC's. people have been happily attaching fold up keyboards to Palm Pilots for a decade now, SVG is perhaps the only sane route forwards for resolution independent apps and animation, and Apple has found religion on both these topics-- yet Android is obstinately behind.

otoh composited gpu acceleration is not nearly the huge obvious win C. Ying boasts it as. the gpu is more power efficient than the cpu for intensive applications, but a smartphones natural state is not intensive use. mr guy of android points out a number of complexities-- opengl drivers only supporting a single context, opengl acceleration proving less speedy than software implementations. it places huge emphasis on driver support, which any embedded wonk will tell you is a scary prospect.

although gpu acceleration is becoming better supported, keep in mind that the smartphone market expands, both up and down, with broadcom & others putting out chips designed for sub $75 smartphones. android is definitely not winning the race for the top atm, but performance has always been a rather lonely market-- that said, unsmooth experiences dont fly. software graphics focus insures a base level of performance for everyone, and strikes me as the right place to start to insure a speedy experience not subject to external interrupts.

the current silkiness situation is a little sad. it'll get better. more relevant to me is Android not playing in the ecosystem-- external peripherials, UPnP/DLNA, SVG-- these are the real things that will make or break not Android, but the open digital web, and Android is failing, badly, even in compare with non-web-enthusiasts Apple.

tarandeep• 16 years ago

Have you guys tried MIUI on Froyo ? It gives me 60FPS for scrolling, panning and almost everything on a Nexus One: http://www.youtube.com/watch?v=Or-9T44Bt7M It has got NO GPU acceleration. That PWNs it

wzdd2• 16 years ago

Apparently touch responsiveness is improved in 2.3. However this article doesn't really explain what's wrong with it. I've been doing latency benchmarks for touch recently, and the highest latency I've seen for touch delivery is 10ms. Now 10ms is pretty terrible for what should be a very straightforward thing, but on the other hand it's only 2/3 of a frame, at 60 FPS.

The argument in favour of a GPU-accellerated GUI which is not mentioned here is battery life. I haven't been testing this side of things, but just because a CPU can do a 60 FPS GUI doesn't mean it should. Deferring it to the GPU would certainly reduce power consumption while navigating a UI. Whether this is non-negligible in terms of overall battery life, I'm not sure.

tgflynn2• 16 years ago

Are those 10ms with a realistic app/usage scenario ? I wouldn't think users would complain about 10ms, isn't that below the threshold of perception ?

wtallis• 16 years ago

A stutter 2/3 of a frame long means there's a 2/3 chance that you'll draw a frame that doesn't match the current finger position, followed by a frame that catches up more than it should have had to.

The ~40ms input lag shown by most desktop IPS LCD monitors is noticeable even though it falls below the reaction time. On a touch display, it should be even easier to notice input lag, because you can see your finger moving and the screen around it not keeping up. 10ms sensor lag may be below the threshold of perception for circumstances like this, but probably not by much. If you add even a few milliseconds of processing lag after your app receives the touch event, you'll be behind by a frame.

pvaibhav• 16 years ago

I work with digital audio/MIDI and a latency of 15-20 ms between pressing a key and hearing the sound is easily audible making it near impossible to play expressive music. 10ms can be felt and is 'annoying', under that is acceptable. I usually keep latency down to 5ms, at most 7ms, only letting it increase, e.g. during mixing when responsiveness is not utterly essential.

joezydeco• 16 years ago

You're seeing sub-10 mSec response times on projected capacitive screens? Care to share what you're analyzing? Cypress PSoC needs a good 20-30 to get a lock.

wzdd2• 16 years ago

That's interesting. I'm not measuring touches, I'm injecting events into the Linux kernel by writing to /dev/input/event? and measuring how long it takes for them to get to the application. That's because I am working on software optimisation, so I'm interested in the overhead introduced to events processing by the framework rather than the hardware.

buymorechuck4• 16 years ago

Actually, I wrote about battery life at the very end of the article. I already discussed battery life in a previous post. I certainly feel that battery life is one of the most important issues in mobile, hence, "efficiency"

As for touch responsiveness, I don't have enough current knowledge of Android's situation and touch hardware in general to write on that topic.

rektide5• 16 years ago

Both Charles Ying of Satine and wzdd seem sympathetic to this notion that the gpu is a power saving device. Surely it is when doing complex work. On the other hand, most smartphone usage consists of relatively simple screens and basic if any transitions. If it's only going to take 3ms of CPU work to run a 150ms animation, by all means, figure out how to keep the OS from interrupting the high priority graphics task and make the CPU do it. Particularly if the GPU would have to be on across the entire 150ms. If not, what is the cost of copying the graphical content between the GPU and the CPU? How much CPU does it take to initialize and set up the GPU for this extremely simple task you ask of it? Is the CPU going to be able to sleep while the GPU is running the animation, and if not what kind of context switching and control costs are going to be imposed on the CPU to manage the graphics state?

We've barely begun to see multi-core cell phone GPU's, but the point of the GPU is basically to run lots of work at a nice low clock rate where voltages can be dropped. If you dont have a lot of work to do, there's really no sense waking up the GPU and having it's extremely dumb poorly-branching cores chug away at figuring out, say, the SVG animations that a semi-competent ARM core could knock off right quick. You're just wasting power turning on the GPU.

Smoothness and polish, I suspect, are much more a questions of resource allocation than CPU capability, particularly when you've got a sub 640x480 surface and a 1GHz core.

jemmons2• 16 years ago

User: I think this feels laggy.

Developer: No you don't. I have the benchmarks to prove it.

DaveSF• 16 years ago

This may be related to overall linux interactivity scheduling. There has been a running debate on the Linux kernel mailing list about this for years. The Linux scheduler is more efficient for server-workloads and compiles, but it is not as good for interactivity.

Very recent kernels have a new scheduler which supports a new scheduling algorithm which people are anicdotally reporting better interactivity with. Perhaps this will help the situation.

yoden• 16 years ago

I think the android bug linked in the article is much more informative than the article itself.

Basically: 1) Android phones can already animate the screen @ 60 fps in software in most cases. 2) When they can't, it's not because the CPU is too slow, it's because there is other stuff happening (GC, etc.). A GPU can't fix that.

buymorechuck4• 16 years ago

The GPU can offload most of the graphics intensive tasks from the CPU, utilizing the GPU instead of letting it sit idle. freeing the CPU to focus on GC tasks. In addition, most of the animation work done with Dalvik code can be moved into a dedicated thread / task with highly optimized code backing it.

comex• 16 years ago

Anyone who thinks Android's animation is "fast enough" is either using a faster device than I have ever tried, or has never tried to compare, say, scrolling lists versus the iPhone.

I'm not saying the GPU is the answer, and maybe 2.3 is the answer, but ignoring the problem is not.

(Edit: So why does a GC pause stop animation, and not just by taking up CPU time, since moving to the GPU would help in that case? Is the animation code itself written in Java and waiting on the GC? If so, I bet rewriting it in C would be a massive help. Also, yuck.)

eru• 16 years ago

Or use a GC that doesn't need to stop the world. (Or at least only does so for very small intervals.)

ulrich• 16 years ago

Gingerbread has a concurrent GC, so animation should not stop completely. However, on a single core system I wouldn't expect this to have a huge impact. But this is just my personal guess.

dave1619• 16 years ago

I think the author nails it. Without GPU acceleration Android really is a notch below iOS in terms of UX. In my opinion, even WebOS and Windows 7 have much smoother scrolling and touch response than Android. But if Google wakes up and adds GPU acceleration and better eye-candy, watch out Apple.

asdasdrasdasdr• 16 years ago

What is the main performance difference between Vista SP1 and Windows 7? Hardware accelerated UI.

simonh• 16 years ago

Purely by chance, I was trying out a galaxy tab at a store today. I loaded up the Financial Times web site, which took an age because the store wifi was crap and tried scrolling the web page and it stuttered quite a bit at first then smoothed up, zooming as juddery as well. Did the same thing on an ipad - slow loading about the same, but the instant the page was rendered it scrolled and zoomed perfectly.

Also the widget for editing the web page URL on the Tab was a horrible usability car crash. What's up with that? Disclosure - iPhone user, but Android curious.

typo_pl• 16 years ago

Android's VM can improve perf-wise - see http://blogs.oracle.com/javaseembedded/2010/11/how_does_andr...

michaelward82• 16 years ago

It doesn't matter what the solution is, but Google need to prioritise UI responsiveness above all else. Until that happens, the UI on Android will suck.

The Care and Feeding of the Android GPU

Related Stories