Crashing competing media players on Android

http://felix.abecassis.me/2014/04/crashing-competing-media-players-on-android/

by etix • 12 years ago

49 26 12 years ago

felix.abecassis.me

jbk5• 12 years ago

Hardware-acceleration of video decoding on Android is really a mess...

In addition to the point of the article (asserting on a file that is not processed correctly, instead of slowly bailing out), every piece of CPU/GPU behave differently and the Android test suite is extremely limited. On top of that, there are many layers (OMX IL, stagefright, MediaCodec, OMX AL, Java Media) that interact together in a complex way and everyone messes in those layers in a different way...

For example, fora long time, even after mediacodec API was out (4.1), you could get the data back from the codec, but the frame format was not standardized and there were no tests in the Android validation! So every GPU was doing something differently.

On top of that, the hw decoders don't even give the profile and level they support, so you have to guess them.

And even those days, there is no way to get a YUV 420 surface that works on most devices, even when using KitKat APIs.

And so many joyful things :)

etix3• 12 years ago

The most questionable thing is why did they keep the assert in production code. Looks like unfinished job to me.

bennyg• 12 years ago

That's how I feel when I read through Android documentation with most things: this just seems unfinished.

jpk4• 12 years ago

When given the option to either abort the process or put hardware into an unrecoverable state and requiring a reboot, you leave the assert in for production.

jaegerpicker2• 12 years ago

The MediaCodec support is SO BAD in Android. I spent months and months fighting it every step of the way to try and get real time streaming video working in a app. YUV 420 vs rbg pixel formats still gives me nightmares. I remember testing 3 different phones and each one returning a different pixel format, two were yuv base (422 or 410 i think but it's been a year and I try to block that out) and one rbg. Bleh! that project made me hate android for about 6 months after I was done with it.

alttab• 12 years ago

Sounds like a solid opportunity for some middleware.

jpk4• 12 years ago

On top of that, the hw decoders don't even give the profile and level they support, so you have to guess them.

We actually put together a codec tester that runs right before the first time a user plays a video precisely because of this. It throws a few frames of every profile/level we plan to support at each decoder component on the device to figure out which profiles we can use and which component to use to decode them. :(

cpeterso3• 12 years ago

I worked on Adobe Flash's Android video support, so I feel your pain. :) Flash sits in the uncomfortable position between an internet's worth of poorly-encoded MP4 files and all the crappy Android devices' hardware decoders.

Google's Stagefright developers told me they did not want to provide an API to query the hardware decoder's profile and level support because some hardware supports H.264 capabilities beyond their state profile or level. Their recommendation was to "just try decoding the video" to see if it works (as if that is something that can be easily determined programmatically), ignoring the fact that playing an unsupported or corrupt file will trigger Stagefright's fatal CHECK() aborts. In Flash, we had to duplicate a lot of Stagefright's MP4 parsing code so Flash could fall back to its H.264 software decoder instead of hitting Stagefright's CHECK() "landmines".

We also had to write almost 10 different color conversion functions for all the random color formats used by long-tail Android devices.

jbk5• 12 years ago

The funny thing is that Android is parsing all your media files in your back, so even if you don't use their demuxer, the Stagefright's CHECK() can bite you...

We also had to write almost 10 different color conversion functions for all the random color formats used by long-tail Android devices. We have those too. Are some of yours shareable?

j_s2• 12 years ago

All video on Android seems to be a mess (except YouTube). For example, I can't figure out what streaming video formats are supported, and/or which apps work best. Does anyone have enough experience to say whether HLS or RTSP is better supported on Android?

etix3• 12 years ago

For Android > 4.0 go for HLS but you'll need a proper encoder.

taiki• 12 years ago

Same's kind of true on iOS. Ultimately, the best solution seems to be on both platforms, ... VLC. Everything else seems to be based on ffmpeg anyway.

lnanek2• 12 years ago

Honestly, media on Android is so bad it doesn't need any help crashing. I worked on a large production app that played sound files in the background when running and about once in a thousand the Android media calls to play the file would just permanently hang. The only way to reliably handle media on Android is to have a watchdog process that checks when your own is hung in the Android code and kills and restarts it from saved state files. This was a call to play just never returning, so there was no error to catch or anything that could be done.

jpk4• 12 years ago

"Since your average Android application does not have sufficient permission to access /dev/* entries on your system, you must use a proxy called the mediaserver in order to communicate with HW codecs."

This is incorrect. Application access to hardware de/encoders isn't a matter of permissions. You can't get at them directly because the api isn't in the SDK/NDK. However, if you're willing to pull in headers from AOSP, and fool around with dlopen/dlsym you can develop and build against libstagefright.

flx42_• 12 years ago

It depends on your SoC, but most of the time your application won't be able to access HW codecs and then you have no choice but using the mediaserver. I think that if you pull OMX functions from libstagefright you are actually using IOMX, it's the regular OMX IL API but using IPC with the mediaserver.

lcedp• 12 years ago

I think both author and you are correct. There is a misunderstanding in the 'permission' term. Author uses it generally and you specifically as in Android permission.

Crashing competing media players on Android

Related Stories