An Adobe Flash developer on why the iPad can’t use Flash

http://www.roughlydrafted.com/2010/02/20/an-adobe-flash-developer-on-why-the-ipad-cant-use-flash/

by pauljonas • 16 years ago

76 87 16 years ago

roughlydrafted.com

slillibri3• 16 years ago

Please note this is an article by a "developer who uses Flash", not a "developer working on Flash".

It seems to me that most of the 'mouseover' problems described can be handled with click-and-hold.

Zev5• 16 years ago

It seems to me that most of the 'mouseover' problems described can be handled with click-and-hold.

Nope; the "Tap and hold" gesture is used by the Safari (and the iPhone OS, for that matter) for other tasks already, namely to bring up the cut/copy/paste element.

slillibri3• 16 years ago

There is no reason that I can think of that just because Mobile Safari uses tap and hold for some functions, that it couldn't pass it along to Flash when performed on a Flash element.

snprbob86• 16 years ago

I find this argument uncompelling. Nearly all actions are committed on mouse-up. This is how the default "click" events work in every environment. Click and hold becomes mouse down and hover, dragging remains hover, release can fire a mouse up, the framework can then inject an extra mouse down as part of the click if necessary. Apps that need more sophisticated behavior can ignore click/tap events and handle mouse up/down on their own.

tvon5• 16 years ago

The problem seems to be that the only event that can be passed to the page is a click, and it's sent on touch-up. Touch-down does nothing (since it could be a click or a drag), and a drag scrolls the page. The only thing they currently send through is a mouse-up, which seems to be sent through as a click.

In pseudocode:

  on touch:
    # do nothing yet

    if drag:
      scroll page
    else: # was just a tap
      send click to page

Granted, this is just how Mobile Safari is handling things, but how else could Mobile Safari handle things?

boucher3• 16 years ago

MobileSafari provides the entire touch sequence to javascript, which means that same sequence can easily be passed to flash.

It isn't that difficult to do a reasonable mapping from touch sequences to mouse sequences.

ugh5• 16 years ago

I really think it comes down to video. Video being the one thing that is easiest to replace. Many sites already do and it works like a charm. It’s a problem and it sure would be nice to watch The Daily Show but it’s down to inconvenience, not deal breaker.

If you want to make pretty much anything else work you will have to put serious work in. Just making an iPhone run Flash won’t it a pleasant experience. Some people might prefer that but there is probably nothing Apple hates more than those kinds of trade offs.

benologist3• 16 years ago

Almost nothing on the internet is "designed" for iPhones, yet nobody's complaining that Safari exists .... just like with Safari the onus is in part on the vendor to make their version suit the platform, and in part on developers and companies to fully tailor their offerings to that platform if they want.

ugh5• 16 years ago

But there is a difference between “not a nice experience but works” and “doesn’t work at all”. And it’s an important one.

tvon5• 16 years ago

On a related note, has anyone been able to get any interactive canvas demos to work on Mobile Safari? I've tried a few, and it seems like there are some related input issues there as well.

e.g. Sketchpad: http://mugtug.com/sketchpad/

mattmanser• 16 years ago

This is such a non-problem, a case of over thinking on author's part.

ugh5• 16 years ago

In what way? How would you solve the problems?

andrewbaron2• 16 years ago

You can tell from this article that the programmer thinks universally and in so doing, excludes a vast area of potential, especially in the future and more specifically in what he notes as "B) Gestures, finger gymnastics". You can stick your one finger up your nose but with more than one finger, you can do a lot more. I for one am ready for more complex interface functionality. I thought this was more obvious though perhaps it should be clearly stated: Get used to the growth in complex multi-touch gestures. More to come soon.

My guess (just a wild one for not being a thinker of touch interface design), would be to solve the solution by first detecting what kind of flash app it is and how much area it takes up, then anytime the finger scrolls over that particular area, there would be a cursor (the user could turn this off, but this is what would happen by default). As the cursor is led by your one finger which never lets up, meanwhile 1,2, 3 or 4 taps with another finger along with other more complex combinations eventually could perform various flash actions. Contrary to Dilger's primary thesis, I believe it's pretty straight forward that Apple is not as limited in thinking to make it happen if they want to, and that even the general audience is ready to take another small step in the gestures department.

freshfunk• 16 years ago

Don't have any special insight into flash but I do know that there certainly seem to be issues with playing flash on my mac (with any browser) vs PC. I'm guessing it has to do with GPU vs CPU issues but whenever I play a video (via flash) my laptop fans go on full blast (probably because the CPU is in overdrive). Quicktime video never seems to have this problem.

Flash also tends to have playback and interaction issues.

radley5• 16 years ago

I don't think the author is aware of Adobe's Slider framework which addresses the most basic needs (mobile device gestures). Beyond that, many of us are fully aware of the challenges of multitouch & gesture and are experimenting with variable-input interfaces (mouse, gesture, multitouch, and button/arrow based remotes).

radley5• 16 years ago

I will agree that the author has done a good job pointing out that many current Flash apps may have problems on the iPad. Many of those would be addressed if the blockade was lifted, but certainly not all.

mambodog4• 16 years ago

The impression I got from (briefly) reading the Adobe Labs page on slider is that it still would require all existing content to be re-written for it... which really negates the main reason for Flash on the Ipad: the dearth of existing Flash content on the web.

radley5• 16 years ago

Slider will require a different front end, but you don't need to toss the entire code base.

As for existing code, yes that will be a problem. But most Flash developers think of projects as evolving and/or disposable. Most of us aren't locked into stuff we did 3, 5, or even 10 years ago. We move forward and enjoy finding solutions to new challenges.

Que• 16 years ago

This seems ridiculous to me, it's almost as if the thought of using say... some sort of User-Agent to identify and differentiate the experience of the software is just well too simple to consider?

mambodog4• 16 years ago

uhh... actually that is covered:

"A) The best case: every Flash app on every site is re-thought by its designers and re-coded by its programmers (if they’re even still available), just for touchscreens. They wouldn’t use mouseovers any more—or else they’d have dual versions of all Flash content, so that mouse users could still benefit from the mouseovers they are used to. That’s a ton of work across the Web, for thousands of parties, and just isn’t going to happen. Plus, with many sites, mouseovers are so fundamental that the very concept of the site would be altered, creating a whole different experience that would annoy and confuse the site’s existing users. (And would this be any easier than simply re-designing without Flash at all? Not always.)"

m_eiman• 16 years ago

You'd still need to re-implement a significant part of each and every Flash app/game. Many of them would require a totally new UI and interaction model.

emehrkay• 16 years ago

Forget about performance, but look at how this guy struggles to pop balloons in this vid

http://gizmodo.com/5476500/htc-desire-rom-puts-flash-on-your...

glhaynes8• 16 years ago

Yep. And that's a really simple game. Good luck to folks trying to play more complicated stuff.

The model just doesn't make sense unless Flash developers program most games' input in two completely different ways. I can't imagine that'll ever happen, but even if it did, it'd be years until then. In the meantime, every alternate-universe, Flash-having iPad user who could see an app but inexplicably could only partially operate it would blame Apple.

mbreese3• 16 years ago

This is what I always wondered about. I mean, flash is designed to operate in a keyboard/mouse environment. How would it's event model map to a multitouch model? I suspect that it could be done, but it would be awkward at best. Also, the biggest argument for flash on the iPhone/ipad has been the large amount of games already available. Well, most (if not all) expect a keyboard and mouse. There is no way they would be able to work unaltered.

ThomPete6• 16 years ago

Isn't most frameworks that deals with GUI designed for keyboard/mouse?

Just because a platform allows for multi touch does not mean that the underlying functionality changes.

Multi Touch primarily gives you an ability to interact with the platform differently but the functionality it gives you access to is by and large the same.

Zoom, rotate etc.

Or what am I missing?

bmelton• 16 years ago

Predominately, you're missing the same thing I missed at first, which is that SO MUCH of Flash works on mouseover -- but how do you have a mouseover event on a touch-only interface?

pietro• 16 years ago

You're missing that in order to use touch, just about every Flash game out there would have to be rewritten.

Flash and its event model are quite flexible, and there's no reason why you shouldn't be able write games for a touch based system in Flash. But you'd have to write them first, and ideally there'd be an event interface specifically for touch, or it would be overly complex.

adolph• 16 years ago

There is a lot to be said for skating to where the puck will be, rather than where it is right now. For example, at one point in time most games were published on 3.5 inch magnetic floppy disks. The publishing media changed over time and that is no longer the case.

ryoshu• 16 years ago

You set up new events that handle low level stroke information or higher level strokes and chords. Multi-touch in Flash is possible; I was doing it three years ago on custom hardware. In some cases games would certainly need to be altered to work with the new events, but the AS3 event system is robust enough that you can write apps that work on multi-touch and non-multi-touch hardware.

pedalpete• 16 years ago

If this is true of flash, why is it not the same for Javascript. If suckerfish style menus can work on iphone, why would hover and clicks be handled differently?

karipatila• 16 years ago

Is that JavaScript or is it just CSS? If a HTML element has a CSS hover property, the first tap will activate that instead of the anchor. But, in any case, if it works for CSS it should play well with others, too.

alanthonyc• 16 years ago

"on a touchscreen, pointing at something without clicking isn’t a mouseover: it’s just holding your finger vaguely in the air"

Good point. Funny too.

mattmaroon• 16 years ago

"Current Flash sites could never be made work well on any touchscreen device, and this cannot be solved by Apple, Adobe, or magical new hardware."

I can think of a few easy fixes. A toggle button that makes touches count as hovers rather than clicks (easy). Hardware that detects a finger hovered near the screen (probably hard). Nobody caring because most things that occur on hover you can do without, and websurfing on even the best phone is so fraught with compromises and frustrating relative to a desktop that you really don't care (free).

glhaynes8• 16 years ago

"Nobody caring because most things that occur on hover you can do without,"

I don't think that's at all true. Yes, the concept of mouseovers could be done without (if it had been done without from the very beginning), but since so many Flash apps rely on it now, you can't have a decent Flash experience on a device that doesn't have it. Try using many Flash video sites without an ability to mouseover: the controls hide until their moused-over. A click on anything but the controls means "pause".

"and websurfing on even the best phone is so fraught with compromises and frustrating relative to a desktop that you really don't care (free)."

I strongly disagree. The web on phones before iPhone was indeed compromised and frustrating. But iPhone and some other recent phones have such good browsers that one often hears of people saying they reach for their phone more often than their laptop.

yason2• 16 years ago

You can simulate a mouse with existing hardware by decoupling the moving of the pointer and tapping (=clicking).

Therefore, moving your finger around the screen or tapping in a new position would only move the "mouse" pointer and not generate mouse clicks.

The mouse buttons could be mapped to some physical buttons on the device or to virtual buttons on the bottom or corners of the screen (think of your typical laptop touchpad layout).

Even better, the software could support a touchpad mode where the finger coordinates on the screen aren't mapped 1:1 to where the "mouse" pointer moves but as deltas similarly to how a touchpad works. This would allow for a fine-grained mouse movement even with the touchscreen and this would naturally limit the mouse clicking to specific locations / buttons only.

glhaynes8• 16 years ago

This could be done and applications wouldn't have to be rewritten. Likewise, you can just run a desktop OS and desktop apps on a tablet and nothing would have to be rewritten. Everybody thinks that sounds great; once they use it, nobody thinks so anymore. Because it's awful. That's the whole point of iPad using an OS and a collection of apps that from day 1 has been intended for touch.

yason2• 16 years ago

Since you can do that selectively in software there's no reason Flash shouldn't do it while the tablet OS and other applications should.

Touchscreen UI is all good but there will always be a class of applications that want both more precision and mousemove events.

As a personal note, touchscreens are painful to use unless you only have functionality behind big baby-size buttons. I wish I could always switch to such a touchpad mode if I wanted to.

asnyder3• 16 years ago

Maybe I'm not reading the posting right, but I really don't see a problem. Why can't the screen detect a slide over as a mouseover, but consider a tap as a click. Current trackpads already do this on laptops. I can drag my finger around to move the cursor and trigger a click by tapping on the pad.

If my laptop's trackpad can do it, I see no reason why a touchscreen can't do it.

mbreese3• 16 years ago

Because a phone has no concept of a cursor. When you lift your finger off of the phone, it is gone. Motion on a trackpad is saved by the computer as a change to the cursor position. This can be simulated by saving state in the touchDown/touchUp events, but it isn't like polling for the current mouse position.

The other thing to think about is that if you do have a mouseover event, you are likely going to try and display something (like a tooltip). On a phone, this would be covered by your finger.

asnyder3• 16 years ago

I don't understand why it doesn't have a concept of a cursor? Windows Mobile phones have the concept of cursor and I can have a mouseover by holding the cursor over something whether it's with a finger or stylus, and then click by tapping either with my finger or stylus.

What's the disadvantage to having an invisible cursor so that mouseovers work? Couldn't the cursor "appear" to the system when your finger is on the screen, similar to how the cursor shows for restaurant POS terminals when your finger is on the screen and then disappears when your finger is off it. Clearly I'm missing something.

mambodog4• 16 years ago

A touchscreen is not a trackpad. Their interaction models are markedly different, though they might (superficially) appear similar.

Slashed• 16 years ago

Exactly. When I was reading this, the first thing that came to my mind was that my MacBook has a multi-touch trackpad. Probably the solution would be to enable a mouse pointer over flash content.

lancer383• 16 years ago

But gaming is a much lesser experience with a trackpad then it is with mouse and keyboard; my Bloonz scores with and without a mouse prove it. Also, this doesn't get around the many games that require a keyboard.

asnyder3• 16 years ago

What's the reason for the downvoting of this and my follow-up post? This is very disconcerting, it discourages me from participating or contributing to an active conversation. HN etiquette requires a reason for downvoting.

As a long time member this is very frustrating. I'm sure I'll get even more downvotes for pointing this out.

darshan• 16 years ago

There aren't a lot of details, but Adobe sounds confident that they've made Flash pretty usable for touchscreen Android devices.

From the second video here: http://androidandme.com/2010/02/news/adobe-teases-us-with-fl...

"From a developer perspective, there's nothing that I needed to do to ensure that my game is functioning on this [touchscreen] tablet device. It operates the same as it does on the desktop or on this tablet device. Obviously the inputs are mapped, and that's taken care of automatically by Flash player. So Flash Player is able to bring hundreds and thousands of casual games that [are] online from day one." (Emphasis mine.)

DougBTX2• 16 years ago

That's just marketing speak, "functioning" is a weasel word trying to suggest, "fun to play," without actually making that claim.

He can get away with saying "inputs are mapped" because the only inputs that can be made - touch ones - are mapped, x,y co-ordinates from screen to player. Inputs which don't exist at all - hover, keyboard - don't exist, so don't need mapped.

cwilson2• 16 years ago

Are people here really suggesting we replace a mouse driven hover state on traditional computers with a touch driven tap and hold on touch screen devices? How does that make one lick of sense? What do you do while you're tapping and holding? Use another finger to navigate around your hover based controls? This is so counter intuitive I'm a bit baffled people are even suggesting it.

Half of the brilliance of computing moving in the way of touch is that you start to interact with digital things the way you would handle actual physical objects. People have been training themselves to use a mouse to move a cursor around a screen for the past 25 years, which is really awkward when you think about it.

With touch things work exactly how you expect them to. There have been dozens of studies with very young children who have had 0 experience with traditional computers who simply "get" touch devices, because its human nature.

I believe Apple understands this and they are not going to spend significant time solving problems that do not need to be solved, especially when said problems really just need to be unsolved and forgotten.

mbreese3• 16 years ago

I agree, there is a huge difference in a touch interface and a mouse/keyboard interface. I don't think that the answer is simply to simulate one with the other.

There have been dozens of studies with very young children who have had 0 experience with traditional computers who simply "get" touch devices, because its human nature.

Case in point: my 2 year old. He can use my iPhone with no problems. He can swipe the main screen left and right to find the icon he's looking for. Start the program, and be on his way. He can also start up iTunes on it, and play his music videos that I've loaded onto it. He also completely "gets" how to use the picture viewer. Now, if iTunes is playing my music and doesn't come up with the video screen, he's lost, but for the most part, it works well. The touch interface makes so much sense that he figured out how to get around in 2 minutes.

buster2• 16 years ago

And how is a mouse pointer different to your finger pointing at something? You know, there are even Operating Systems where the mouse pointer is a hand which you use to point the virtual index finger at things. Now, i can use my real finger to point at things, it is indeed the same f*ing thing.

ThomPete6• 16 years ago

"That’s not because of slow mobile performance, battery drain or crashes. It’s because of the hover or mouseover problem."

That is a problem with mouseover NOT with Flash.

The same thing can then be said about any website out there running on jQuery or Ruby that uses the mouse over for tooltip or menu pulldowns.

Just because the input is different does by no means suggest that flash couldn't be used.

The argument of performance is the only one I can see right now that have any merit.

buster2• 16 years ago

I strongly disagree with the developers article. He fails to explain the simplest solution (holding the finger). This is indeed the same as moving the mouse cursor on a touchpad, so wtf..

But what i really dislike is the notion of "it is not possible, never ever, it just won't work" without objective proof why something wouldn't work.. just stating that a problem is there and based on that "fact" he goes on and on.. How many websites would be unusable with a touchpad like interace? None.

It makes me sad that this guy obviously tries to make sense of steve jobs most stupid decission of the last years, because it is clear that Apple doesn't want flash/air on the iphone not because of the interface (clearly apple would be probably the best company to resolve this issue!) but because of the threat of mobile apps and games to the appstore. First it will be games, but if i can develop my apps for adobe air for all major operating systems and smartphones.. this is bigger than most people think.

I still hope that the apple community will eventually start to revolt when flash and air applications are available for all other platforms instead of just following apples stupid business decisions with such excuses.

glhaynes8• 16 years ago

"First it will be games, but if i can develop my apps for adobe air for all major operating systems and smartphones.. this is bigger than most people think."

Right, that worked great for Java.

Terretta• 16 years ago

I hate the idea of Flash on an HTML5 appliance, but the article and (as of 10 AM ET) the other posts in this thread miss one obvious way of supporting this with no extra buttons/toggles and no double clicks, click-hold-clicks, or other unusual behaviors.

If a user touches a Flash zone, show a cursor, and have the cursor moved by the iPhone's accelerometer. Tap to click.

For elements such as the video player example, the cursor (just as on PC) can fade after time, and a tap or shake would re-show it.

For most of the scenarios he mentions, using accelerometer to drive a hovered cursor would be a particularly intuitive solution.

wanderr• 16 years ago

I work for a company whose main site is a flash RIA. It's true that our site doesn't work out of the box on mobile flash enabled phones, although in our case it's mostly due to memory constraints. However we've created a new interface reusing most of the code we had already written for the main site, but optimized for mobile devices and without all the embedded images and memory intensive features included on the main site.

Doing that took about 2 weeks of serious dev time. Meanwhile, the native apps we're writing for each platform have taken 6 months each and still need a fair amount of work. HTML 5 is out of the question because it is missing some critical features (mainly content protection of any kind), so mobile flash is great for us.

Having had the chance to use flash enabled mobile devices on various websites, the complaints listed in this article are valid; I wouldn't use it to play flash games, but it is usable for media consumption, even YouTube's. Normal, non-mobile YouTube videos play just fine in mobile flash, for example.

eldenbishop• 16 years ago

I think a lot of these comments are missing the point. There is a huge difference between a mouse and a finger and you cannot just abstract this difference away by pretending that hover is only used for tooltips. Flash programmers from the very beginning embraced the platform because it allowed dynamic and flowing interaction with the movement of the mouse pointer. I can think of dozens of very common Flash techniques that will never work with a finger without being re-writen.

* Combo boxes that only expose the drop handle of hover. * Controls (video) that only appear when you move the pointer * Camera controls where the mouse motion is used to anchor view. * Animated buttons, flyouts etc.

As an example I did a google search for "flash examples" which led me to http://ziggystudio.com/v1/main.html

This app has an "AUDIO" link in the upper right that scrolls out on mouse over to expose the actual controls. This app simply will not work in a finger environment and would have to be re-written. I would argue virtually 100% of the flash applications written will have broken functionality in a click+drag only environment.

Most of these apps will "work" but in a massively degraded state. Many of them will simply be incapable of being used.

You cannot gracefully degrade to a click/drag system from a hover/click/drag system. They are too different. Flash was developed to maximize the mouse paradigm. It will never work well with a finger interface.

bumbledraven• 16 years ago

It would be cool if a touchscreen device could fire a mouseover event when a finger was NEAR the screen, and only fire a mousedown event when the finger TOUCHES the screen.

An Adobe Flash developer on why the iPad can’t use Flash

Related Stories