How 'Minority Report' Trapped Us in a World of Bad Interfaces

https://theawl.com/how-minority-report-trapped-us-in-a-world-of-bad-interfaces-d8d2d2af41da#.rwihipewn

by LeratoAustini • 10 years ago

138 102 10 years ago

maldusiecle2• 10 years ago

Crazy that this guy criticizes the Minority Report interfaces and then makes the terrible claim that interfaces ought to make use of humans' ability to grip. Not all humans have strong grips, particularly the elderly. There's a reason newer buildings use lever handles over doorknobs.

My hope would be that the interfaces of the future would be more accessible to the disabled, not less.

ksk2• 10 years ago

Wow, I guess the article really struck a nerve with you huh?

Taking advantage of major human ability != making it an exclusive requirement to use said interface.

Also, suggestion != claim.

chc• 10 years ago

You sound a lot more het up about this than the parent comment. The parent just suggested that this article is ignoring some things that should be considered if you want to talk about designing interfaces for real people. It's important to design things that don't fail when confronted with edge cases.

mistercow2• 10 years ago

It seems pretty obvious that accessibility is always going to require extra thought. Some people are paralyzed from the neck down, but we still have keyboards.

In any case, why do you think that using gripping as a UI action implies that you'd need a strong grip to use it?

maldusiecle2• 10 years ago

Sure, it's always going to require extra thought, but why create problems when you don't need to? There's this entire train of reasoning that seems to be that a rich interface is one that places as many demands as possible on the human body. That thinking will inevitably lead to problems of this type. If grip isn't necessary, why make it necessary? I could have made a number of objections of this type to the article, but this seems the most obvious and understandable one.

SiVal• 10 years ago

It seems pretty obvious that accessibility is always going to require extra thought

Actually, I think this extra thought could help everyone by treating disabilities as just special cases of the general idea that different people prefer different input/output devices and having apps (mostly) stop processing low-level input (keystrokes, pixel hit testing) and switch to handling more abstract "primitives".

The OS would have a set of more abstract primitives: enter text, zoom in, zoom out, rotate, 3d move, 3d rotate, select item, select 2d region, select 3d region, deselect, grab, cancel, undo, etc. You, the user, would need a set of I/O devices that provided one or more implementations of each primitive in the checklist.

One user would send an "undo" to an app with a click of the tongue, another with a hand gesture, another by pressing the ESC key, but the app just gets an "undo" event from the OS.

For 2d region select, for example, one would use a joystick, another uses voice, another uses eye movement. The app doesn't process pixel hit testing to do its own region select, it lets the OS tell it what region of the 2d surface was selected. The app makers don't make the app "accessible" themselves, they just make it respond to the checklist of higher-level primitives.

You could provide at least the core functionality of most apps by this means after which an app could possibly add some lower-level processing in the form of "shortcuts".

When I'm forced to use a watch interface away from my keyboard, I am in a sense "disabled". When driving in my car with eyes on the road, I am "disabled". I'm not a gamer, so relative to a gamer with mad joystick skillz, I am "disabled". Well, fine, just change the approach to software interaction to a more abstract type of interaction that treats us all as being variously abled.

crimsonalucard• 10 years ago

Well not all humans have fingers either. There's a lot of vets and amputees who have no arms or legs so all user interfaces should be eye tracking based.

Honestly, common sense says the guy is talking about interfaces for the general population, not the disabled. If most people can grip then make interfaces that grip. Don't force everyone to read braille just because a few people are blind.

falcolas• 10 years ago

To be fair, waving your arms around, or hitting targets 10x smaller than the finger you're using to hit them is no more accessible than twisting a knob or moving a slider.

No single interface will ever suit all potential users, so any accessible design has to keep this in mind.

glangdale• 10 years ago

Hmmn, I'm betting that a user interface that requires 'gripping' might possibly not require someone with the grip strength of a gorilla. And your self-serving invocation of the disabled fails to notice that most gestural or touchscreen interfaces are atrocious for the elderly or disabled, and have a tendency to misconstrue 'tremor' as intentional movement.

Generally if you want to design for the disabled, you probably are designing for the disabled, rather than using them as fodder for an argument.

nisse723• 10 years ago

There's a reason newer buildings use lever handles over doorknobs

For the cats.

TeMPOraL• 10 years ago

Yeah, as a teenager I had to disassemble my handle and reinstall it vertically in order to make the cat stop jumping on it in the middle of the night and waking me up in the process...

(I like cats. The problem then was that I had way too much stuff in various cabinets and the cat liked to make a grand tour through them, throwing a lot of stuff to the ground in the proces...)

coldcode• 10 years ago

It was a good interface for predicting future crimes of bad interface design.

dmtroyer• 10 years ago

T9 Texting RIP.

muad• 10 years ago

I am getting rid of my smartphone after it breaks and downgrading to a flip phone.

I am actually looking forward to being able to type with one hand again.

nisse723• 10 years ago

With the thumb of the same hand that's holding the phone? Welcome to RSI hell!

clock_tower3• 10 years ago

Cost is another driver here; touchscreens are significantly cheaper than buttons, and can be used to do more things -- at the cost of being awful for typing on.

samstave• 10 years ago

what happened to "Oblong" technologies (the guys who actually did the minority report UI/UX and turned it into a conference room app/system)

PT_2014• 10 years ago

They morphed their high end concept stuff into smart meeting room and team collaboration tool platform called Mezzanine (- see http://www.oblong.com/) which is sold into large enterprise class customers e.g IBM, SAP, Accenture, NASA, GE etc.

rbanffy3• 10 years ago

I kept imagining the kind of BS the actors are talking about the fake graphics.

quote• 10 years ago

They recently received 65M funding. You can supposedly buy that stuff.

kristopolous2• 10 years ago

if you thought the interface was bad, try programming with their software stack.

tomjakubowski• 10 years ago

Interesting. Would you care to elaborate?

sandover• 10 years ago

Oblong sells a conference room product and we also do custom work.

I work at Oblong and recently gave a short (unofficial) talk which covers some past and current activities.

https://creativemornings.com/talks/brandon-harvey/1

My favorite sentence from the piece is "there’s a huge gap between what looks good on film and what is natural to use". Yep!

realfinkployd• 10 years ago

Don't pretend you don't want a 3d printer with a Tony Stark design UI :)

atarian• 10 years ago

And despite his acute taste for good interfaces, he offers no solution to the "trap".

eridius4• 10 years ago

It's perfectly legitimate to criticize something without offering an alternative solution.

lj3• 10 years ago

Is it? What value does criticism bring if there's nothing actionable in the criticism?

hx873• 10 years ago

It isn't legitimate to do so without at least offering some idea of how the object of criticism could be improved though. If you criticize something, that means you know it could be better in some way, and you should at least point out those ways.

panic• 10 years ago

The linked article by Bret Victor on the future of computer interfaces is worth reading: http://worrydream.com/ABriefRantOnTheFutureOfInteractionDesi...

Of course, Pictures Under Glass interfaces have advantages too. Touching a screen has become a familiar way to interact with technology: a more tactile interface might be harder to learn. To make a screen-based interface, you only need to draw a 2D picture and respond to 2D touches, not design a 3D object which can be manipulated in space. You can buy touch panels off the shelf instead of having to manufacture expensive custom 3D parts.

But these are advantages of easiness and cheapness, not quality. If designers and engineers can put in the time to make a physical interface and users can put in the time to learn it, you can improve quite a bit on Pictures Under Glass. Think about how terrible it would be to replace a car's steering wheel with a touch screen, for example.

So that's really what the article is saying: we should think of touch screens as the cheap, easy option, not the exciting, futuristic one. Our futuristic starship control panel should be a custom-manufactured physical interface instead of a cheap glass panel.

makmanalp4• 10 years ago

Well, so the thing that I think people don't realize about gestural interfaces is that gestures are hard to do and tiring to do all day. They are also very low bandwidth. Typing is hard to beat in terms of bandwidth / strain. But there are things that gesturing is better at, and I like that that article focuses on that. I think the mistake is to try to get a catchall. That's why we have both a keyboard and a touchpad / mouse. Each does a different kind of task. Touchscreen typing, on the other hand, is a step back. Game console typing is like 20 steps back.

douche2• 10 years ago

I'm not sure mouse and keyboard will be unseated from the top spot until we get to the point of straight mind-machine interfaces.

nitrogen3• 10 years ago

What are some good examples of gestures that work better than alternative input devices? It seems to me like the obvious choices of zooming, scrolling, and rotating things can all be done better with buttons, a mouse, a wheel, or a knob. I can press the back button on my mouse a lot faster than I can two-finger-swipe to go back a page.

WorldMaker• 10 years ago

«I think people don't realize about gestural interfaces is that gestures are hard to do and tiring to do all day.»

To be critical, this is a biological feature much more than it is a bug. Our bodies don't have very good feedback mechanisms for the small accumulative damage from fine motor control, which leads to things like RSI and Carpal Tunnel with very few in-between indicators.

In a way, moving to a more gestural interface that feels physically tiring is a possible means to better overall ergonomics because people actually start to get real feedback (tiredness, soreness) telling them to take a break every now and then when they are physically straining themselves.

dgudkov4• 10 years ago

Typing is hard to beat in terms of bandwidth / strain

We type only because we can't talk to computers (although it starts changing). Voice control would easily beat typing.

toomanybeersies2• 10 years ago

Our futuristic starship control panel should be a custom-manufactured physical interface instead of a cheap glass panel

I disagree. Modern aviation has been moving away from physical controls and towards generic screens. The last gen of fighter aircraft (The F/A-18, later block F-16s, the F-22) have a collection of generic screens with context sensitive buttons on the side for control.

The F-35 takes this a step further and has a single large screen (which is also apparently a touch screen).

In something as complex as an aircraft (or a starship), there are hundreds (or thousands) of different controls. Look at the cockpit of an older aircraft (e.g. a Boeing 747) as an example. Most of the controls in a cockpit are used in certain situations, you don't need to control the landing gear or the brakes when you're at 30,000 feet.

By using generic displays, the relevant controls for the current situation can be shown. Yes, it is more difficult to operate the controls by feel, but this is less of a worry in aviation, where you're usually several thousand feet from anything else, than it is in a car, for example.

So of course, it's all about the expected situation, you wouldn't want a little touch screen that you swipe for the safety on an assault rifle, for instance, because you need to be able to do it by feel in various situations. But a swipe unlock is perfectly acceptable for a phone.

jccalhoun• 10 years ago

I don't know if Minority Report started the trend, but there is one thing that I have really grown tired of in movies with futuristic tech: transparent screens.

I realize that they let the director film the actor without something blocking the camera but they just seem like a lazy way to be "futuristic."

jasonm232• 10 years ago

As a side note what on earth are these?

https://s-media-cache-ak0.pinimg.com/736x/bd/17/68/bd1768e3a...

And why are they in so many sci-fi graphical UIs?

rbanffy3• 10 years ago

Until we are able to dynamically reconfigure our devices to offer context-aware controls that we can grab, touchscreens seem to be a preferable alternative to buttons. You don't want your desk to become the Apollo CM dashboard.

knolax• 10 years ago

context aware controls

you mean key-bindings?A full keyboard already has ~100 keys.

rbanffy3• 10 years ago

A keyboard cannot morph into a joystick.

legulere• 10 years ago

If Jeff Han had designed a keyless entry system for a car, it would’ve involved dragging a secret gesture on the car’s window instead of the car automatically unlocking when you open the handle if you have the key in your pocket.

So basically: "Good design is invisible"

sp332• 10 years ago

Both examples are invisible. But one works with the flow of getting in your car and the other requires arcane gestures.

jamesmcintyre• 10 years ago

One of my favorite design mantra's is "Design dissolving into behavior." - Naoto Fukasawa.

minitech• 10 years ago

That’s not what “invisible” means in this context. A better name might be ”unnoticed”. It’s interface sufficiently natural as not to add to the things you have to think about.

mnutt• 10 years ago

A few days ago I tried out an HTC Vive VR device with Google's Tilt Brush app. It's basically MS Paint in 3D. You hold both Vive controllers and have a brush in one hand and your palette in the other.

I was amazed how intuitive the controller UI was. Some of the controls are on the back of your hand, but it was incredible how natural it was to pick up and use comfortably. I've never been all that fond of the Minority Report floating screen UIs, but other 3D UIs can feel very natural in the right context.

adambmedia• 10 years ago

More productive is to try and understand why Minority Report has successfully captured the imagination of so many, persisted over a decade, spawned an entire genre of syfy interface trope that we see in virtually every film having anything to do with future, beyond film, well into games and now commercial interface development.

The assertion that the gesture interface is some kind of virulent ear-worm, dismisses what may be the most interesting thing, the fact that it has captured the imagination of so many. Why?

hacker_9• 10 years ago

Humans are visual creatures, so if something looks good we want it, regardless of whether it is practical.

peter3032• 10 years ago

There is an article somewhere about how Speilberg commissioned consultants to imagine the near future. What happens you get fresh, imaginative movie imagery loke 2001, Star Wars, Minority Report. Then fifty copycat movies.

jerf• 10 years ago

I don't think it's hard to imagine why the gestural interface survives in the movies. It's very visual. The main actor is moving and full of action and may even be literally jumping around. It's as cinematic as the Hollywood OS representation of hacking is.

Meanwhile, out here in the real world, I'm a real computer programmer and my real interface is even more boring looking than a standard Windows install. Nobody's going to be mining my UI for Hollywood ideas anytime soon, despite the fact that I, on occasion, actually hack.

It's the same reason cars explode when they go over cliffs, bullets spark when they hit things, and all those other tropes.

peter3032• 10 years ago

Some people blame Xerox Parc for locking the world into the office metaphor. Some widely icons like floppy disks and phonsets represent obselete objects.

The ancient Greeks used rooms of a house to organize information. Just think we could of ended up with that instead.

bparsons• 10 years ago

Microsoft tried that in 1995. It was an interface called Microsoft Bob.

https://en.wikipedia.org/wiki/Microsoft_Bob

DonHopkins• 10 years ago

I wrote an iPhone app called "iLoci" [1] that was based on the Method of Loci [2], which had a gestural interface that let you construct and navigate your own networks of locations by dragging rooms around and "kissing" them together to connect and disconnect them. Another way to think of it is as a pie menu editor.

Later I elaborated on the idea in a Unity prototype called MediaGraph [2], which lets you arrange your music in an editable gesture navigable map.

[1] iPhone app iLoci by Don Hopkins @ Mobile Dev Camp: https://www.youtube.com/watch?v=03ddG3jWF98

A talk about iLoci, an iPhone app and server based on the Method of Loci for constructing a Memory Palace, by Don Hopkins, presented at Mobile Dev Camp in Amsterdam.

[2] Method of Loci: https://en.wikipedia.org/wiki/Method_of_loci

[3] MediaGraph Music Navigation with Pie Menus Prototype developed for Will Wright's Stupid Fun Club: https://www.youtube.com/watch?v=2KfeHNIXYUc

This is a demo of a user interface research prototype that I developed for Will Wright at the Stupid Fun Club. It includes pie menus, an editable map of music interconnected with roads, and cellular automata.

It uses one kind of nested hierarchical pie menu to build and edit another kind of geographic networked pie menu.

nisse723• 10 years ago

I want a 2-knob "mouse" like the one I had on my etch-a-sketch 30 years ago! One for X, one for Y...

[ 3 minutes of googling later... ]

Of course it's been done: https://mitxela.com/projects/etch_mouse

will_pseudonym• 10 years ago

I'm curious; why does this appeal to you?

corysama• 10 years ago

Here's a nicely realistic breakdown of the physical affordances involved in useful interfaces in VR.

https://www.youtube.com/watch?v=id86HeV-Vb8&t=5m34s

kayoone• 10 years ago

This is also what comes to mind when comparing the giant touchscreen in a Tesla Model S to the haptic buttons and knobs in most other premium brands. Especially when driving, touchscreens just seem an inferior way to interact with the systems.

toomanybeersies2• 10 years ago

Driving, yes absolutely, because you're constantly about 6 feet away from hitting something.

For flying, touch screens are a great idea, since you have a lot of controls, and attention to where exactly you're going is less critical, since you're usually >1000 ft from the nearest solid object. Also, aircraft usually have autopilot (and often a copilot).

I guess with Tesla's autopilot, it might make the touch screen a bit more usable? (at least while driving on a highway I guess).

clock_tower3• 10 years ago

Now I want to be in position to order a GUI, preferably for some sort of notarization or document-drafting app.

I'll explain that I'm actually thinking of a movie for the UI, and I want it to be as exactly like that as possible; I'm sure they get this request really often, and it won't be anything new to them; and I'll explain that I even brought some scenes from the movie to show them exactly what I wanted. And then I'll be grudgingly persuaded to pop in the relevant DVD...

And the scenes will be from a pirate movie, with an old-timey European pirate/bandit/mercenary chief writing with a quill on parchment while wearing a coat with ruffles and lace.

Only quills will work as pointing devices. To save your work, you sprinkle some sand on the tablet and blow it off. The tablet comes with a breathalyzer attachment, which checks for enough rum in the air to prove that you're a legit pirate but not so much rum that you're signing contracts while drunk. Voice input must be in Ebonics, which is pretty much what the Cavaliers spoke in the English Civil War. Voice output is in a thick Devonshire burr (the classic pirate accent) that makes you think that the app's probably half drunk.

Now, if only someone's eccentric enough to do it...

jackfrodo• 10 years ago

Ha, if you're ever in the area, you seem like you'd appreciate 826 Valencia's pirate supply store in the Mission district of SF.

http://826valencia.org/store/

clock_tower3• 10 years ago

I'll keep it in mind; it looks well worth a visit!

I'm also toying with the idea of getting some reenactor-grade pirate clothing: long-wearing like nobody's business, and relatively easy to care for. I've even found places that sell those leather cavalry boots with turned-down tops! For an extravagant price, of course...

StillBored• 10 years ago

I haven't been a big fan of many of the UI changes in the past few years, but I realized the other day that these changes have actually resulted in _DEATHS_.

http://www.detroitnews.com/story/obituaries/2016/06/20/obit-...

Cause you know, having the shift level position indicate what gear its in, just isn't "cool"...

jpindar• 10 years ago

If you're interested in this kind of thing, you might like the book "Make It So" by Nathan Shedroff and Christopher Noessel, which discusses the interfaces used in movies.

How 'Minority Report' Trapped Us in a World of Bad Interfaces

Related Stories