Microphone turns any surface into a multitouch interface

http://www.extremetech.com/extreme/111847-microphone-turns-any-surface-into-a-multitouch-interface

by mrsebastian • 15 years ago

65 18 15 years ago

bradleyland4• 15 years ago

The title is grossly misleading. This setup detects "gestures" which is not the same as the multitouch interfaces we use with our smartphones. Bloggers threw "multitouch" on the title; not the creator [1].

Nothing in the original article indicates that the setup is able to detect the location of the touch, only the type of touch. A thump, a tap, a swipe, or a scratch. The various gestures are mapped to different sounds, creating a virtual drum set, if you will. The fact that this detection, mapping, and playback happens so instantaneously is amazing.

I don't mean to take away from the accomplishment. This is an extremely novel idea, and the product implications are numerous. It's just a shame that the tech press is distorting the product idea in a way that creates a false expectation.

http://www.brunozamborlin.com/mogees/

r3demon• 15 years ago

Yeah, multitouch is a wrong word here.

mojuba2• 15 years ago

Also this:

> ...contains multiple microphones to create a stereo image of the sounds it hears

led me to believe the device determined the direction of the sound source by measuring phase shifts in different microphones. Now that apparently this device doesn't do that, may I ask why not?

bradleyland4• 15 years ago

I can't find any reference to multiple microphones or phase shift analysis in the source material. This paragraph from the source material, specifically, leads me to believe it's using the acoustics of the touch, not the location:

> Through gesture recognition techniques we detect different kind of fingers-touch and associate them with different sounds.

Notice the language "different kind of", rather than "the location of". I just don't see anything here that would indicate spacial information is gathered.

mrsebastian• 15 years ago

Hrm, if you watch the video, the tone definitely changes with the location of the hit -- as in, with the same radius from the microphone, a different note is produced depending on where a finger strikes the surface. Gestures are also included, as you say.

With regard to multitouch -- a) some gestures involve more than one point of contact, and b) there are some bits in the video where it looks like two fingers are striking a surface at the same time and making two tones/noises.

bradleyland4• 15 years ago

Depending upon the type of material, the tone will change when you strike it. I still think it's a mistake to say that this turns a surface in to a "multitouch" surface, because of the vast differences between this and what has commonly been referred to as multitouch.

Again, extremely cool technology, but you couldn't for example, attach a microphone to a wall and couple this with a projector to control the OS. That's the inference that gets drawn when you say "multitouch".

raphman2• 15 years ago

Agreed. Unfortunately, this seems to be pretty common in online tech journalism. My own experience: We published a paper on another touch sensing technique last year which got featured in a short article for Technology Review. The author interviewed both of us and wrote a very good article. Over the following days, a number of sites like Engadget picked up the story and apparently only used content from the TR article. IIRC not a single author reported additional details or even contacted us to confirm this story.

This preference of speed over accuracy leads to stuff like 'Wi-Fi Makes Trees Sick, Study Says' [1].

[1] http://www.pcworld.com/article/211219/wifi_makes_trees_sick_...

raphman2• 15 years ago

More or less same technique, presented at UIST in October 2011:

http://chrisharrison.net/index.php/Research/TapSense

ww5202• 15 years ago

This is a very creative idea and execution. The next step would be to use two microphones sitting at a distance to form a triangulation to determine the 2D location of the touch. Then multitouch can be truly supported.

edlea• 15 years ago

I can't work out if that would be possible or not.

It might be possible to plot the location of a single point in 2D, but how would you differentiate for multitouch? If you have 2 microphones you might be able to calculate the distance from each microphone of a touch by timing how long it takes for the sound of the tap to reach each microphone. However, as the speed of sound in wood appears to be ~4000 m/s (http://www.engineeringtoolbox.com/sound-speed-solids-d_713.h...) you'd need to be "listening" at 400kHz to get centimetre accuracy over a 1m square area.

I can't see how it would be possible to then differentiate between two points moving at the same time (i.e. a pinch gesture). I'm just assuming regular microphones are used as the article suggests - I'm not an acoustic engineer so hopefully I've missed something.

kabdib• 15 years ago

Use a phased array of mics. Classic technology (well, 1960s).

Probably much harder with random mic placement.

ww5202• 15 years ago

Interesting problem. Haven't thought of the sampling rate required. The location resolution probably will be poor for precise multitouch function. But even a single 2D location is enormously useful - turning any surface into a touch screen.

kabdib mentioned phrased array, which can be very helpful. It might solve the sampling rate issue since it's analog. I found this http://en.wikipedia.org/wiki/Beamforming. May be some of the techniques in radar/sonar can be borrowed here.

mhb• 15 years ago

Not exactly the same, but worth mentioning:

Acoustic tablets have a long technological history, as seen from this 1971 patent...

http://en.wikipedia.org/wiki/Acoustic_tablet

omegant• 15 years ago

Why Is this site so wrong with iphone safary?( stuck page, difficult to scroll down, unable to go back to HN...). I always promise myself I'll not touch again this web with a 5' pole( and always forget :/)

NHQ• 15 years ago

Contrary to the bloggers report, there is more info and source available:

http://ftm.ircam.fr/index.php/A_brief_introduction_to_FTM

http://imtr.ircam.fr/imtr/Gesture_Follower

Microphone turns any surface into a multitouch interface

Related Stories