PointerPointer: points to your cursor

http://www.pointerpointer.com/

by nthitz • 14 years ago

367 65 14 years ago

This is pretty neat, I wonder if it has to be human trained or if there's some sort of computer vision algorithm determining pointing.

siim• 14 years ago

After looking into to the code I found out that this app is made by using Voronoi diagrams. [1]

The actual positions are saved in a json file. [2]

[1] http://en.wikipedia.org/wiki/Voronoi_diagram

[2]http://www.pointerpointer.com/gridPositions.json

slig• 14 years ago

Yes, but how did the author create that .json file? Manually or using some ML algorithm?

Groxx• 14 years ago

Definitely good for a chuckle. It helps a lot that the photos are very well aimed, really makes it feel magical :)

pyre• 14 years ago

The images are adjusted. I was able to get the same picture 3 times in a row by moving cursor slightly, but the finger was always aligned exactly the same on the cursor, even though I had moved it.

Inufu2• 14 years ago

doesn't really work for me with Chrome on Windows 7 - an image flashes for a split-second, then it's back to "Finding pointer" (I'm not even touching my mouse)

simonbrown• 14 years ago

It seems to work for me when I move the cursor off the page, though that misses the point.

AsylumWarden• 14 years ago

Same here. Just flashes the picture really fast in the latest stable release of Chrome under Windows 7. Works beautifully in the latest Firefox though. Windows IE isn't supported at all apparently (I tried IE 9).

Are the developers willing to share some insight into how this works? The picture selection is right on and there seems to be a good number of pictures at that.

zentiggr• 14 years ago

Agreed - never quite settles to actually show anything, just keeps asking me to hold still even though my hand's off the mouse...

Inufu2• 14 years ago

intersting: on Ubuntu 12.04, still with Chrome, it works

LoneWolf• 14 years ago

Same problem windows 7 x64 Chrome 19.0.1084.52

jQueryIsAwesome• 14 years ago

It looks like Chrome is firing the "mousemove" event when it shouldn't.

One way to make it work is to right click where you want it inside the square it and then left click anywhere outside the square.

In the Javascript looks like the error could be patched in Flasher.js line 72; save the last event.clientX and event.clientY and then check if the mouse coordinates have actually changed.

saraid2162• 14 years ago

Snow Leopard/Chrome 19 works fine.

Seems like it's Win/Chrome.

sparknlaunch• 14 years ago

Doesn't work in Chrome but the 'right mouse click' hack works.

There are likely some good opportunities for this code however speed of response may be a problem? Runs a little slow on my PC.

patrick_t• 14 years ago

doesn't work on Windows 7 Chrome 20.0.1132.27

tshadwell• 14 years ago

Works fine in Chromium 21.0.1136.0 (136796).

hackNightly• 14 years ago

Just wasted about 20 minutes of my life playing with this. I would really like to know the technology behind. Is there some sort of ML going on, or is this human trained? Either way, very entertaining and scary accurate.

xd• 14 years ago

You wasted 20 minutes ... what makes you think someone didn't waste 20 days pulling the image collection together. Occam's razor ;)

almost• 14 years ago

All the images are listed here:

http://www.pointerpointer.com/gridPositions.json

There are around 900 of them! I'd have used Amazon's Mechanical Turk...

Very cool effect, I also wasted way to much time playing with it :)

B-Con• 14 years ago

That's what I assumed. If I just moved my mouse a couple pixels I usually got the same pointing image back. I think they just divided it up into regions and then load the appropriate image for each region.

However, if that's all they're doing, it seems odd that it takes so long to load... Maybe some of that's just a built-in delay to keep from constantly cycling images from someone who's bumping the mouse.

ww520• 14 years ago

Something as low tech as having a predefined map of {location : picture} would do it. Break up the screen into a grid of NxN quadrants, like 20x20. Then just find 400 pictures with a finger near each quadrant. Translate cursor to a quadrant x,y; look up picture. Done.

DiabloD3• 14 years ago

On Firefox, never goes past "Pointer found, pointing". No image loads, works on Chrome though.

runako• 14 years ago

Interesting example of how a hard problem (image recognition/understanding) can be faked for some cases by solving an easier problem (lookup table, a few hours of manual intervention.)

taylorbuley• 14 years ago

Seeing http://www.pointerpointer.com/gridPositions.json really took away some of the magic for me. Still, very clever and shows that web magic can be in the eye of the mouse-holder!

ma2rten• 14 years ago

Erm.. even there were some computer vision algorithm involved, he would probably want to run it offline.

ajasmin• 14 years ago

So there's no advanced computer vision algorithms involved :(

cocoflunchy• 14 years ago

Doesn't load for me, on Chrome/FF/IE9...

paulirish• 14 years ago

I made a quick video explaining how the webapp works: http://youtu.be/Z2ZXW2HBLPM

Essentially, using the gridPositions.json to populate a Voronoi Grid (toggle display of the <canvas> in your browser). Then your cursor position is used against the grid to find which shape encloses it which maps back to the source JPEG which is then displayed.

snowwrestler2• 14 years ago

I watched your video. It does not seem to explain how the fingertip position in each photo was calculated, which it seems to me is really the heart of the magic.

Flam• 14 years ago

He says how it's basically a json file with every image mapped to coords of each voronoi "tile". I guess it was handmade in advance

joejohnson• 14 years ago

Yes, I was wondering the same thing. The difficult engineering problem here is first assigning pointing locations for every photo in the database. Because he didn't explain this step, I guess he just manually assigned each location for every photo.

paulgb• 14 years ago

If anyone doesn't care to explore the DOM themselves, this will show the Voronoi diagram and make the photo on top of it transparent. For Chrome, Ctrl+Shift+J and paste it into the console.

  document.getElementById('data').setAttribute('style','');  
  document.getElementById('content').setAttribute('style','opacity:0.7;left:0;top:0;margin:0');
  document.getElementById('underlay').setAttribute('style','left:0;top:0;margin:0');

ionforce• 14 years ago

I don't see the point of this. The images load far too slowly.

hanbam• 14 years ago

I was actually expecting that somebody did some machine learning / matching over a large set of pictures and found all those images / coordinates.

shashashasha• 14 years ago

A faster (older) version of this by the same people, with camera flashes: http://flitser.org/

timmaxw• 14 years ago

Warning: at least one of the images is not "family-friendly". Are the images automatically collected?

hoprocker2• 14 years ago

It's probably easier to automate the picture harvesting here via an intensity filter. I'm still stumped about how the finger-recognition was accomplished in the OP.

chousho• 14 years ago

Haha, very clever. Although I seem to be running into problems whenever I put the mouse into one of the 4 corners of the screen. It can find anything more centrally located, though (apparently?)

alpb• 14 years ago

There are serious problems around the upper right corner. I did not get any results from there. I'm wondering whether the pictures are originally pointing that coordinate or are they resized+translated to some offset?

Geee• 14 years ago

You'll have to move the cursor left a bit, to get the same image, but with visible finger. The images are indeed offset so that in the upper right corner the finger actually doesn't fit in the frame.

kephra• 14 years ago

One suggestion is to move the jquery from jquery.com to your domain for two reasons:

1: It will be one click less, for those who use noscript. 2: It will allow your site to survive the case that jquery.com changes anything, or goes down.

Besides that its a nice toy.

jdpage• 14 years ago

Alternately, use the Google CDN.

sporksmith• 14 years ago

Fun :). I did something similar with a bunch of photos of a stuffed monkey. I made a music video instead of an interactive web app, though the latter would probably not be too difficult since I wrote the software using Processing. http://sporksmith.wordpress.com/travels-of-code-monkey/

joshu• 14 years ago

Why does it take so long to load?

apu• 14 years ago

I was wondering the same thing. With a precomputed voronoi diagram and only a few hundred cells, lookups should be almost instantaneous.

Inetgate• 14 years ago

Is this a some kind of concentration game?

jsilence• 14 years ago

awesome!

edwinyzh• 14 years ago

What's this? I saw some pictures flashed after some loading indicators are shown...

Andi2• 14 years ago

What is the point about this beyond showing pictures where the finger is next to my cursor? (Why is this popular?) It's not more interesting than a circumference search. You could implement that with a simple rtree.

AznHisoka• 14 years ago

How can I get a pointer to my cursor in Javascript? I might find this useful in automating certain processes.

gouranga• 14 years ago

Doesn't work in IE9. That was the limit of my experience :(

PidGin128• 14 years ago

This is technically interesting, but voyeur-istically creepy.

PointerPointer: points to your cursor

Related Stories