Carnegie Mellon face recognition study has unsettling results

http://www.heinz.cmu.edu/~acquisti/face-recognition-study-FAQ/

by lemming • 15 years ago

201 91 15 years ago

heinz.cmu.edu

Reminds me of the TV series Lie To Me (03x13) called "Killer App" where the Zuckerberg character's social network uses cell phone camera technology to point one's phone at anyone's face in order to get their profile info.

Also reminded me a bit of the iris scanners in Leon, Mexico http://singularityhub.com/2010/09/26/iris-scanning-set-to-se...

wgx• 15 years ago

Meanwhile, in non-fiction...

http://www.digitalbuzzblog.com/facelook-coca-colas-facial-re...

qF2• 15 years ago

"Unsettling" might be a overstating it a bit. The first slideshow[1] explains (on slide 38, Limitations) that face recognition techniques still have a long way to go. More clearly stated on slide 39:

"Face recognition of everyone/everywhere/all the time is not yet feasible. However: Current technological trends suggest that most current limitations will keep fading over time"

[1] http://www.heinz.cmu.edu/~acquisti/face-recognition-study-FA...

lemming2• 15 years ago

However: Current technological trends suggest that most current limitations will keep fading over time

I'd say that is pretty unsettling, but then I worry about privacy. Given how facial recognition has advanced in the last 5 years, it's plausible that you could have an Android app to do their SSN trick with, say, 50% accuracy in the next couple of years - certainly 5.

jbondeson2• 15 years ago

No that really isn't plausible. Read their original SSN prediction paper [1] and you will see that the highest they got their SSN prediction (with < 1000 attempts, which is quite a few) was 8.5% for SSN's assigned after 1988. For 1979-1988 that number was 0.8%. Only in the smallest states with issuance after 1996 do they get their best numbers of +60% matches.

That's only the SSN portion. When you add in the facial recognition, the requirement of a facebook account with pictures and a correlation to place of birth, you're going to see those numbers plummet.

At best this is a clever hack that highlights the risks inherent with publicizing your life on places like facebook.

[1] http://www.pnas.org/content/106/27/10975.full.pdf

mcantelon• 15 years ago

I'm a bit unsettled when I see the US Army is funding this, given that they're also working on autonomous "terminator" drones.

libraryatnight2• 15 years ago

The robot assassins are coming. But in seriousness, I found this unsettling too.

neworbit2• 15 years ago

They fund a lot of things, but yes, I don't think I'd like to be on the receiving end of "You bear a distinct resemblance to someone we'd like to explode."

rhizome• 15 years ago

Current technological trends suggest that most current limitations will keep fading over time"

That may be so, but history and theory both tell us that the remaining issues (after eliminating "most current limitations") are going to be hard-ass bugs that prevent the technology from actually working, and that isn't even accounting for the Red Queen Syndrome.

A long way to go, but a short path to grants.

vectorpush2• 15 years ago

The maturation of this technology has additional implications even in the absence of an open social graph. Imagine sophisticated FRT used to track individual customer activity at retail locations. Timestamps could be used to map customers to their precise purchasing behaviors with infinite granularity. It's not far out to imagine a world where average profitability per visit is calculated upon arrival, now imagine an employee with a bug in her ear, marking you down as the guy with a 3 year history of strolling into Best Buy to browse TVs with the saleswomen, but never actually buying anything.

delinka• 15 years ago

"Timestamps could be used to map customers to their precise purchasing behaviors with infinite granularity."

Between debit cards and loyalty cards, this is pretty much already happening. In the US, I see few people using cash without the loyalty card (because the loyalty card gets you a discount.)

Of course, the remainder of your comment would require this new technology. And would be creepy as hell.

dhyasama• 15 years ago

The difference is you could be identified when you arrive versus when you are checking out. Your experience within the store would change. For example, if you spend a lot of money then you might be greeted with VIP welcome wagon. If you don't, you might be greeted with a cold shoulder.

vectorpush2• 15 years ago

The granularity I'm referring to is in the nuanced behavior of an individual in the act of shopping. Apparently, the algorithm has established that you follow a pattern of contemplating Cadburry chocolate bar purchases on every visit, well now your Inbox contains a buy 2 get 1 Free Cadburry coupon that only works for you.

Once this is cheap enough that anyone with a high def webcam can consistently track/identify faces, this technology is going to be used by everyone.

learc83• 15 years ago

This has major implications for loss prevention as well. External theft counts for about .5% of total revenue for the average retailer, which translates directly to a large percent of profit lost since margins are so low for many of them.

There is a lot of money to be made for someone who perfects a machine learning technology to detect shoplifters.

paganel• 15 years ago

> There is a lot of money to be made for someone who perfects a machine learning technology to detect shoplifters

This whole line looks like having been taken out of a Philip K. Dick novel. You're right, of course, which makes it even more creepy.

philwelch3• 15 years ago

What's more unsettling is the fundamental fact that any kind of social norm or expectation based upon the limits of our present technology is unsustainable.

ajryan• 15 years ago

“His very first story, he told me as he was dying, was set in Camelot, the court of King Arthur in Britain: Merlin the Court Magician casts a spell that allows him to equip the Knights of the Round Table with Thompson submachine guns and drums of .45-caliber dumdums.

Sir Galahad, the purest in heart and mind, familiarizes himself with this new virtue-compelling appliance. While doing so, he puts a slug through the Holy Grail and makes a Swiss cheese of Queen Guinevere.” ― Kurt Vonnegut, Timequake

sp3322• 15 years ago

I don't think it well be that bad, eventually. It's already creepy to walk into someone else's home uninvited, or enter a public bathroom marked for a different gender, or even just stare at someone for a while. None of these social norms are based on limitations of technology.

philwelch3• 15 years ago

Which means those norms are not necessarily unsustainable according to my statement.

adolph• 15 years ago

Q. Were the tests IRB approved?

Yes, they were approved. As in our previous studies, no SSNs (or faces) were harmed during the writing of this paper.

praeclarum• 15 years ago

Except, of course, for those few SSNs and addresses that Paul the grad student sold so he could play with a botnet. Oh, and we gave the database and algorithms over to the DoD who sponsored our work. I'm sure they won't harm any SSNs or faces either.

cletus• 15 years ago

This reminds me of a quote from Eric Schmidt (disclaimer: I work for Google) from last year. The full quote [1] is:

> I think judgment matters. If you have something that you don't want anyone to know, maybe you shouldn't be doing it in the first place. If you really need that kind of privacy, the reality is that search engines -- including Google -- do retain this information for some time and it's important, for example, that we are all subject in the United States to the Patriot Act and it is possible that all that information could be made available to the authorities.

which is entirely reasonable, yet he gets lambasted in the press as being "creepy" [2] based on a sensationalist snippet.

If you really want something to remain hidden, you have to remember that any services you use are subject to law enforcement, security breaches, bankruptcy procedures (there was a case not too long ago where a bankruptcy court judge ruled that the creditors' interests overruled the TOS and allowed the sale of information otherwise prohibited by the TOS; I can't find a link right now), etc.

With the ever-decreasing cost of mass computing, you will increasingly see just what an illusion online privacy really is.

Another example: some sites use shared avatar sites like Gravatar. I've often thought it would be trivial to crawl the Web and link identities from various forums based on using the exact same Gravatar image. In fact, someone's probably already done this.

Facial recognition is just one obvious way this will manifest itself. Another that I don't think has even been explored yet is what I would probably call "textual fingerprinting" meaning that the more you write in one post the more likely you can extract some identifiable style, choice of words and so on and that can be linked to other things you've written.

[1] http://www.schneier.com/blog/archives/2009/12/my_reaction_to...

[2] http://latimesblogs.latimes.com/technology/2011/01/animation...

mmastrac• 15 years ago

Even worse, Gravatar images are generally linked via MD5 hash. If you see two Gravatars with the same hash on two different sites, you've found the same person. Combine that with a large spam list of email addresses, or brute force simple addresses from common suffixes like @gmail.com and @hotmail.com and you've got identities.

jodrellblank• 15 years ago

Or combine with another service which uses MD5 hashes differently. You used to be able to copy a Gravatar hash into a Disqus url and see the email address, if that person had ever made any Disqus comments on any blog anywhere using their email address.

No brute-force necessary.

aidos• 15 years ago

And worse again, I just discovered that there's no way to delete a Gravatar account. Even if you unlink your email address from the account it still keeps it linked behind the scenes (you can sign in with your "deleted" address).

AJ0072• 15 years ago

While I don't disagree that Eric Schmidt's quote was taken out of context, "If you have something that you don't want anyone to know, maybe you shouldn't be doing it in the first place" is bullshit.

There are plenty of things you and I don't want anyone to know. I don't want the world to know when I'm on vacation. I don't want my neighbors to know I left my balcony door open and I just have the screen door closed so I conserve energy. I don't want every girl I meet to know that I collect comic books. The idea that I shouldn't be doing any of those things is completely absurd.

In the same fashion, would you like everyone to know what parking spot your teenage daughter parked in, in the back of a mall parking lot Friday night? Or where your kids go to daycare? (hypothetical, I don't have kids but you bet I'd be concerned about them using foursquare) Could we change Eric Schmidt's quote to, "If your kids are doing something you don't want everyone to know, may be they shouldn't be doing it in the first place?" I don't think so. Your kids aren't that much different than your 60 year old mother or 70 year old dad. That is a lot of people.

It was hardly 10 years ago that the message was - never meet anyone you first met online. What happened to that? Now the message is, share all your personal private information with the entire world, you have nothing to worry about. We are Facebook/Google and these features are amazing. That is a lie.

Perhaps the parallel increase in information access by law enforcement has made us feel safer. The same technology criminals use to exploit us is used to track them. When something bad happens to someone we know more about what happened, quicker. While this may be an acceptable trade off in a first world country I do not believe the same applies in one such as Mexico.

Yeah online privacy is nearly non existent. Especially so when you are reckless and use connected identity systems. That doesn't mean the message should be "give up." That is the message Google and Facebook's PR machines are sending because they can make a lot more money from advertising when they do it.

As soon as hacker news switches to facebook's commenting system or any other public id, I'll never make another post here again. I have left communities I've been active in because of this. The message to our friends and family, and if we own businesses, the world, should be stop sharing all of this information publicly. There is no reason to. Its not safe for you and its not safe for your family.

true_religion• 15 years ago

> There are plenty of things you and I don't want anyone to know. I don't want the world to know when I'm on vacation. I don't want my neighbors to know I left my balcony door open and I just have the screen door closed so I conserve energy. I don't want every girl I meet to know that I collect comic books. The idea that I shouldn't be doing any of those things is completely absurd.

The point is that if the world really wanted to find out about these activities, there is someone in the world with the information who knows of it.

Just taking one of your examples: comic books---you buy them from some source, and that source likely knows who you are.

Absolute privacy is impossible unless no one is interested in your activities, and your activities don't involve any other human being or leave any trace.

troyastorino• 15 years ago

> Yeah online privacy is nearly non existent. Especially so when you are reckless and use connected identity systems. That doesn't mean the message should be "give up."

I don't think the question is over whether it is worth it to fight for online privacy, but rather if online privacy is the right end goal to begin with.

People are very uncomfortable with radical openness and transparency of their personal lives because it is a radical departure from the way life used to operate. People have always been uncomfortable with technologically driven cultural changes (easy examples would be the original Luddite movement or luddites today), and probably always will be. Online privacy gives an ironic twist on the situation since many hackers, the people who should be pushing the world forward, are disturbed by this change and want to hold it back.

Facebook, and more recently Google (with their real name push), have moved to envision this different mode of human interaction. In its extreme incarnation, your digital and 'physical' lives are inextricably merged. I don't think this is a trend that can or should be fought: it would mean fighting the integration of technology with day to day life, and so would eventually be analagous to fighting the benefits technology can bring to human life. With this integration, though, your personal life accessible to those who want it. Yes, having everything be open is very different and uncomfortable. But look at the environment created by allowing total anonymity: 4chan /b/. I feel comfortable saying that I would rather see human interaction move in a direction where you are responsible for all of your actions, rather than one where you can avoid all responsibility.

wnight5• 15 years ago

> "If you have something that you don't want anyone to know, maybe you shouldn't be doing it in the first place" is bullshit.

No, it's fact.

> There are plenty of things you and I don't want anyone to know.

What good does that do us though?

> I don't want the world to know when I'm on vacation.

But they can easily. A webcam watching the street could make it easy to tell whose lights are on a timer, or not on at all. Google not needed.

> I don't want every girl I meet to know that I collect comic books. The idea that I shouldn't be doing any of those things is completely absurd.

No, it's perfectly reasonable. After all it's the only way you'll get what you want.

And if you find a girl who doesn't know you collect comics she'll be disappointed when she figures it out. Wear it on your sleeve and be happy. Ditto porn, whatever.

> It was hardly 10 years ago that the message was - never meet anyone you first met online.

That's what FOX and CNN were telling you to panic about, yes. Do you really live your life in so much fear? Never physically meeting people you met online because they might be serial killers?

> In the same fashion, would you like everyone to know what parking spot your teenage daughter parked in, in the back of a mall parking lot Friday night?

Think of the threat vectors. There are a lot of pretty girls out there so this one specific girl isn't going to be sought by online creeps around the world. Everyone knowing doesn't change anything, except perhaps to let me keep an eye out for her.

If anyone hurts her it'd be a random attacker who spotted her in the mall or a friend/family who had stalked her. The first wouldn't care where her car was - she'd lead them to it. The second would recognize the car.

> Or where your kids go to daycare?

Oh yes. Because your crotch-fruit are so special that perverts would see them online and have to have them, specifically. Again, the only threat to your children specifically is from your family, and they know which daycare you use.

> Perhaps the parallel increase in information access by law enforcement has made us feel safer.

Hah! Those snoops are the problem. Technology is making us safer by letting us finally know what our 'servants' are really up to.

Radim• 15 years ago

Re. "textual fingerprinting" -- this is a well-established field (decades old), especially in forensics. It more commonly goes under the name "authorship attribution".

For a taste, see e.g. http://www.ncfta.ca/papers/emailforensics.pdf or http://ai.arizona.edu/intranet/papers/isi_visualizing_author... .

jonnathanson• 15 years ago

"Another that I don't think has even been explored yet is what I would probably call "textual fingerprinting" meaning that the more you write in one post the more likely you can extract some identifiable style, choice of words and so on and that can be linked to other things you've written."

Forensic textual analysis predates the internet, actually, and I can say with near certainty that someone out there is working on this right now. (Probably the CIA, and/or another agency involved in analyzing communications between persons of interest).

Back in undergrad, I was involved in a project that used textual analysis to surmise authorship of "anonymous" works of fiction from Shakespeare's day. Additionally, the authorship of a lot of works attributed to Shakespeare is often called into question, and so we tried to map for consistency or inconsistency in his works. (Spoiler alert: he turned up pretty clean; he -- or at least someone using his name -- is most likely the author of his works).

Now, obviously, this project was a bit high-touch and involved a lot of attention lavished on a single body of work by a handful of presumed authors. But that was a long time ago, and I'm sure that the technology has grown more scalable and sophisticated by now.

boredguy8• 15 years ago

> Spoiler alert: he turned up pretty clean; he -- or at least someone using his name -- is most likely the author of his works

Do you have a handy reference?

anigbrowl4• 15 years ago

Forensic textual analysis predates the internet, actually, and I can say with near certainty that someone out there is working on this right now.

Plagiarism detection software is increasingly common in education. I've actually been looking for something similar to filter news commentary systems, to get a sense of what %age of comments on political stories are actually astroturf.

neworbit2• 15 years ago

Well, it is kind of creepy. I wouldn't want a "social network helper tool" to out me if I was still in the closet. Some things should be your own decision.

5hoom• 15 years ago

Thats just it. It's not for someone else to decide what information you project about yourself.

Types of information that are of little consequence for one person (eg. religion, sexuality, political-affiliation) could have devastating real world consequences for another if made accessible.

ezy4• 15 years ago

Generally, people who subscribe to the ridiculous notion that "If you have something that you don't want anyone to know, maybe you shouldn't be doing it in the first place.", either lead extremely pedestrian personal lives, or are rich enough to be able to buy their way out of repression. Most big company CEOs are a little of both, in my experience.

sliverstorm3• 15 years ago

I usually just take it as a more PC way of saying "If you have something that you don't want anyone to know, either learn to cover it up or don't do it in the first place".

I mean, since when has it been somebody else's problem that you couldn't hide your secrets well enough?

philwelch3• 15 years ago

Lots of people lead far-from-pedestrian personal lives but don't feel compelled to hide them. It wasn't too long ago that merely being gay was a horrible secret you hid from people; do you really think society is going to grow less rather than more tolerant as time goes on?

shoham• 15 years ago

All good points. Where do you stand, on an ethical basis, with how far is too far for any entity to knowingly take this kind of info and manipulate your life for your (and their) 'benefit'. I like Google, but when you mention 'textual fingerprinting' it's already the case that Google does some of this when you search, and for use in Adsense, right? Maybe not reading your texts/posts yet but Google has 100's of millions of people's search and browsing fingerprints, whether they know it or not...

dspillett• 15 years ago

> which is entirely reasonable, yet he gets lambasted in the press as being "creepy" [2] based on a sensationalist snippet.

The creepiness factor in his comment, and in the way advertisers and those selling us to advertisers operate, isn't really about the information being out there. It is about being followed by an entity you don't know and have no particular reason to trust.

We don't like being followed like that, there is something in us that takes it as a threat. It is something we share with most animals and is a sensible reaction that evolved by making animals with that reaction less susceptible to certain threats (in the wild, being followed usually means you are a target for something you are not going to like, probably involving claws and teeth). Even in modern life it can be a useful reaction - if someone follows you down an alley you get ready to take action just in case that person turning down the alley after you isn't a perfectly innocent coincidence.

Being followed by Facebook/Google/any-one-else-on-the-planet-who-wants-to-sell-info-about-my-life is not the same as the physical risk that being followed in RealLife(tm) could imply, but our brains are not wired to see the difference. Anyway, people knowing more about you that you know about them can be a threat in some circumstances, or at least could put you at a disadvantage.

A secondary issue is that people sometimes want/need to hide things because other people's reactions could have unreasonable negative repercussions. If you were a gay man in a place where being so could get you strung up for what local laws (or local vigilantes) consider to be moral deviancy, "just stop being gay then you'll have nothing to hide" simply isn't an option and you wouldn't want the information "leaking" out by virtue of services giving you personalised adverts relevant to it while using facebook (or any other ad funded service) in public. That may be stretching the point a little far, but there are reasons to want to keep things about yourself to yourself (without wanting to completely disconnect that part of yourself from your private online activities) other than it being illegal, which if I remember rightly was what Eric Schmidt was talking about when he said that oft-quoted-out-of-context phrase, and sometimes something being illegal doesn't make it wrong (morality being a lot more subjective than peopel like to think - what is wrong to me might be fine to you).

RexRollman• 15 years ago

So basically, we aren't far from turning Flickr and Facebook photos into a surveillance system? Someone could be found that way without even having an account.

MarkPNeyer3• 15 years ago

i figured a while ago that any notion of online privacy was silly. i said 'fuck it' and decided to just be totally open about who i am and what i think online. i talk openly about having bipolar disorder and experiencing psychosis, about beliefs i have that are considered unusual or downright insane, and about whatever i'm feeling at the moment.

it seems to work out pretty well for me. i was even thinking of publishing everything i buy for the world to see, both as an exercise to control my spending and for fun.

pork2• 15 years ago

So you've now attached your name to admissions of psychosis. I understand your motivations on some level, but I sincerely hope that you don't regret doing so in the future. You wouldn't staple your medical information on your business card, but I feel you've done just that here. Good luck, and I mean that sincerely.

MarkPNeyer3• 15 years ago

i've done this so many times it's not funny. it hasn't stopped me from getting a job yet. my thinking is that this stuff will come out eventually anyhow, so it's better to be up front about it.

people seem to like honesty.

jasontsui• 15 years ago

Ive done the same myself. To me, the real issue was how time consuming it was to "cover my tracks", so to speak. After taking a look at it, I decided there wasnt much to hide. Sure, you may not want everyone to know you were at the movies last night, or that youre a regular at dubstep shows, but what are the possible negative outcomes? Having people know more about me hasnt been as dangerous some make it out to be. On the other hand, its given me plenty of opportunities to connect with similar people. Also, the idea of activities in your personal life interfering with your work image is outdated. Everyones got a personal life, why should we back away from that? Its who you are.

froseph• 15 years ago

Equally unsettling is the potential of people tracking enabled by the government's increased use of surveillance cameras and large database of photo ids.

libraryatnight2• 15 years ago

It's moments like this where I wish I could somehow delete myself from the internet and come back fresh armed with what I know now.

Personal electronic reset button.

randomdata• 15 years ago

Even if you are careful, all it takes is a kind friend on a site like Facebook to tag you in a photo (you don't even have to be a member to be tagged) and you are back to square one.

On the upside, I am super excited for Facebook Goggles. Augmented reality profiles shown for everyone you look at. Out in the dating scene? Know who is single and who is not just by looking at them. Business convention? Find out who everyone works for at a glance. Okay, maybe it isn't an upside, but it is bound to happen anyway.

bitops• 15 years ago

It's exciting and also deeply disturbing. I agree with all the positive benefits.

But after a while I think there's a real tendency to just reduce people to a set if metrics in your mind. It seems scary to me that the more "connected" we are we are often just self-selecting or being selected into some group based on patterns.

ja27• 15 years ago

Is anyone watching TechStars on Bloomberg? When I saw the Wiji guys and their adaptive advertising, I was immediately thinking of the implications if you can do this good of a job of facial recognition. Imagine if every ad screen you came near knew as much about you as Google or Facebook do. In some ways I'm kind of excited about it - it's like living in the future - but it's scary as heck for privacy concerns.

http://immersivelabs.com/

lukejduncan• 15 years ago

tldr: study shows that it's easy to find you on social networks based online on pictures. Experiments were done to show that you can be found between sites (e.g. a picture on an adult dating site can be used to find you on Facebook) and by taking pictures of random students on campus.

idonthack• 15 years ago

Anyone with half a brain has seen this coming for years.

astrofinch2• 15 years ago

You could trivially "opt out" of this by ensuring that no publicly available photos of you were tagged on Facebook.

saucetenuto• 15 years ago

How do you ensure that? Recall that the person who uploads the photo controls how widely it is shared, and that you can be tagged in a photo even if you're not subscribed to Facebook.

astrofinch2• 15 years ago

Stay registered and remove tags you don't like.

stretchwithme• 15 years ago

Its going to become very difficult to commit all kinds of crime.

And that is worth some loss of anonymity, provided we still have individual rights and the rule of law. Without those it won't matter what technology can do.

yaix• 15 years ago

It will come sooner or later. Point your smartphone at any person in the street and know who he or she is in a second. The technology is too trivial to stop it.

More important would be to now begin thinking about regulations to limit abuse of that technology.

But most likely governments will wait (again) until the system has been abused by big business before limiting regulations are enacted.

Carnegie Mellon face recognition study has unsettling results

Related Stories