A Face Recognition Algorithm That Finally Outperforms Humans

https://medium.com/the-physics-arxiv-blog/the-face-recognition-algorithm-that-finally-outperforms-humans-2c567adbf7fc

by Mz • 12 years ago

147 31 12 years ago

chriskanan• 12 years ago

This isn't really face recognition, this is face verification. In computer vision, face recognition usually means tell me who this person is (in psychology it means "have you seen this person before?"). Face verification gives an algorithm (or a person) two images of faces and asks, "Are these the same person?"

Their result is impressive, and it improves a bit over Facebook's recent result on the same dataset with their DeepFace system (97.35% for DeepFace vs 97.53% for people vs 98.52% for the system discussed in the article).

Also, it is interesting that they are not using deep learning for this. They are using a Discriminative Gaussian Process Latent Variable Model.

sp332• 12 years ago

I think the breakthrough with Facebook's system was the idea to transform the face before running the training or verification on it. And the new system seems to use the same idea: The new algorithm works by normalising each face into a 150 x 120 pixel image, by transforming it based on five image landmarks: the position of both eyes, the nose and the two corners of the mouth.

apu8• 12 years ago

No. Alignment is done by every face verification method, and even the "new" 3d alignment done here is on par with other alignment methods in practice today.

At their talk at CVPR, they made clear that the biggest factor in getting their numbers was having tons of training data and a high-capacity learning model. The paper even shows that if you replace their 3d alignment with a standard 2d affine alignment (something people were doing 15 years ago), you lose only 1.5% on LFW (table 2).

pessimizer3• 12 years ago

Face verification gives an algorithm (or a person) two images of faces and asks, "Are these the same person?"
97.53% for people

I'm surprised that people are that good - I think I'd be horrible at it. From googling, I can see that number came from Facebook - would you happen to know where to find the paper?

Also, it is interesting that they are not using deep learning for this. They are using a Discriminative Gaussian Process Latent Variable Model.

Does using this different model gain some sort of space or processing efficiency over DeepFace?

Also: does anyone know the point of fighting to close this 3% gap? That seems good enough for any purpose. (Is there any other purpose to this technology other than mass surveillance?)

apu8• 12 years ago

Here's the paper: https://www.facebook.com/publications/546316888800776/

The methods are quite different, but I think at root, it's just more being able to take advantage of much more training data.

The 97.53% "human performance" number comes from my paper from a few years ago [0]. We initially thought that there's no way we would be able to do that well on this task (manually) either, but then we tried it and our scores were much higher (very close to 100%). So the verification task on LFW is actually not that hard, although in part it's because the dataset consists of public figures and therefore you can recognize many of the people outright if you've seen them before. (We tried to measure this in our human studies but found it to be inconclusive.)

3% is not the right way to look at error rates, btw. See [1] for why. But also, face verification is more of a low-level building block for recognition, because most real-world applications don't directly care about saying whether two faces are the same person or not ("verification"), but "who is this person?" ("recognition"). A simple way to build recognition from verification is to compare a test face against all faces in the database, and then take the one that gets ranked highest. In that (simplistic) scenario, your recognition rate scales (very roughly) as acc^lg(N), where N is the number of people in your database and acc is the verification accuracy. So if, e.g., you had 100,000 people to recognize from and your verification accuracy was 97.53%, you might have a recognition accuracy of ~66% (very roughly!), which is not great.

Finally, there are already lots of applications to face recognition other than surveillance, and many more waiting to be discovered.

[0] http://homes.cs.washington.edu/~neeraj/projects/faceverifica...

[1] https://news.ycombinator.com/item?id=7393438

dheera2• 12 years ago

I'm pretty sure in reality Facebook also uses your social network graph to restrict the candidate set and get higher recognition accuracy. This makes it hard to compare Facebook's results to a pure recognition algorithm.

On that note, Facebook could do even better by restricting the candidate set based on time, precise location, and compass orientation, given that most mobile users have Facebook installed and are running it in their pockets when they get their picture taken by others. (They could do rough recognition purely based on position and orientation without even looking at the camera image, if they really wanted to, so with the camera image it could really be near 100% accurate, and even work if you take a picture of a friend's back.)

apu8• 12 years ago

Sure, but the numbers in the research paper are "pure" (i.e., not using all this additional information). In production, I'm sure they must be using all these additional cues as well.

dkhar• 12 years ago

DeepFace's figures[1] also come from running it on the LFW dataset[2]. While Facebook's production tech will be more accurate because of what you say, the Deepface algorithm has a lot of raw power, too!

[1] http://www.cs.toronto.edu/~ranzato/publications/taigman_cvpr...

[2] http://vis-www.cs.umass.edu/lfw/index.html

hcarvalhoalves2• 12 years ago

Also, it is interesting that they are not using deep learning for this. They are using a Discriminative Gaussian Process Latent Variable Model.

It's my understanding that deep learning is more interesting when you have a large dataset and need to generalize (e.g., recognize faces sideways, upside-down, etc), otherwise, when the requirements are more predictable, the hand-tuned algorithms tend to perform better. The task of finding whether two pictures are from the same person doesn't seem appropriate for DL (compared to e.g. find whether a picture contains a person).

Would that be correct?

apu8• 12 years ago

I'm not sure that's the distinction I'd use. To me, it seems more to be a matter of the kind and amount of training data you have.

If you have a ton of labeled data (either for the exact task you're doing, or a related one), DL often wins because it's sufficiently high-capacity to learn sophisticated models from all that data.

On the other hand, if you don't have enough data, then the deep network isn't going to be learnable, in which case hand-tuned methods might do better, depending on how much "domain expertise" you can bake into the system. Faces are a very constrained domain, and since there has been enormous work on them for decades now, it's one area that non-DL algorithms probably still have somewhat of a fighting chance (until people gather & label large enough databases, of course).

jastanton• 12 years ago

even with Facebook's 97.35% they still confuse me with my toddlers :P

PeterisP• 12 years ago

Actually, for humans this actually is a face recognition task - from the given examples, it seems that the dataset involves pictures of publicly known people, so it's comparing apples and oranges as humans are given an entirely different task.

Computers are given 2 pictures and asked "Are these the same person?"

However, if people are given the same 2 pictures, and one of them is a well-known actor; then the question is "Is the second picture the same person as the many pictures of this actor that I have seen during my life?" which is a rather different task.

If you want a fair comparison, then either you have to use portraits of people that nobody knows, or the computer systems need to use databases of celebrity pictures for that.

apu8• 12 years ago

sigh no, it doesn't "outperform humans" on "face recognition". In particular, see my previous comments [1] and [2] for discussion on why this method might be doing well.

As for "outperforming humans", a more accurate statement might be, "this algorithm outperforms (for this simplistic task) one experiment done with a limited set of humans on this one particular dataset which has been in the community for 10 years now and is thus highly gameable."

But I realize that's a lot less pithy.

In particular, this dataset is nearing saturation, and whenever that happens, differences in the accuracy numbers often don't mean much. So for example with Facebook's number at 97.53% and this paper's at 98.52%, you're talking about the difference between getting 148 pairs of faces wrong vs 89 pairs wrong. In practical terms, as a researcher working with a dataset like this, you very quickly learn to focus on just the ones your algorithm gets wrong, and it's impossible to not subconsciously try to optimize for getting those few cases correct, even if those techniques wouldn't actually help in the general case.

[1] https://news.ycombinator.com/item?id=7637866

[2] https://news.ycombinator.com/item?id=7638269

darkmighty• 12 years ago

https://class.coursera.org/ml-005/lecture/39

galapago• 12 years ago

https://www.youtube.com/watch?v=DQWI1kvmwRg

yamaneko• 12 years ago

I just love all the analysis you've done about these face recognition systems. They're so insightful!

yummybear• 12 years ago

You seem to know something about this - how accurate are the best face detection algorithms on real-world datasets?

KaiserPro• 12 years ago

depends on the lighting, camera setup, CPU, available processing time and finally your intended purpose.

Face detection/recognition is a broad subject, and the applications are broad.

There are things like HAAR cascades which are able to pickout a face like object from others (or anything else that it's been "trained" to do), however they can't tell faces apart. They can be tuned so that they can be used in realtime apps (like the autofocus on cameras)

HAAR cascades are limited in a number of ways: they can't tell faces apart, and you need a different cascade for different views (profile/portrait/other) they can also have trouble with skin colour as well(depending on training).

more advanced algorithms are able to workout face orientation in realtime (google hangout moustaches and the like) but once again they arn't able tell between two faces.

However there are no accurate real-time (or anywhere near realtime) algorithms that are able to tell faces apart (i.e. put a name to a face in a crowd) In fact I would go so far as to say that there are no non-realtime ones either.

drpgq2• 12 years ago

If you want to see results of state of the art commercial systems see NIST'a recent FRVT 2013's test results:

http://www.nist.gov/itl/iad/ig/frvt-2013.cfm

gizmodo59• 12 years ago

I had to say this, the articles from medium.com has more focus on PR than the quality of content. Today itself I saw two articles, Schrodinger's cat and this.

shreyassaxena• 12 years ago

No it is wrong, the actual human performance on full images is close to 99%. [1] Ctrl+F human

The performance of human it reports is on cropped images shown to human, which is not a fair comparison.

[1] http://vis-www.cs.umass.edu/lfw/results.html

apu8• 12 years ago

No, the 97.53 is more fair. See here: https://news.ycombinator.com/item?id=7638269

drpgq2• 12 years ago

NIST's latest FRVT results for commercial face recognition (FRVT 2013) is available here:

http://www.nist.gov/itl/iad/ig/frvt-2013.cfm

fenollp• 12 years ago

But when the algorithm is faced with images that are entirely different from the training set, it often fails.

Over training it is.

A Face Recognition Algorithm That Finally Outperforms Humans

Related Stories