Facial recognition can predict person’s political orientation with 72% accuracy

https://www.nature.com/articles/s41598-020-79310-1

341 353 5 years ago

I find this fascinating, with no intention to start a political battle, how might somebody’s biology influence their political leanings?

toomuchtodo3• 5 years ago

https://www.scientificamerican.com/article/the-genes-of-left...

https://www.sciencedirect.com/science/article/pii/S235215462...

https://onlinelibrary.wiley.com/doi/abs/10.1111/pops.12230

imwillofficial12• 5 years ago

Thank you!

zapdrive• 5 years ago

Skin color. Features resulting from inbreeding. Features due to your gender. Features resulting from gender change.

dogsgobork• 5 years ago

Inbreeding?

imwillofficial12• 5 years ago

I don’t understand what you mean here, can you elaborate?

minitoar• 5 years ago

It’s pretty well established that age, gender, and skin color are correlated with one’s voting habits. Here are some stats: https://www.pewresearch.org/fact-tank/2020/10/26/what-the-20...

rory2• 5 years ago

From the article:

Accuracy remained high (69%) even when controlling for age, gender, and ethnicity.

jpxw2• 5 years ago

https://en.m.wikipedia.org/wiki/Biology_and_political_orient...

endisneigh• 5 years ago

What's a useful application of this, other than more advertising?

toomuchtodo3• 5 years ago

Dating app that constrains to like minded potential partners, instead of relying on political signaling in profiles.

jagged-chisel• 5 years ago

More echo chamber...

cwkoss4• 5 years ago

We know incest increases the likelihood of genetic disease. I wonder if a dating app could make a "faces too similar" filter to reduce chances of genetic disease in offspring based on just faces.

worldsayshi• 5 years ago

Finding potential dissidents?

goatinaboat2• 5 years ago

Like https://www.nytimes.com/2020/12/16/technology/alibaba-china-...

mmaunder• 5 years ago

Understanding why it is predictable and using that to either exclude ossified groups from targeted campaigns to save resources or focusing on them more tightly if they’re swing.

dilawar• 5 years ago

May be the world polarising into two kind of extreme polarisization: one with a sense of humor and one without it. And it has a noticeable effect on the face. I'd love to see few samples of faces from each class.

Pfhreak3• 5 years ago

I'm incredibly curious which political group has a sense of humor in your mind. Because I'm going to guess most folks would say, "my political group".

imwillofficial12• 5 years ago

Can you read my mind?!? ::Looks at you with suspicion::

In all seriousness, I haven’t followed closely, so I don’t know who hates Dr. Seuss lately, they are the baddies.

umvi2• 5 years ago

Both the far left and far right have no sense of humor because both have lost all sense of reasonableness.

On the right you have Trump cultists who loyally cling to every word he utters, regardless of it is true or not, spreading lies and misinformation on social media about voter fraud, vaccines, etc. You also have covid denial on the right where "the constitution" is used as a bludgeon to resist any reasonable public health policy like mask usage.

On the left you have the wokeness mobs that go around trying to cancel/censor/destroy every historical figure/book/statue who didn't live a perfect life and destroying careers of anyone who says the wrong word or makes a bad analogy. You also have covid zealotry on the left where "covid deaths prevented" is prioritized above all else and we can't re-open schools, re-open businesses, visit grandparents, or otherwise get back to normal life (even after mass vaccination) until 100% of all remaining unknowns are known (even if it takes another 2 years of masking, zooming, and hermiting).

These are all very recent examples.

A12-B• 5 years ago

The far left is pretty good at making fun of the right for saying silly things like this, actually.

iujjkfjdkkdkf• 5 years ago

This made me think a bit - I imagine most of the polarization these days comes from a vocal minority of either side, and that 95%+ of people are much more relaxed about politics (and still have a sense of humour) than we imagine from the media.

So it would be interesting to see if hyperpartisans could be separated from moderates by their appearance (just to be clear, I dont believe in phrenology, I imagine its information leaking from other aspects of presentation and background).

With respect to humour, the modern lack of humor seems to focus on not offending people, which I admit I dont understand but I could guess is predicated on the feeling that laughter is somehow equated to divisive mocking, as opposed to fun. I bring this up only because it reminds me a lot of Umberto Eco's "The Name of The Rose" where the 13th century religious leaders were arguing that laughter was inappropriate, and Jesus never laughed, somehow rooted in the belief that finding humor in things admitted the possibility of laughing at aspects of religion and therefore not taking them seriously.

Anyway, your comment made me think, so thanks.

rsynnott5• 5 years ago

Or alternatively, two groups who each think the other has no sense of humour.

bichiliad• 5 years ago

Accuracy was similar across countries (the U.S., Canada, and the UK)

This sounds more like people who are from minority groups (or who look like they are, to a computer vision algorithm) are more likely to agree with left-leaning policies, which probably has more to do with the policies in those countries than it does with any sort of genetic features. I feel like this might not work as well in non-Western countries, for example.

ucha• 5 years ago

"Accuracy remained high (69%) even when controlling for age, gender, and ethnicity."

callesgg2• 5 years ago

So your race accounts for less than 3% despite 89% of all black people voting left, that seams strange.

I did not do the full math but just the ballpark numbers I entered in to my calculator says that it should account for about 11%.

* 89% according to numbers from nbc

Edit: So say I assume that every African American person I see votes left. 13% of the population is African American 89% of them actually vote left. 13% * 89% = 11% Then I simply guess on all non African American a 50/50 shot. I should then be right 50% + 11% = 61% of the time.

drivingmenuts2• 5 years ago

Their facial images (one per person) were obtained from their profiles on Facebook or a popular dating website.

Not sure I’m entirely comfortable with that.

jpxw2• 5 years ago

Perhaps conservatives and liberals upload different kinds of face images to the internet.

drivingmenuts2• 5 years ago

Not the issue.

The issue is potentially being included in a study unintentionally, that is impossible to anonymize, without prior knowledge or consent.

zbendefy• 5 years ago

Altough random guessing is 50% right?

kicat2• 5 years ago

I only skimmed the paper, so I'm not claiming to know much about it, but one thing to keep in mind here is that a fair coin has a 50% accuracy using the same terminology as the headline. I'm not saying 72% is not an interesting achievement, its just that "you can do about 50% better than random chance" describes my gut feeling about how much you could actually see in someones face.

stanrivers2• 5 years ago

This is an important point - 72% is interesting, but its a 22% added to the chance to guess correctly... still interesting though

rory2• 5 years ago

It says in the article that humans got just 55% (so 10% better than random chance) on the same test.

bryanlarsen• 5 years ago

The humans are probably overthinking it. You get ~55% by assuming by answering "Biden" for everybody.

sct202• 5 years ago

I wonder if that 55% is from mturk or other survey sites that can be somewhat questionable in terms of quality with how much people are paying attention versus maximizing their hourly survey earnings.

itsdrewmiller3• 5 years ago

It says on a similar test - it's a reference to a different study with a different data set.

wycy2• 5 years ago

They do note the random chance bit, and they also note that it's better than humans could judge on their own and even, surprisingly, better than judged by a personality questionnaire.

Political orientation was correctly classified in 72% of liberal–conservative face pairs, remarkably better than chance (50%), human accuracy (55%), or one afforded by a 100-item personality questionnaire (66%).

klmadfejno5• 5 years ago

Were the humans experts or just random people though?

The real question is whether the tool can beat a lookup table of age, race, and gender probabilities. The tool isn't going to be winning points of phrenology here. Weight, hair color, and hairstyle would also likely tell you a lot.

I don't have any particular reason to believe this tool wouldn't work, but let's not pretend it's getting their by phrenology-esque topologies of people's faces.

A randomly chosen black individual in the united states has a > 72% chance of leaning democrat. A randomly chosen hispanic individual is ~55-65% chance of leaning democrat. I don't find it crazy to imagine they've got a few other smaller features to boost it.

simias• 5 years ago

72% is a very significant deviation from 50% though, I wasn't expecting such a result.

The highest predictive power was afforded by head orientation (58%), followed by emotional expression (57%). Liberals tended to face the camera more directly, were more likely to express surprise, and less likely to express disgust.

Emotional expression makes some sense in hindsight but I wouldn't have though that head orientation would correlate. It's interesting to know how we betray ourselves with these minute details of body language.

rsynnott5• 5 years ago

Though, these seems a bit odd; how many people are expressing _disgust_ in dating profiles?

statstutor• 5 years ago

A 72% score on this scale is equivalent to confidently knowing 44% of the answers, and coin-flipping the rest.

It's doing something that untrained humans are not capable of [edited to add: although humans were apparently tested by a different method, so this is not properly comparable], but is still a failing grade by usual methods of assessing human knowledge.

stanrivers2• 5 years ago

So... now, is it because people that came from the same regions a long time ago all happened to tend towards a certain political leaning and that just has stayed within families for that long?! Like a culture thing, but within families?

In general, though, its amazing how little control we have over who we are despite the feeling that we are in control of it.

roamerz• 5 years ago

Could be but not across the board true. My parents are deeply liberal to the point of the Constitution be damned so long as their views are enacted in some way. I am just the opposite - a constitution conservative. I am aware of several friends families that are the same or opposite.

I think the key to you last comment is how much intellectual control someone has over their emotional response. I like to think I am pretty good with that at this present time. That was not always the case and may not always be the case.

MattGaiser2• 5 years ago

I want a facial recognition tool that tells me what information I am leaking. We need a haveibeenpwned for facial recognition.

yikesshescute4• 5 years ago

Why play defense when you can play offense?

marstall3• 5 years ago

how about this tool in realtime, a kind of biofeedback mechanism that trains you to look conservative, liberal, sexy, dangerous, harmless etc.?

cwkoss4• 5 years ago

That could definitely be productized, even if it didn't work very well.

anigbrowl10• 5 years ago

We have that, it's called acting class.

zwieback• 5 years ago

The article discusses the underlying correlations at work but I think the title is a bit sensationalistic. Would expect more from Nature.

k0htlane• 5 years ago

Note that it's not published in the journal Nature, but in another (lower-impact) journal of the same publisher (Scientific Reports).

UnpossibleJim• 5 years ago

So, unless I read the abstract wrong, this is identifying age, race, and gender, then making categorical qualifiers based on those distinctions. While using ML to do the facial recognition to distinguish a person's age, race and gender is neat - categorizing their political affiliation with a 72% accuracy rate is fairly nominal, given the tools used by modern parties to garner donations and directed online advertising - no offense.

tyingq2• 5 years ago

"VGGFace224 was used to convert facial images into face descriptors, or 2,048-value-long vectors subsuming their core features."

So they had more than age, race, and gender, but it doesn't really say how things were weighted.

harias• 5 years ago

So they had more than age, race, and gender,

Doesn't have to be that way. It could be age, age+1, age+2 ....

ad404b8a372f2b93• 5 years ago

I don't really buy this study. If humans can't get a better accuracy than 55%, I'm convinced this vector is leaking some obvious (maybe high-frequency) information that doesn't have anything to do with the face. E.g location of the person.

theonlyklas• 5 years ago

I believe that these descriptors are created only based off the visual image:

A face descriptor is obtained from the learned networks as follows: the centre 224 × 224 crop of the face image is used. The shorter side is resized to 256, and the CNNs descriptor is computed for this region by extracting the deep features from the layer adjacent to the classifier layer. This leads to a 2048 dimensional descriptor, which is then L2 normalised.

https://www.arxiv-vanity.com/papers/1710.08092/

hirundo• 5 years ago

They tested this question specifically:

Both in real life and in our sample, the classification of political orientation is to some extent enabled by demographic traits clearly displayed on participants’ faces. For example ... white people, older people, and males are more likely to be conservatives. What would an algorithm’s accuracy be when distinguishing between faces of people of the same age, gender, and ethnicity? To answer this question, classification accuracies were recomputed using only face pairs of the same age, gender, and ethnicity ... The accuracy dropped by only 3.5% on average

11thEarlOfMar• 5 years ago

From there, it would seem that cues might come from how 'kempt' they appear, whether the head shot was from a party or for a resume, perhaps color and style of clothing, ... I.e., maybe not strictly the face.

kevinventullo2• 5 years ago

If the total population sampling is the same, I would expect the accuracies to remain the same. E.g. if I can get 72% accuracy in the total population just by looking at age/race/gender, doesn’t that exactly mean the accuracies in each individual category are on average 72%?

rsynnott5• 5 years ago

I suspect there are a lot of less obvious things they'd need to control for. Off the top of my head, weight would be an obvious one; in developed countries urban areas (particularly large urban areas) generally have a lower average BMI than rural and suburban areas, and there's also typically a major political difference between rural and urban areas.

karaterobot• 5 years ago

Does anyone know what the accuracy of a prediction is if you use only those three factors -- age, race, and gender?

gamegoblin• 5 years ago

All numbers are Biden-Trump in 2020:

People under 30: 60-36.

White men: 38-61

Black women: 90-9

So there are definitely some strong predictors there.

Source: https://www.businessinsider.com/2016-2020-electoral-maps-exi...

boomboomsubban• 5 years ago

You can basically determine this by looking at voter/exit poll results, and any single criteria would give ~55% at best.

Defenestresque• 5 years ago

This is the exact question I came to the comments to find.

The abstract states:

Accuracy remained high (69%) even when controlling for age, gender, and ethnicity.

To give some context, chance is 50%, human guess is 55% and a 100-question questionnaire is 66%.

Personally, I am surprised that the accuracy remained that high when controlling for the three variables I would have considered most telling in the determination (age, gender and race).

I'd be very curious to know what exactly the algorithm is determining from the face photos outside of those obvious variables. I know with a ML algorithm it's practically impossible to determine why the classification was made, but does anyone human here have any thoughts?

ebegnwgnen2• 5 years ago

"The dating website sample was provided by a popular dating website in 2017. It contains profile images uploaded by 977,777 users; their location (country); and self-reported political orientation, gender, and age."

It doesn't sound like people were consenting/aware that they'll endup on a facial recognition study.

But I'm not even surprised dating website would sell their such data

MattGaiser2• 5 years ago

There are plenty of scraped dating website datasets lying around.

ebegnwgnen2• 5 years ago

Yes, but I suppose you can't use them in a publication on Nature if they are scrapped illegally.

This dataset directly from the dating website

jklein11• 5 years ago

It's possible that the dating website got 1 million users and couldn't find product market fit. They then pivoted to selling their users data.

invalidusernam3• 5 years ago

It's likely that users did consent when they checked the "I've read the terms and conditions" checkbox when signing up.

karpierz• 5 years ago

This isn't a study that shows that people's faces indicate their political leanings.

It's a study which shows that pictures that people select to represent themselves publicly have features that indicate political leaning.

kicat2• 5 years ago

Agree. Even the few pixels bordering the face in the sample image can show she's outside. She chose a smiling picture, she's wearing makeup, etc...

jonbronson• 5 years ago

That's a really good observation to note. The prior embedded in their image data is their own bias of what is a "good" representation of themselves.

bsenftner• 5 years ago

Yep, I'd look very closely for training data bias.

derekam• 5 years ago

Exactly. There was some fuss a while back about a similar classifier for sexuality. It turned out to be guessing mostly based on head tilt, personal hygiene and whether the person was wearing glasses. The physiognomy component was ~nonexistent even though it was publicized as though it weren’t. People intentionally if at times subconsciously present themselves in a way that signals information to kindred spirits. You’d need to bring in hundreds of people, wash them and basically take mugshots to control for that.

jfengel2• 5 years ago

I think that conclusion is at least as interesting as physiognomy. It's remarkable that a computer could be more sensitive to it than people.

jbob2000• 5 years ago

Profile photo with a person wearing a baseball cap and Oakleys? Yup, that’s a republican.

hntrader4• 5 years ago

If it's that easy then why was human guessing only 55 percent accurate?

fc3737452• 5 years ago

images were tightly cropped around the face and resized to 224 × 224 pixels

Karawebnetwork• 5 years ago

One can only wonder what the result would be by using the equivalent of government ID photos (neutral expression, no smile or make-up, solid backgrounds).

nathias• 5 years ago

The problem wit this is bad interpretation, people look at it and think oh this means political orientation is somehow inherent to biology or some such nonsense. Any data that can make predictions is just a sign that in this data there are some patterns, but there is never a straightforward interpretation for it.

Also, whoever makes the first app for this will go viral 100%.

umvi2• 5 years ago

Surprising, yet not surprising. As the article mentions:

Both in real life and in our sample, the classification of political orientation is to some extent enabled by demographic traits clearly displayed on participants’ faces. For example, as evidenced in literature and Table 1, in the U.S., white people, older people, and males are more likely to be conservatives.

Most people can predict a person's political orientation of their own country with >50% accuracy as well just by looking at a face. Black or latino? probably liberal. Old white person? probably conservative. If you can see more than just their face it's even easier (wearing religious paraphernalia? LGBT paraphernalia? etc?)

What I thought was interesting was:

The algorithm could successfully predict political orientation across countries

I was under the impression that "liberal" and "conservative" had different meanings in UK vs. USA so how could it do this?

rsynnott5• 5 years ago

I was under the impression that "liberal" and "conservative" had different meanings in UK vs. USA so how could it do this?

I assume they're using the US definition (meaning "left wing" and "right wing", more or less).

greeneggs• 5 years ago

I am curious how much additional information, if any, is learned from a person's face. From a full-body photo, with the face blacked out, what can be learned?

Agingcoder• 5 years ago

I haven't read the paper yet, but it's not that surprising. I think it's about sociology more than biology.

In the US, African Americans vote overwhelmingly for democrats for example. Skin color therefore becomes a very good predictor of political orientation. You can probably extend this to states being populated from various migration waves, say 'people who look like Danes vote for republicans because state X was populated by Danes and votes Republican' . Carry this across generations/education, and you may have an explanation.

asah• 5 years ago

"only"

hughes• 5 years ago

Their facial images were obtained from their profiles on Facebook or a popular dating website

So, this seems to be doing profile picture recognition, not facial recognition. It's not like they're saying there are physical features in your face that give away your political affiliation - that is to say, putting two people's bodies under the same photographic conditions would probably not create this kind of signal.

What this is probably training on is the cues for cultural values that we self-select in our most deliberately promoted images of ourselves. If you have a carefully-chosen profile picture, it probably includes signals of what's important to you, especially if you're using it to attract people with similar values.

function_seven2• 5 years ago

This meme[0] comes to mind. Is this AI picking up on the same types of cues represented below?

[0] https://imgur.com/FK0RzKM

bluecatswim• 5 years ago

Or this.

https://i.kym-cdn.com/photos/images/newsfeed/001/330/806/8fc...

weberer2• 5 years ago

The paper says predicting based on sunglasses use is only 52% accurate, so probably not.

fc3737452• 5 years ago

images were tightly cropped around the face and resized to 224 × 224 pixels

marstall3• 5 years ago

still leaves the micro- and macro-expressions, small grooming cues (makeup, no makeup, eyebrows trimmed or not), hairline, head angle vs. camera, lighting etc. These are all things that humans very specifically deploy to define themselves and their grouping, and communicate with others. So I am guessing a whole universe of personal yes-no qualities, political and otherwise, are encoded there, quite intentionally.

aboringusername• 5 years ago

I assume there's no way to actually verify how someone may choose to vote, assuming there's no record of that?

I think there's huge value now that everything is being sent into a "machine" or "the algorithm" in fucking with it.

Order sex toys from Amazon, show them you're into outrageous books and fool them into creating a fake profile of "you", based on your spending, browsing and other data you generate.

I'd love to ask a machine what it knows about me, how accurate it is, and then switch it all up. I'm too old to vote now (and will probably be dead soon) but I'd love to pick a position completely unexpected just to throw it off.

Poison the well

jfk13• 5 years ago

I'm too old to vote now

There are places with a maximum voting age?

rsynnott5• 5 years ago

Presumably they're a cardinal. Cardinals aged over 80 aren't allowed vote for pope.

yikesshescute4• 5 years ago

The machine will know which sort of person tries to fool it in the way that you are trying to fool it.

larkost• 5 years ago

From Amazon's perspective you have not poisoned the well... you are someone who is likely to buy those things. Amazon does not care if you really like or even use them, it only cares if you buy them.

cirenehc4• 5 years ago

"Accuracy remained high (69%) even when controlling for age, gender, and ethnicity."

So if I just assign the majority label to all of the population of a given demographics group, I would get the same result right? i.e., predicting "left" for all minorities under 30. You would also get ~70% accuracy.

anigbrowl10• 5 years ago

What do you think 'controlling for age [,etc]' means?

cirenehc4• 5 years ago

Please state what you want to say. No need to be passive aggressive.

the_arun• 5 years ago

It is like saying - My intuition usually works great. 78% of the times I have been successful with my assumptions/speculations.

touringmachine• 5 years ago

could we please not

ampdepolymerase• 5 years ago

Who wants to take bets on how long before the paper gets cancelled and forced retraction?

mc32• 5 years ago

I can see making inferences from dress, makeup, style, etc. But, this is surprising; “facial morphology”

This skims somewhat close to people who tried to infer criminality from facial and cranial characteristics... a century ago

goatinaboat2• 5 years ago

But, this is surprising; “facial morphology”

You could probably get this from how fat their face was, or how unhealthy their complexion, e.g. https://www.vice.com/en/article/j5e3z7/gym-bros-more-likely-...

ip26• 5 years ago

I don’t think it’s crazy, what if rural areas (which lean conservative) have biased ethnicity? For example, if the Irish, Italians & Asians mostly immigrated to urban areas while Germans and French mostly immigrated to rural areas, then if you can guess the ethnicity from facial structure you can make a guess on political leanings.

souprock2• 5 years ago

A century ago, phrenology failed.

Would it fail if you added a modern AI? How about if you had an MRI that provided more-direct information for the AI model? Feeding actual brain structure data into an AI is far more sophisticated than measuring heads and feeling for lumps.

astrea• 5 years ago

I question the dataset used for this and the basis. They used "self-reported political orientation, age, and gender" and "facial images (one per person) were obtained from their profiles on Facebook or a popular dating website". I first question the ethics of what sounds like a Facebook scrape. Second, I wonder how well they normalized across the terrible filters and frames and variations in pose. Finally, I question the basis in regards to the ability to gauge one's "openness to experience" let alone, say, "opinion on immigration policy" from micro-expressions in the face (or at least those which could be interpreted by a VGG based facial recognition algorithm).

Edit: One last rant about this palm-reading-esque pseudoscience: I hate that this was put out in the universe, and thus potentially giving the wrong person ideas.

Kluny2• 5 years ago

It's an easy trick when there's only two options: red and blue.

- The rest of the word.

yikesshescute4• 5 years ago

Then going long or short on a security is easy?

Kluny2• 5 years ago

Sure, you could probably train an AI to get 72% accuracy on that as well. It would be just as meaningless.

motohagiography• 5 years ago

These self-selected, naturalistic images combine many potential cues to political orientation, ranging from facial expression and self-presentation to facial morphology.

I was going to ask how they controlled for the presence of compound bows & deer, trucks, and tank tops in the photos, but this implies they did not. Another one would be lighting palette, since that's going to be biased to regions as well. There is something to be said for it, as I'd say I have a %72 chance of guessing someones political orientation by looking at them as well, which someone once explained to me as being the effect of testosterone levels on the region around their eyes, but that sounded like folksy bro science.

ralusek• 5 years ago

I was going to ask the same about colored hair, facial asymmetry, hammer and sickles on shirts, and scowls.

bigwavedave• 5 years ago

I was going to ask how they controlled for the presence of compound bows & deer, trucks, and tank tops in the photos, but this implies they did not. Another one would be lighting palette

No need to read into implications. From TFA:

"The procedure is presented in Fig. 1: To minimize the role of the background and non-facial features, images were tightly cropped around the face and resized to 224 × 224 pixels."

While tightly cropping the face isn't perfect, it does address three of the four things you were specifically wondering about.

jlev• 5 years ago

When are people going to stop rebuilding high tech phrenology?

Bakary• 5 years ago

When it will cease to be profitable or seemingly so

human2• 5 years ago

My political orientiations have changed a few times over the last 10 years but my face hasn’t. This can’t actually work...

_cs2017_• 5 years ago

First, your face has changed over the past 10 years because of the aging, the way you take care of your face, your health, your mental and emotional state, your sleep, your nutrition, and many other factors.

Second, the model in the paper did not only look at the face, but at the entire photograph. How you choose to present yourself on a dating site has also changed over the 10 years: the quality of the photo, what haircut you choose, how closely you shave, whether you're tanned, your facial expression, whether you wear glasses, what objects, landscapes, and colors can be seen in the background, etc.

Finally, and most importantly: Even if your dating site profile picture hasn't changed in lockstep with your political leanings, that's fine -- they can make some errors and still get the 72% accuracy they report.

relaxing2• 5 years ago

72% of the time, it works every time...

Kranar• 5 years ago

You think your face hasn't changed over the last 10 years?

You think there isn't a relationship between age and politics?

human2• 5 years ago

You’re right that I’ve gained a few pounds and look older. Obviously I was exagerating to make a point. But I guess it’s all about the 72% which means it far from perfect, but still better than a coin toss.

mabbo2• 5 years ago

I think the key line is how much better this system is than humans attempting the same task:

Political orientation was correctly classified in 72% of liberal–conservative face pairs, remarkably better than chance (50%), human accuracy (55%), or one afforded by a 100-item personality questionnaire (66%).

This isn't a matter of "recognize that old white people are conservative", because people will do that already, and they know all those biases. And the system doesn't lose much accuracy when comparing otherwise similar people.

This system is picking up on things we don't notice. Maybe it's the photos themselves (they are self-selected), maybe it's micro expressions in the face, maybe it's something else entirely.

But damn it's neat and maybe frightening. Imagine if your next hiring manager had a quiet little camera in the corner and chose you based on your predicted approval of unions.

sillysaurusx4• 5 years ago

But damn it's neat and maybe frightening. Imagine if your next hiring manager had a quiet little camera in the corner and chose you based on your predicted approval of unions.

This is the dystopian vision of the future I come to HN for. Bravo!

(It's also why I do ML, since I'll be on the front lines to notice if something like that is being deployed. Or at least somewhat more likely.)

mabbo2• 5 years ago

This is the dystopian vision of the future I come to HN for

It's what I'm here for. Fascinating technology- how might I ruin the world with it?

coldelectrons2• 5 years ago

I'm sure that's exactly what the tech giants will do in their feudal-governed cities they are going to carve out of Nevada.

cambalache2• 5 years ago

This system is picking up on things we don't notice.

In the case of men at least some studies have found a correlation between testosterone level and political orientation, something that matches well with my anecdotal observations. It is not outlandish to think this manifests in visual cues in even something as limited as a pic.

https://www.mdcthereporter.com/low-testosterone-left/

drno123• 5 years ago

I have no idea why this comment got downvoted. This is the most scientific explanation possible. In males, more testosterone induces more risk taking, while low testosterone is linked to risk avoidance. Left-wing policies (“nanny state”) cater to people who avoid risks; people who are risk takers prefer right wing policies. Testosterone also affects muscle growth and facial characteristics.

whimsicalism• 5 years ago

I would like to see more comparisons against the best humans at a task

boublepop• 5 years ago

This isn't a matter of "recognize that old white people are conservative", because people will do that already

I think your giving too much credit to humans, and ignoring the fact that most people would lose a lot of point due to a bias towards expectation that people who are attractive or look well off or happy would share their own political beliefs.

cmehdy2• 5 years ago

What makes you think your hiring manager isn't already doing that based on their interaction with you and your looks? Even simply subconsciously?

dj_mc_merlin• 5 years ago

The point is this system would perform better than them. Maybe we will need to learn how to lie to robots?

yarcob• 5 years ago

As noted by gus_massa in another thread, the 55% figure for human accuracy is from a different study with a different datasets, so it's questionable how comparable the results are.

csomar• 5 years ago

This should be impressive considering that humans are very good at picking other human faces.

SolarNet• 5 years ago

I think the key line is how much better this system is than humans attempting the same task:

My key issue is: why does that matter, humans should not be doing it in the first place, why do we need a machine that's even better at it?

anigbrowl10• 5 years ago

Presumably some humans are terrible at this and others are as good or better than the machine. Of course, even someone who was very good (say 80% accurate) would be wrong about 1 person in 5.

Being honest about this equips a person with a useful hunch capability that will provide a long-term edge (perhaps as a detective, or in sales, or any of many other contexts where people-reading can help); less honest practitioners might become con artists, or consultants/coaches whose methods are not reproducible or even formalized such as Dave Grossman: https://www.insider.com/bulletproof-dave-grossman-police-tra...

paulsutter• 5 years ago

the relative universality of the conservative–liberal spectrum

Grey tribe members may beg to differ

https://www.gsb.stanford.edu/insights/rise-liberaltarian

input_sh• 5 years ago

Anyone not living in a two-party system begs to differ, especially those living in countries where liberals are conservatives (in a sense the term conservative is used within the US).

swebs• 5 years ago

Not really. Other country's liberals are the USA's Libertarians. They're much different than neocons.

spamizbad• 5 years ago

If you can guesstimate someones age bracket, gender, and whether or not they're white you can also probably hit a similar accuracy.

Out_of_Characte2• 5 years ago

I just wish people called it 'correlates' instead of prediction since ML algo's often fall in the correlation category, not the predicting one.

adolph2• 5 years ago

Agreed. Without the addition of causal inference this is phrenology in ML drag. To their credit the authors say "Here, we explore correlations between political orientation and a range of interpretable facial features. . ."

https://en.wikipedia.org/wiki/Phrenology

https://www.goodreads.com/book/show/36204378-the-book-of-why

yikesshescute4• 5 years ago

Is there some technical definition of “predict” that you are assuming?

Out_of_Characte2• 5 years ago

Predict can give the false impression that a causal relationship has been defined when you might only have correlations.

gruez• 5 years ago

Can we run the model in reverse (eg. deepdream) to see what the stereotypical liberal/conservative looks like?

jm_l• 5 years ago

The main classifier they use is using logistic regression, so no. They do mention that a deep nn had similar performance, but even if you deep-dreamed it you wouldn't get images, you would get feature sets, since they first run the photos through a feature extraction model (i.e. the input to the nn is not raw pixels).

kthejoker2• 5 years ago

Oh man, a whole new generation will get thisliberaldoesnotexist.com, thisconservativesdoesnotexist.com, thisfascistdoesnotexist.com ..

On a serious note, sounds like the dating and hiring sites are going to need to hire an all new layer of ethicists and legal teams to defend against algorithmic discrimination.

utborin• 5 years ago

There's an avalanche of people commenting on this who didn't bother to check the article before raising their methodological objections, so let's get these out of the way here.

- Yes, they controlled for objects appearing in the pictures that might indicate political affiliation. The images are tightly cropped around the face. See Methods.

- Yes, this is significantly better than both a coin flip and a human classifier. They gave the same test to humans, who did much worse than the model. See Abstract, Introduction, and Results.

- Yes, this is doing more than just detecting a person's race, age, and/or gender. The classifier is still accurate when they compare people with the same race, age, and gender. See Results.

If you want to discuss actual limitations in the study, here are some the author points out:

- "A more detailed picture could be obtained by exploring the links between political orientation and facial features extracted from images taken in a standardized setting while controlling for facial hair, grooming, facial expression, and head orientation."

- "Another factor affecting classification accuracy is the quality of the political orientation estimates. While the dichotomous representation used here (i.e., conservative vs. liberal) is widely used in the literature, it offers only a crude estimate of the complex interpersonal differences in ideology. Moreover, self-reported political labels suffer from the reference group effect: respondents’ tendency to assess their traits in the context of the salient comparison group."

vipa123• 5 years ago

You sir/madame, are a saint!

gzer02• 5 years ago

Good points and summary. Adding on to that, the following was from the abstract about 2 sentences in. Seems like some users do enjoy jumping straight to the comments!

* Political orientation was correctly classified in 72% of liberal–conservative face pairs, remarkably better than chance (50%), human accuracy (55%), or one afforded by a 100-item personality questionnaire (66%).

* Accuracy was similar across countries (the U.S., Canada, and the UK), environments (Facebook and dating websites), and when comparing faces across samples.

* Accuracy remained high (69%) even when controlling for age, gender, and ethnicity.

hntrader4• 5 years ago

"controlling for ... ethnicity"

There's significant signal buried in here that is likely not controlled for, depending on how granular their controls are.

White-German and White-Italian are much more (10-13 percent) likely to be conservative leaning than White-Irish or White-British.

Hispanic-Cuban are more likely to be conservative leaning than Hispanic-Mexican.

fcantournet• 5 years ago

So even filtering for the extremely limited number of things that you allow us to find problematic, by their own admission : - they know that it's actually probably "working" (lol 72%) because of other biases they didn't take into account - Ultimately it puts people into 2 bins which are both huge and disparate and reduce complexe multi-dimensional elements to a meaningless binary.

cool cool cool.

SamBam3• 5 years ago

I don't really understand this comment. You don't think it's surprising or interesting that it could predict with 70% accuracy the political orientation of two different older white males? Or two younger non-white females?

I found this to be a surprising result.

What the mechanism? Is it the haircut? The facial expression? The quality of the photo? Maybe. Do those count as "biases that they don't take into account?" Could be. Would it be interesting to learn what those "biases" are? Yes!

gzer02• 5 years ago

This is statistically significant. The implications are massive. The authors are signalling it might be worthwhile to take a look at the exponential growth of these technologies.

72% is much better than:

* random chance (50%)

* human accuracy (55%)

* 100-item personality questionnaire (66%)

pretendscholar• 5 years ago

Scoffing at 22% over random? Lets play poker some time.

andybak• 5 years ago

Surely the ability to sort people into meaningless bins on facial features alone is worthy of note?

Let's say you created an arbitrary classification ("number of letters in street address" or "cosine of age in minutes") and an alorithm could predict that based on a photo alone. That would surely indicate that the classification wasn't arbitrary and there was something more interesting happening?

At the very least you'd want to dig deeper.

abecedarius• 5 years ago

lol 72%

It takes 33 bits to single out a human. 72% means a substantial fraction of a bit -- it's not much, but combine it with a few dozen other clues of similar magnitude and you're really getting somewhere.

5423542342353• 5 years ago

Ultimately it puts people into 2 bins which are both huge and disparate and reduce complexe multi-dimensional elements to a meaningless binary

Binary, yes. Meaningless, no. Having multi-dimensional information is better than binary information, but having binary information is better than no information (or guessing). Telling me the temperature, humidity, wind, cloud cover, and precipitation outside gives me more information than just telling me it is hot or cold outside. But knowing if it is hot or cold outside is still far better than not knowing anything. You can still make decisions and take actions based on limited information, which you could not do with no information.

mywittyname2• 5 years ago

There's evidence of inheritability of political orientation, since it is tied to personality traits. Given that, it makes sense that it is possible to predict political orientation through looks alone. After all, looks are also inherited traits.

The article gets into this a little bit. They are able to predict traits like intelligence and honesty by looks alone.

MikeHolman• 5 years ago

I think this is a very dangerous line of study. I can easily see these results getting used to forward racism and other appearance based discrimination.

jtbayly• 5 years ago

Personality is only weakly tied to political view according to the above comment... so... ?

tcmb• 5 years ago

That reasoning sounds like a logical fallacy:

1. Personality traits are inherited 2. Personality traits influence political views 3. Looks are inherited -> Therefore, looks influence political views (?)

"A is X and does Y" "B is also X, so B does Y as well."

retrac• 5 years ago

I suspect we're going to see the same thing we saw in the model that can detect sexual orientation with much better than chance odds. It was, apparently, detecting that gay men tend to take or select images of themselves for a dating profile using a different angle, better lighting, and possibly differences in things like hairstyle and beards.

Very anecdotally, I've noticed a possible weak correlation between certain kinds of beard styles and political leanings in men.

SamBam3• 5 years ago

Obviously it's looking at something. I feel like everyone is jumping to assuming the authors are implying it's somehow in the person's facial structure, but it doesn't say that.

If it's looking at the quality of the photo, or the trim of the beard, that's still interesting. Among other things, it means that you know analyses like this might start cropping up everywhere you submit your photo (job application cover letter?), and also that humans might be doing this innately with photos and not even realizing it.

thechao• 5 years ago

This is all discussed in the paper. The answer is that liberals are more likely to show surprise, less likely to show disgust and faced the camera directly. Much less suggestive was the wearing of beards and glasses where were very nearly in the noise.

lwansbrough• 5 years ago

Perhaps it’s checking if the person took a selfie in a car with sunglasses on.

rednerrus• 5 years ago

People are desperately trying to tell you who they are with everything that they do.

TomSwirly• 5 years ago

Please consider that "conservatism" in 2021 isn't just a political leaning - it requires one to believe a very great number of hateful beliefs that are provably false, and therefore to be constantly disgusted and angry.

Is it so unreasonable that these feelings of disgust and anger are simply readable by machines on people's faces?

nullserver• 5 years ago

Got into Art a few years ago. Took classes and everything.

People like when I take photos now because my come out better. I simply got a lot better at understanding lighting and angles.

gus_massa4• 5 years ago

- Yes, this is significantly better than both a coin flip and a human classifier. They gave the same test to humans, who did much worse than the model. See Abstract, Introduction, and Results.

Where do they say that they gave the same test for to humans? All I find is the reference [15] that points to https://www.researchgate.net/publication/232255935_Accuracy_... that cite a previous article about a different set of photos and a different question.

robocat• 5 years ago

And some humans may be very good classifiers, or would be if trained.

dmurray3• 5 years ago

They very much didn't. Downvoted the GP for taking the snarky tone about "an avalanche of people commenting on this who didn't bother to check the article before raising their methodological objections", but then mis-characterizing the methodology himself.

dheera• 5 years ago

I would be very interested if instead if they first passed it through an indepedently-trained emotion classifier, and then look to see if the emotion vectors alone can predict liberal vs. conservative to some degree.

bhk3• 5 years ago

The 72% number is the result when not controlling for demographics. When controlling for demographics, the results ranged from 65% to 71% accuracy.

https://www.nature.com/articles/s41598-020-79310-1/figures/2

It makes we wonder what the accuracy would be if they controlled for demographics at a smaller granularity, like sub-ethnicities.

Furthermore, it appears that the Canadian dating site data set was 54% conservative, so an algorithm that always guessed conservative would be correct 54% of the time. From the article I can't tell what the balance was for the other data sets.

itsdrewmiller3• 5 years ago

"The accuracy is expressed as AUC, or a fraction of correct guesses when distinguishing between all possible pairs of faces—one conservative and one liberal." - so no way to guess better than 50%.

bhk3• 5 years ago

Also: "Overall, the average out-of-sample accuracy was 68%..." (again, this does not control for demographics).

I would not call it "prediction" when the "predictor" was trained on the data set you're testing it against. The out-of-sample number would be more fairly called prediction. It's unclear what the number would be for out-of-sample accuracy corrected for demographics, but we could extrapolate a guess of 64%.

anigbrowl10• 5 years ago

avalanche of people commenting on this who didn't bother to check the article before raising their methodological objections

This is a norm nowadays, and a bad one. In some cases it's carelessness or laziness (why go to the effort of reading the article when posting an uninformed contradiction will provide free explanations), in others trolling or deliberate propagation of misinformation.

gvd• 5 years ago

IS it a meat head / not meat head detector?

PicassoCTs• 5 years ago

Seems like a lot of medical/living history can be derived from the face. Rings around the eyes tell of 2nd jobs and long hours. Sunburned skin tells about work outside exposed to UV. Bad shaves tell about a lack of time and money, lack of makeup, tells a story about a lack of time for luxurys. Substance abuse skin tells of hard times.

All in all, this sounds like a nice tool to detect anglo-sphere wide signs of a middle class that lost out to globalization. Which in itself makes it valuable to judge credit-history by look.

timoth3y• 5 years ago

What would an algorithm’s accuracy be when distinguishing between faces of people of the same age, gender, and ethnicity? To answer this question, classification accuracies were recomputed using only face pairs of the same age, gender, and ethnicity

Perhaps I am reading this wrong, but this is not controlling for demographics. If, for example, 70% of older white men are conservative, you could reach 70% accuracy by predicting all older white men are conservative.

Yes, it's much better than a coin flip, but random 50/50 guessing would be a ver bad strategy.

It's really a shame that they did not have humans and their system classify the same out-of-sample data so we could compare apples to apples. Perhaps they will in later research.

dwd• 5 years ago

I'm surprised they didn't classify and weight for facial features such as general symmetry or eye distance.

lwhi• 5 years ago

I think people automatically jump to the conclusion that studies like this are linking the development of innate physical characteristics with political allegiance.

I can understand why this line of enquiry is troubling; as it obviously ties in with branches of science in the 20th century that were immoral.

Logically, some physiology will affect our reasoning from a pretty deferred position. Our genetics predespose our brain chemistry. I'm led to believe the development of certain hormones has been shown to have an affect on our physical features; for instance, increased testosterone, providing a more prominent browline. It doesn't feel proposterous that such physiology might have _some_ bearing on the way we align our worldview.

It also doesn't surprise me that these factors might be able to be used to infer correlation when used with a very large dataset.

People are nuanced though, and I struggle with the concept that we are total slaves to bodies we're born with. I believe choice (through nurture) allows us a high degree of freedom to counterbalance the initial physiological stack our genetics encourages.

If this is true, how is this model's reasoning able to successfully predict political allegiance?

I'm imagining the way we present ourselves provides subtle nods to prominent figures we respect and cues towards our politics. We leak information through body language, dress, expression. Is this where the extra inference comes from?

system16• 5 years ago

The highest predictive power was afforded by head orientation (58%), followed by emotional expression (57%). Liberals tended to face the camera more directly, were more likely to express surprise, and less likely to express disgust. Facial hair and eyewear predicted political orientation with minimal accuracy (51–52%).

This is really interesting. I never considered head orientation or expression to be a factor. Then again, it sorta makes sense. Speaking very generally, liberal leaning people on social media probably tend to be more likely to post pictures of themselves in a humorous or "soy face" expression, and conservative-leaning types may try to look strong or aggressive.

Also, I automatically assumed features like "handlebar moustache" would mean more likely to be conservative, but it sounds like facial hair wasn't as big a factor.

reaperducer• 5 years ago

Generally speaking, these sorts of things don't make any sense to me. People change as they age, as they get new experiences, when they live in different places. How can someone's face tell you what their political orientation is?

When I lived in Texas, I was considered a liberal. When I moved to Chicago, I was considered a conservative. When I moved to Seattle, I was considered a conservative. When I moved to the desert southwest, I was considered a liberal. Nothing about my face changed. Just my address.

randcraw2• 5 years ago

Assuming this analysis has no problems with method, it's likely that the separation of any two groups on a continuous spectrum cannot ever reach 100% accuracy. In fact for such multivariate groupings as opposing political persuasion, 72% may in fact be as good a score as is possible.

Representing the conservative and liberal groups as gaussian mixes of multiple atributes, I would expect those two peaks to overlap. Perhaps the real surprise is that they overlap no more than 28%.

alcover3• 5 years ago

How would we collectively react if ML proves accurate in classifying anti-social traits or worse ? Mythomania, narcissic perversion, pedofilia, etc ?

Employers, condos, schools, many communities will want to ML-screen candidates, be it officially or not..

laurent92• 5 years ago

We already do it, at least subconsciously. We’ve also developed a side of society who actively tries to counter it, by actively seeking out-of-the-normative profiles: The skater look in companies is now a thing that helps you get sympathy, being female opens up sone avenues, Atlassian’s CEO wearing a mohawk or Jack Dorsey’s looks are all symbols of a society which started searching for non-normative people.

Perhaps we’ll require AI to do the same. Otherwise AI will be an excuse to be racist, saying “It’s the stats!”.

adolph2• 5 years ago

Ethical AI seems to be a work in progress.

https://www.cnn.com/2021/02/19/tech/google-ai-ethics-investi...

alcover3• 5 years ago

OK but attributes like the mohawk are 'playful' in this setting. It distinguishes its bearer by making him look aggressive while still being a good, collaborative member.

On the opposite, ML could indicate that in spite of one's hippy looks he really is a potential ruthless monster.

randcraw2• 5 years ago

I wonder if the study accurately reflects the lack of political common ground in the population. Are only 28% of us “moderates”?

It’d be interesting to do the same analysis with photos of people taken 10, 20, 30, 40,... 100 years ago to see if the intersection (28%) grows/shrinks historically. Were we always so easily politically separable by our appearance alone?

Has our appearance always revealed our political leanings, or do we now dress in order to make a political statement?

crazygringo3• 5 years ago

There's no definitive definition of "moderate".

People who want to argue most people are moderate say the political middle is the 75% middle of the bell curve, while people who want to argue most people are partisan say the political middle is only the middle 20% of the bell curve.

What is accurate to say is that political views in Congress have bifurcated (not a bell curve), and that people in America have "sorted" -- decades ago many conservatives were Democrats and many liberals were Republicans, but now Republicans are virtually all conservative and Democrats are virtually all liberal. And people's political identities have become more important to them.

But it also continues to be accurate that when people are surveyed according to their actual political positions on issues (as opposed to party affiliation), political views of citizens are still strongly bell-shaped -- they cluster in the middle. (Contrasted with congresspeople who cluster at two peaks.)

JumpCrisscross2• 5 years ago

of the bell curve

There is no bell curve. Political orientation is largely a self-referencing phenomenon. Outside the political elite, who have a high degree of ideological coherence, knowing a person's views on one issue is a loose predictor on their views on another. Gun-toting anti-abortion lesbians and pro-immigration free-market feminists are real, and they aren't some striking minority.

EricE• 5 years ago

Sigh - modern day phrenology, but wrapped in AI. Just what we need :p

jrd259• 5 years ago

So now I want to know 1. what is the eigenface for each of the two clusters? What does the perfect liberal look like? 2. What is the political orientation of https://thispersondoesnotexist.com/

inglor_cz3• 5 years ago

I wonder how much data about a person can you squeeze out of a high-definition video, analyzing not only face, but grimaces, tone of voice, rhythm of speech etc.

That should give you a lot more hints than a still image.

The worst possible uses: filtering out undesirable people applying for jobs or for college, firing people who are suspect of belonging to a different political tribe.

marstall3• 5 years ago

theory: seeing as these are pictures that were very specifically "taken" in a given context - with a camera, by someone, for something (dating, facebook, etc.), would it be possible that the algorithm is picking up differences more in how people relate to those conditions, vs facial morphology? For example in the example photo, i would have also guessed that the model was a liberal. But I think it was more about the something in the WAY she was smiling and posing for the camera, and i guess also her specific grooming - hair and makeup.

cwkoss4• 5 years ago

I wonder if accuracy for women is higher? I'd expect so, because women, via makeup, tend to have more cultural information encoded on their faces.

corpMaverick2• 5 years ago

Who knew. Being a progressive/moderate makes people more attractive...and improves your sense of humor.

bsaul• 5 years ago

Has there been any attempt to do the same kind of work for IQ prediction ?

throwitaway1235• 5 years ago

This study proves that the liberal surprise face meme is real.

firebaze• 5 years ago

Reality please don't bite me. This will probably be an Oxymoron :)

An interesting question, to me, is: how would you (personally) react if such an controversial and stigmatized factoid would be proven true at more than 5 sigma?

Would you be shocked? Or would you accept the outcome? And which of both would tell what of yourself?

NoOneNew• 5 years ago

Oh cool, are we going to bring back phrenology as long as its an AI powered black box. Sounds wonderful. It's going to be impossible for any of this to be abused. I say we should give AI power to a select few people who can make sweeping decisions in everyone's life. Oo Oo can we bring back eugenics too? What's wrong with an AI powered method of selecting the best genes to continue the human race. Then we can sterilize the degenerates the almighty algorithm picks out.

I swear to fuck. All this praise to repeat the sins of the 20th century all over again? But oh no, it's okay because it's done by "software engineers".

That and you "atheists techies" are just as fanatically religious as jihadists. Instead of a deity, you worship silicon valley and algorithms.

Of course tech companies can be trusted with our data.

Theres nothing to fear from putting your life on social media if you have nothing to hide.

A select few should have absolute say on what we are allowed to even consider "free speech".

Tech companies aren't in it for profit, they're in it to save the world because they're the "educated elite".

Just... why is this not being shot down? Are you that blind to where it's going to lead? Its literally the same steps every other totialirian psychopath took. Mass identification on bullshit pseudoscience. Then comes the extermination.

cwkoss4• 5 years ago

Evil plan: release this model as an app, heavily branded to skew usage towards my political enemies, and suggest it to be used to find "<opposite side> infiltrators in your midst".

Accuracy issues are a feature not a bug! Now you've sown a bunch of discontent and suspicion within their communities.

nerpderp82• 5 years ago

Is it also 72% accurate for income level?

I think the incomes of people on the right skew lower, or they have fashions for makeup, facial hair etc. I'd be interested in an adversarial attack on the classifier.

IndySun• 5 years ago

The predicted 'answer' for the vast majority of people is going to be, basically, left or right (politically speaking), for maybe 80 or even 90% and more people, a 50/50 choice. How far does the 72% drop for correctly predicting the smaller political factions, I wonder?

patwolf• 5 years ago

At some point in grade school I was home sick for a few days and ended up watching C-SPAN for several hours on end. I don't recall exactly what was going on politically during that time, but many congresspersons were standing up and giving speeches for a few minutes at a time.

I eventually started a game in my mind where I'd try to guess their political affiliation before the chyron appeared. I'm pretty sure by the end I was getting it correct more than 50% of the time. Everyone was dressed similarly, but I remember looking closely at their tie patterns and hair cuts as clues.

joncp• 5 years ago

Upvoted for your use of chyron

xnx• 5 years ago

Isn't it possible in many ML models to then synthesize of the images of the features that most strongly predict the orientation? Like an archetypical "conservative" or "liberal". Could this help identify if the model is picking up on something like facial expression, or facial hair?

karmasimida• 5 years ago

Is this surprising?

We all know there is a demographics/ethnics factor in people's political affiliation.

mensetmanusman• 5 years ago

A data point suggesting bias is more accurate than a coin flip...

lucas_membrane• 5 years ago

That headline is a bit misleading. The facial recognition has to see two faces known to have different political orientations in order to deduce which face is which. The 72% classification correctness is about what one would expect if only 25% of persons had visible features that identified their political orientation and the other 75% were completely inscrutable.

ml_hardware• 5 years ago

Haven't seen this mentioned yet, but ResNet-50 is an old model. I would probably expect multiple-percentage-point gains from using a better (or honestly just larger) architecture and better training methodology.

Throw an ML engineer at this task and you could probably do * way * better than 72%.

sethc2• 5 years ago

And I always thought it was funny how in classic books from authors like Jane Austen they’d always talk about a person’s physiognomy, and how you could draw strong conclusions about them from it. Maybe it wasn’t so bogus after all?

anigbrowl10• 5 years ago

I think there's quite likely something to that. I seem to have better-than-average intuition in this area and it's handy, but not reliable enough that I'm willing to build causal explanations on it.

Sadly, uncritical application of physiognomy has led to many bad outcomes involving discrimination or institutionalization, and the ideas are often used to prop up racism or other sorts of prejudice, so it's not much different from pseudosciences like phrenology or palm reading. Consider too that in Austen's time manual labor was much more common and social/job mobility much lower, so there were a lot of subtle clues that could be picked up from someone's appearance. Think how many folk tales center on someone's ability (or not) to cross social boundaries by modifying their appearance.

btbuildem• 5 years ago

They claim facial morphology was one of the predictors. Pretty wild if true, but how does genetics / heredity play into political leanings?

jhoechtl• 5 years ago

This is a fascinating result. On the other hand it is also telling .. and maybe dangerous? If we can predict political orientation, can we predict if someone is gay? If he is jewish? What comes next?

Would't this legitimate all these voices who ever said I can tell you by just looking?

Applejinx5• 5 years ago

As someone who's studied facial expression in cartooning and tried to track down how that's done through face muscles, this tracks.

Not only that, it is hackable, and somewhat mutable… and I think that can reflect back into one's general attitude on life. I'm going to share some personal notes on my own face hacking done to serve my purposes as a youtuber and open source coder…

The key phrase here is, "The highest predictive power was afforded by head orientation (58%), followed by emotional expression (57%). Liberals tended to face the camera more directly, were more likely to express surprise, and less likely to express disgust." People will respond to you based on what your face is doing, and be more or less favorably disposed to you if you 'match' where they're at.

Guy Kawasaki's on record as trying to maximize his ability to Duchenne smile (crinkle the outside edges of the eyes as your cheeks go up) in order to better influence others. You can make special efforts to dry your skin there, the better to form heavy wrinkles that can come into play, signalling affability and well-disposedness as a proper Duchenne smile would do.

But there's another area. If you fret a lot, or glower, your brow comes down and wrinkles form where your brow meets your nose. This signals suspicion, disgust, hostility. My face-hacking involves putting Nivea cream there and on my forehead, keeping that skin more flexible and mobile, for a more open affable look. But if you're targeting a conservative audience you can do the opposite: look at Tucker Carlson sometime. You can cultivate a world-weary scowl and it will increase your trust with people sharing a similar facial expression, and tend to concentrate your viewer's expressions into ones similar to your own (while you tell them scowl-worthy things), so long as you have their basic trust to start with.

This is all very malleable. Very hackable. You can do it on purpose. I don't know if Tucker Carlson does scowl exercises, but I know if he botoxed his brow scrunch, he would be less effective as a political commentator, because he would be telegraphing the intended reaction to his information more weakly.

We're looking at a general connection between human resting facial expression, and human overall outlook on life. I didn't expect to run across this study but I find it absolutely plausible. Almost axiomatic. You can even frame it in ways that appear to favor one political side or the other, but the underlying principle tells us a lot about how political orientations arise.

nkurz2• 5 years ago

It seems strange to me that the authors of paper seem to have made no attempt to determine which factors the model is using to differentiate between images. Or even to ask the human comparisons how they obtained their greater than chance results. Instead, both the humans and the computer model seem to be considered to be considered perfect "black boxes", which give entirely inscrutable answers.

It's not that it's always trivial to figure out what's happening inside a model like this, but I'm surprised that no attempt seems to have been made: modifying images and seeing which increases the scores, bringing in a co-author who actually understands what the model is doing, seeing if they can obtain similar results with a more interpretable model. Am I wrong, and in fact fining out what's actually happening is impossible, or of no benefit?

austincheney• 5 years ago

Had the political orientation estimates been more precise (i.e., had less error), the accuracy of the face-based algorithm could have been higher.

That is an extremely loose assumption.

bolangi• 5 years ago

So basically not at all.

throw7• 5 years ago

This is hilarious study. This is like Zuckerberg calling us dumbfucks.

pvaldes• 5 years ago

Too much close to a Phrenology 2.0 IMAO

jmpman• 5 years ago

Is the model available? Would be interesting to see how it classifies members of Congress, governors, state legislators, etc. Assuming politicians weren’t used for the training set? (Also assuming Kanye wasn’t either?)

Facial recognition can predict person’s political orientation with 72% accuracy

Related Stories