A Horrible Experiment

https://mobile.twitter.com/bascule/status/1307440596668182528

by marvindanig • 6 years ago

87 48 6 years ago

mobile.twitter.com

pityJuke• 6 years ago

Some more info:

- This apparently done via a NN [1]

- There are mixed results, with various factors playing a factor [2].

Worth digging into this thread, and the original thread where this was figured out. It's intriguing.

[1] https://blog.twitter.com/engineering/en_us/topics/infrastruc...

[2] https://mobile.twitter.com/NotAFile/status/13073372942491033...

DanBC2• 6 years ago

greenduck• 6 years ago

This is pretty egregious.

For those of us working on machine learning, does anybody know of datasets that have decent representation of different colors of skin. That seems like the cause here and thinking back, my datasets aren't the most diverse out there.

TazeTSchnitzel2• 6 years ago

ML inherits the biases of the data it is trained on. If you want “unbiased” ML models, you must train them on data that is carefully constructed to avoid the biases deemed undesirable.

GuB-42• 6 years ago

I remember I took an introduction class in machine learning when I was a student.

I was surprised how important bias removal was. Preparing the dataset was more than half of the work. And the class was more about statistics than anything else.

It wasn't about racial bias, ethics, or anything like that. More like if you feed too many "5" in your handwriting recognition algorithm, it will will start seeing 5 everywhere.

Lammy3• 6 years ago

No it isn’t. Look at the massive empty space this person had to construct between the two content areas of those tall images to get it to behave like this. No reasonable person would ever create something like that accidentally in a way that would create this behavior.

svrb3• 6 years ago

The dataset linked from this thread on Twitter may serve your needs too?: https://twitter.com/vinayprabhu/status/1307497736191635458?s...

kingosticks2• 6 years ago

There's a whole lot of random people chiming in and guessing on Twitter (shocker) so it is worth pointing out that:

The algorithm does not do face detection at all (it actually replaced a previous algorithm which did).

From https://twitter.com/ZehanWang/status/1307461285811032066?s=2...

Not to say there is or isn't a problem to be fixed, more work is needed to understand. Such as the analysis at https://twitter.com/vinayprabhu/status/1307460502017028096?s...

8fGTBjZxBcHq• 6 years ago

Why is it worth pointing that out? Like you said, the problem is there either way. What are lay people supposed to gain from hearing that the algorithm's designers don't consider it to be based on face detection?

kingosticks2• 6 years ago

Because it has totally derailed many of the twitter threads. Thankfully we get less conspiracy theories here but I figured we could try and avoid it altogether.

Gys• 6 years ago

It seems this needs a twitter account to see? Maybe someone can post a screenshot?

acqq2• 6 years ago

And for those of us who need a textual description to get it, maybe someone can post an explanation or description of what is expected and what's an observed behavior?

Anybody, please?

Nzen• 6 years ago

Twitter crops rather than rescales posted images to some 'interesting' subset of an image that doesn't fit the preferred aspect ratio. So, the person behind this account created an antagonistic situation: made a very tall, thin image with Mitch Mconnell on the top and Barak Obama on the bottom, and another with them swapped between top and bottom. In the initial post, the cropped thumbnail shows Mitch Mconnell for both. The thread has two more situations, one with the ties (red/blue) flipped and the colors inverted. The only time it 'prefers' Obama's end of the antagonistic image is with inverted colors and him on top.

yc-kraln• 6 years ago

Twitter has an algorithm to pick which part of oblong images to showcase in the feed. People have been posting examples where, given the "choice" between a white person and a person of color, the white person is always picked, regardless of orientation or where they are in the picture.

This was noticed in a thread about how Zoom's background detection seems to not work great for people of color, and how the WebEx cross-talk protection seems to favor men over women.

fr2null• 6 years ago

Interestingly, swapping the glasses does seem to make the thumbnail Obama. [1]

1. https://twitter.com/SergioSemJ/status/1307493041742254080?s=...

nichos• 6 years ago

So does the smile: https://twitter.com/MASCARPOWN/status/1307454405059452928?s=...

And actually swapping the smile:

https://twitter.com/DrWhen2/status/1307497434872905728?s=19

Twitter is nothing if not full of victims and people talking about things they know nothing about.

dazbradbury• 6 years ago

https://twitter.com/SergioSemJ/status/1307493041742254080?s=...

One theory is it's actually the glasses increasing the likelihood of correctly identifying a face (or "feature" element in the photo). Interesting looking through the various tests in the thread, as clearly lots of variables here, but no way this will be exhaustive.

Suspect this might kick off some serious arguments at Twitter HQ for allowing users to pick the cropping vs. an algorithm regardless of any bias found one way or not.

DanBC2• 6 years ago

Here's an example where neither person is wearing glasses.

https://twitter.com/LOVEMUNY/status/1307789227535499266

It also includes an example where the black man is wearing glasses and the white man isn't.

nailer• 6 years ago

Misleading - it seems to be looking for contrast

https://twitter.com/kim/status/1307548258491801600?s=19

UPDATE: someone wrote a boy to test the conspiracy and properly debunked it, source code is online:

https://twitter.com/vinayprabhu/status/1307497736191635458?s...

Kednicma• 6 years ago

Humans can't make unbiased computers. Every computer which humans make is indelibly human as well. This is the anthropic bias.

noxer10• 6 years ago

It's not a biased if isn't a result of a human bias. If the algo picks the spot with the highest contrast for example, that can not be biased it may lead to unwanted results but its not biased.

Aeronwen• 6 years ago

Someone decided the algo was producing usable results when it was picking the highest contrast areas.

TazeTSchnitzel2• 6 years ago

Wonderful example of the ethics of ML. The people whose eyes they tracked gravitated towards certain kinds of faces (not necessarily just based on skin colour), so now Twitter's model crops other faces out of pictures, among other problems (now it likes cleavage apparently).

The old approach may have been a bit mechanical but it avoided the non-neutrality of unconscious human perception. The assumption that what human eyes gravitate to is what an image should be cropped to is a very big and questionable one. The first thing I look at in a picture is not always the most important part, and the image as a whole has value.

dmitriid• 6 years ago

It's funny how these big companies willingly convert a non-problem into massive problems with hundreds of hours of engineering effort and huge social implications.

Non-problem: don't mess with the images, and display them as is.

Problem: see Twitter

t0astbread• 6 years ago

Why does Twitter even try so hard to make preview crops "meaningful"? Sure, it produces interesting results sometimes (sometimes funny, sometimes less so) but isn't it a lot of work for not a lot of benefit?

noxer10• 6 years ago

Agree, resize would in most cases be way more useful.

Lammy3• 6 years ago

I used to get annoyed by this too but came around to accepting it after building a Web (Jekyll) photo processing pipeline of my own. Art direction is very difficult to do well when you have to be able to handle any user-uploaded photo that might be in any aspect ratio. Desktops favor displaying wide photos, and handsets favor displaying tall ones. Cropping everything down to a square lets you handle any combination of photo and display aspect ratios with minimal Badness.

I’ve never seen the Twitter-dot-com codebase but would be willing to bet it’s probably using VIPS’ smartcrop feature (:crop => :attention) https://www.rubydoc.info/gems/ruby-vips/Vips/Interesting

matthewmacleod2• 6 years ago

Cropping well is quite a hard problem!

I know it seems on the surface like “just resize it” is a good approach, but it’s not super uncommon to see quite extreme aspect images, and even for normal use there are lots of different constraints in different systems that mean a really simple system can produce undesirable results.

I used to get this all the time back when I built custom content management systems. Users would upload images which were automatically cropped into various sizes for use in particular bits of content; really long or wide images would look awful when scaled, and automatic fixed-size crops would often end up cutting out the actual interesting bit of the image. That’s what various “automatically generate interesting thumbnails” algorithms try to avoid.

In the end for me the best tool was auto-crop-with-preview-and-manual-override, and I’m beyond infuriated every time I try to upload an image to Twitter and it doesn’t allow this obvious and simple way to avoid the issue.

daenz• 6 years ago

Isn't Mitch McConnell in the news right now far more "intensely" than Obama? If I was writing a system to pick a thumbnail, I would pick the thumbnail that is most relevant to current tweets, which is definitely McConnell.

A quick way to test this would be to use two well known figures, where the PoC is definitely more in the news than the other.

noxer10• 6 years ago

The real problem I see here is the default assumption from many people that these is racial bias in the algorithm with no evidence thereof whatsoever. Thats how twitter works, shitstorm first, research later, evidence or corrections for false assumptions - never.

matthewmacleod2• 6 years ago

I mean “no evidence whatsoever” is a pretty big stretch. There’s the initial example, a bunch of other examples of the same effect posted in the thread, and it’s already a really well-known and obvious effect that we have seen occur many times before.

noxer10• 6 years ago

That called "anecdotal" not an evidence.

svrb3• 6 years ago

It's also how HN works. In a comment far below there's a link to a decent controlled experiment completely debunking this claim (outcome using controlled dataset: 40 white to 52 black faces chosen). But that comment is heavily downvoted.

https://twitter.com/vinayprabhu/status/1307497736191635458?s...

noxer10• 6 years ago

People want it to be true. Instead they should want to know if its true.

acqq2• 6 years ago

(outcome using controlled dataset: 40 white to 52 black faces chosen). But that comment is heavily downvoted.

Maybe because

1) If that's the comment you refer to, it also contains the claim "Misleading - it seems to be looking for contrast" which as far as I see isn't provably true?

2) in the end tweet of the experiment it's claimed: "I've created @cropping_bias to run the complete the experiment. Waiting for @Twitter to approve Dev credentials" and the https://twitter.com/cropping_bias is still empty at the moment?

Edit: personally, the interesting experiments for me are like "a person against a drawing" -- the simpler the competition is the bigger the chance to figure out what the "preference" could be:

https://mobile.twitter.com/griffonatrix/status/1307463528337...

(Luckily, I don't depend on Twitter, but I'd suggest that those who do should have the chance to do the "manual override.")

rambojazz• 6 years ago

I'm out of the loop. What's happening here? All I see is a picture.

s_gourichon• 6 years ago

Glad you asked. There are two pictures in each tweet, shown left and right.

You can click on each to see the full picture.

Each picture is actually two portraits (Mitch McConnell and Barack Obama), one above one below, separated by a big gap.

Since apparently Twitter automatically chooses to crop pictures in default tweet display, the OP posted different cases to see where Twitter would crop the image. And it happens that Twitter chose one of the portrait always, until the OP reversed color. Some people wonder if it is because Barack Obama is black. Whether that's the reason or not, the observable fact remains.

A Horrible Experiment

Related Stories