Microsoft turned down facial-recognition sales on human rights concerns

https://www.reuters.com/article/us-microsoft-ai/microsoft-turned-down-facial-recognition-sales-on-human-rights-concerns-idUSKCN1RS2FV

by dredmorbius • 7 years ago

461 111 7 years ago

reuters.com

Isamu• 7 years ago

This is short but interesting.

Microsoft is aware of the downsides of false positives. In this case California police officers would use facial recognition on individuals, but Microsoft recognized that the false positives could disproportionately target women and minorities. That could result in massive backlash against the company.

So they are calling for greater regulation of AI-related tech, considering the human rights issues. No mention of what "regulation" could mean here.

samayylmao• 7 years ago

Seems like a situation where doing the right thing is luckily in the best interest of Microsoft.

matt40773• 7 years ago

That’s doesn’t square with Amazon’s decision to do the opposite, except in that this “best for the company” argument is so mendable it can be used to argue for anything and its opposite.

A company can make moral decisions, and the fiction that lawsuits are likely or even plausible is almost entirely mythical.

SilasX3• 7 years ago

Wait, what? So if the false positives were evenly distributed across races, then that's no big deal?

vkou• 7 years ago

Unfortunately, in the current political climate, yes. More specifically, a smaller group of people would care about that, then the group of people that cares about the current, non-even mis-identification, so there would be less political will in opposing it.

Cigarettes are considered harmful to everyone, you don't magically get the ability to resist addictive, harmful substances when you turn 21, yet we ban sale of them to minors - for similar reasons.

bilbo0s• 7 years ago

were evenly distributed across races, then that's no big deal?...

These are the perverse incentives when the standard is "equal treatment under the law", or "equal protection under the law". If they can find some way to treat everyone in a crappy fashion, the law doesn't have a problem with that. The successful class actions tend to come only when it can be shown that you are only treating some people crappy.

That would be fairly easy to show in this case. Simply have random whites and blacks stand in front of a camera running MS software and compare the accuracy rates. (Come to think of it, a demo like that would be pretty powerful in a courtroom too. Maybe there is more to this than just PR MS is worried about?)

EDIT:

I just realized how bad that sounded. To be clear, I'm not saying we should treat some people unfairly. I'm saying the law doesn't care whether we treat people fairly or unfairly, so long as they all get treated the same. I was positing that maybe it would be better if we could try to make the law incentivize treating everyone fairly.

laughinghan3• 7 years ago

I'm confused by the premise of your question. Are under the impression that it's possible for a law enforcement tool to have zero false positives?

Nothing has ever had nor will ever have zero chance of false positives. Even DNA evidence can have false positives. Law enforcement, the justice system, and society overall has always had and will always have (highly imperfect) mechanisms to deal with nonzero false positive rates.

Therefore, it is of course important that the false positive rate not be higher than average for a subgroup, as they will be disproportionately affected, since our systems will be set up to deal with the lower average false positive rate.

PeterStuer2• 7 years ago

In short: yes.

Longer form:

Algorithms 'discriminate' (as in differentiate) because that is exactly the job they are tasked with. Is this a picture of a person on list A or not? Is this a picture of criminal activity X? They discriminate on a large number of, often hidden, unknown or not understood features in the data.

In many countries the laws dictate that some features such as race, religion, sexual orientation, ... are protected, as in not legally allowed to be used in differentiation. (Take note that in many countries certain national security/safety related organizations are exempt from certain regulations).

The models that are used in facial recognition rely (in part) on 'sub-symbolic' probabilistic feedback systems, that in many cases defy post rationalization: we do not have a convincing or specific 'story' about how 'the machine' decides in each cases. This means we can not deductively prove that the above mentioned 'illegal' forms of discrimination were not used (note that it is not sufficient to show that e.g. 'race' or 'gender' was not explicitly used as a feature in the input, as it could be strongly correlated with other inputs or derivations thereof (e.g. type of shampoo bought, zip-code, food preference, ...)

So we rely on things like testing post training deployed model to 'vet' the systems aren't biased in the ways we by law and regulation have deemed that they should not be. We test whether the output distribution shifts when we only feed in males vs a mixed gender test set etc.

In practice this means that in compliance testing we replace deductive reasoning with correlation. We accept that this will yield false positives, but this choice is partly due to technical limitations (apart from a few well published cases, understanding and explaining how e.g. a deep learning derived model actually 'works' in a rational synoptic way is still beyond us), and part due to ideological stances we have come to.

So, yes, we choose to accept false positives that are presented as evenly distributed across specific protected groups or features, while not 'making a fuzz' over others. These are inherently cultural, political, moral and empathic decisions, not 'logical' ones.

chris_mc2• 7 years ago

My big question with face rec is this: what happens when the tech is so good that people can find out the identity of someone whose "special" photos were stolen or shared on the internet? That's going to be a huge deal and will cause huge problems for some people if not handled correctly.

forgotmypw9• 7 years ago

In the future, all information will be public, and this will change many things.

dsfyu404ed2• 7 years ago

That doesn't mean a heck of a lot of people won't get screwed over on the way there.

matt40773• 7 years ago

I think society has actually gotten better at this. A strategically leaked sex tape is already an acceptable career move for celebrities, and I don’t think many people would necessary care about it in politics, either.

tracker1• 7 years ago

I think it's often more about hypocrisy than it is about the act/actions. It seems to, at least from my pov, work out worst for conservative/republican politicians than left leaning ones. Of course when you get bad actors on the left, this also means things are overlooked or forgiven that really shouldn't be.

terryschiavo22• 7 years ago

Heresy! How dare you insinuate Kim leaked her sex tape solely for publicity!

chris_mc2• 7 years ago

I don't care about the elites, I care about people who are not well known who get exposed due to this technology, or who's endangered because their stalker who happened on their nude stolen photos can now track them in real time.

Celebrities and politicians have money and power to ensure they recover from this event, your sister or wife doesn't.

dredmorbius• 7 years ago

That problem exists now:

https://www.ynotcam.com/2019/04/10/sharesome-launches-findth...

And has for some years:

https://www.theguardian.com/technology/2016/may/17/findface-...

dsfyu404ed2• 7 years ago

The tech is nowhere near as good as humans are. It's just leveraging the fact that computers can search through a much bigger data set. It's going to take a long time before it gets better than humans considering the fact that progress generally plateaus.

skybrian3• 7 years ago

Another meme from cryptography: attacks only get easier; they don't get harder.

While both are true, they aren't any help at predicting the level at which progress plateaus.

drdaeman• 7 years ago

This had already happened. There are quite a lot of online services that run facial recognition and match provided photos against the datasets derived from porn.

tdeck• 7 years ago

Already happened: https://mashable.com/2016/05/03/facial-recognition-russia-sh...

ergothus4• 7 years ago

I've been harsh to Microsoft in the past (for what I consider good reasons) but I want to give credit where credit is due - this is exactly the sort of behavior that many have been asking for from companies. Foresee a problem, don't do the thing.

It doesn't matter that there's an argument of this refusal being good business - truth is, big companies being confident that consumer outrage won't translate into enough market impact to say they shouldn't do the bad thing has become the norm. Nice to see an instance where that's the not the case.

Y_Y2• 7 years ago

Now that they've gotten the credit for refusing, they'll probably wait a year and do it anyway with some slick PR. You can link this comment then.

SkyBelow• 7 years ago

That's an overly pessimistic view. I give them 3 years.

naikrovek4• 7 years ago

Now that they've gotten the credit for refusing, they'll probably wait a year and do it anyway with some slick PR. You can link this comment then.

This attitude concerns me, a bit. You're mad at Microsoft (it appears) for something they haven't done, and for using "slick PR" that doesn't exist.

What's the basis for the anger?

tru3_power• 7 years ago

I honestly have very little doubt that this will happen.

amrrs• 7 years ago

In the other news, Microsoft accused of being 'complicit' in persecution of 1 million Muslims after helping China develop sinister AI capabilities.

http://www.businessinsider.com/microsoft-china-muslim-crackd...

chefandy4• 7 years ago

Rubio told the FT: "It is deeply disturbing that an American company would be actively working with the Chinese military to further build up the government's surveillance network against its own people [...]"

I can see why the US government is perturbed. The more patriotic thing to do would be helping the NSA, FBI, and a bunch of good, old-fashioned American TLAs build up our own government surveillance network against our own people, and maybe helping the CIA and US Military build up its surveillance network of the Chinese people.

ddebernardy• 7 years ago

Isn't the actual problem lawsuit related?

If I am not mistaking, the shareholders are entitled to sue under the current doctrine for not acting in their best interest.

MS would arguably defend itself making the case that key employees might resign, or key accounts might leave, or whatever else its legal team comes up with. MS might even win such a case for all we know.

Still, a court case could be an unwanted distraction. Or an interesting test case, depending on viewpoint.

jdc• 7 years ago

There is no legal requirement to maximize shareholder "value".

https://www.investopedia.com/terms/s/shareholder-value.asp

Bombthecat2• 7 years ago

What prevents another company from doing this from China or Saudi Arabia?

ergothus4• 7 years ago

Absolutely nothing.

But I find "someone else might do it" to be a bit of a weak excuse to do anything. Every bad thing that COULD happen that HASN'T happened is because someone (usually multiple someones) chose not to do it.

It's really that simple. The first step in no one doing any particular thing is to not do it. And while there are cases where this forces a tough choice, the majority of such cases aren't that tough. Like here: The question is not "do we make mega bucks and be evil, or be poor but good". The question here is "Do we make mega bucks being evil or do we go try to make mega bucks elsewhere?"

jlarocco• 7 years ago

If another company takes the contract anyway, then that company deserves to be publicly shamed and scorned.

Doesn't change the fact that Microsoft did the right thing.

JohnFen• 7 years ago

I'm not able to say this as much as I wish I could, but good on Microsoft!

anbop• 7 years ago

Microsoft is actually lobbying pretty hard in Washington to ban private use of facial recognition. The cynic will say that it’s because they realize that Google and Facebook will outdo them here, but I couldn’t care less about their motives.

StefanKarpinski• 7 years ago

I think it’s not so much that Google and Facebook will outdo them in this area, but about why they are so much more vested in and therefore better at technologies that are opposed to privacy. Google and Facebook as businesses are all about collecting information about their users and monetizing it, which is inherently anti-privacy. Apple and Microsoft actually sell something (hardware and software, respectively), so the smart move for them is to position themselves as firmly pro-privacy—because they can whereas Google and Facebook cannot (without finding new business models). We will increasingly see Apple and Microsoft using privacy features as a competitive advantage against Google and Facebook. We’ve already been seeing this with Apple quite a lot recently, now Microsoft is following suit.

vturner• 7 years ago

They have a long road to walk on privacy with Windows. Isn't 10 a privacy minefield?

Shivetya• 7 years ago

So one day we will wake up in a world where government skunk works developed the technology under our noses deploying it without proper oversight. all under lock and key of courts and laws similar to those that established FISA courts.

No thank you. The tech world needs to acknowledge that the real danger is facial recognition will be developed outside the public eye, without the input and oversight of the technical community that keep the Orwellian nature many suspect in check.

Which means, lead from within. Push companies that want to work on this technology to engage with law makers to insure that privacy rights are respected. to insure the technology does not false incriminate and when unsure it is enshrined in law that it cannot be used to do so. Lead within to establish limits of the use of the data, how it is stored, and how it is guaranteed to deleted within a set period. Limit how law enforcement or other government agencies gain access or use the data. There are no whistle blowers when we are not involved

tl;dr

we either lead in making sure it works and respects our rights or watch as its done in a back room deal with intelligence agencies and the hawks in Congress.

alfl23• 7 years ago

Couldn't implement the tech, decides to have marketing spin off instead.

fizwhiz• 7 years ago

Citation needed?

m0zg4• 7 years ago

What's the point of turning it down? I can train a superhuman-performance facial recognition model with consumer hardware sitting in my garage. The cat has been out of the bag for close to a decade. What leads to abuse in this case is not the technology, but the policy, and withholding technology accomplishes nothing policy-wise simply because it's accessible to just about anyone with a GPU.

rhizome4• 7 years ago

I can train a superhuman-performance facial recognition model with consumer hardware sitting in my garage.

I don't think you can, but it depends entirely on what you mean by "superhuman performance."

m0zg4• 7 years ago

Means it will distinguish faces better than a human in side-by-side testing. This is actually a thing, and it has been for a while. Humans aren't very good at telling if it's "the same guy" or not.

laughinghan3• 7 years ago

But making news for withholding that technology, thereby drawing attention to and sparking discussion about the issue, can affect public sentiment, which in turn affects policy.

There's also supply and demand. Reducing supply of the technology increases its costs.

Plus, it's good PR for them.

m0zg4• 7 years ago

The past 20 years (along with PATRIOT act and FISA courts) show that we can "discuss" until we're blue in the face and nothing is going to change. This also seems to be remarkably bipartisan, for a change. The only set of actors that can change anything in US congress are corporate donors and PACs, and Microsoft is a donor (through its employees and PACs, don't know about directly).

Reducing the supply of this particular technology is not going to increase its costs, because literally anyone with a recent GPU and some motivation can just download it from GitHub at this point. The most worrisome user of it, the US government and its various three-letter agencies, already have and extensively use this tech. Casinos have been using it for well over a decade to spot people who are "too consistently lucky".

I do buy your point regarding good PR. It's just completely ineffectual wrt its stated goals.

balozi• 7 years ago

The facial-recognition tech race has already been lost to the Chinese anyway.

rhizome4• 7 years ago

They have one that achieves high accuracy without requiring human QC? I'm skeptical.

Rebelgecko• 7 years ago

That's nice, although I wonder how effective it is to try and prevent governments from getting their hands on technology that is made available to corporations. What's to stop some consulting company from signing up for an Azure account and selling the same service to the "non-Free" country in the article?

Razengan3• 7 years ago

A future of global surveillance seems inevitable at this rate, but society will evolve means to thwart it.

Wearing masks in public is already a norm in some East-Asian countries, and it’s a form of fashion as well [0]. Just add glasses to that, which will become common if AR smartglssses take off.

Maybe we’ll see people walking around in fashionable neo-tribal’ish masks [1] (as decorations around AR glasses) to hinder intrusive face/retina scanning.

[0] https://www.bing.com/images/search?q=face+mask+fashion

[1] https://www.bing.com/images/search?q=killmonger+mask

lkadr• 7 years ago

Some Europeans countries (France, Belgium, Netherlands, Austria ) and China have banned hiding one's face one's face in public. Those Asian face masks would be OK, but tribal masks wouldn't.

Razengan3• 7 years ago

Do those bans include Islamic Hijab etc.?

Again, if those medical/hygienic face masks are OK, wouldn't decorative frills (say, feathers) around sunglasses be OK too? That combination would effectively cover enough of a face.

snaky• 7 years ago

China has begun rolling out new surveillance software capable of recognising people simply by the way that they walk.
The "gait recognition" technology, developed by Chinese artificial intelligence firm Watrix, is capable of identifying individuals from the shape and movement of their silhouette from up to 50 metres away, even if their face is hidden.

https://www.independent.co.uk/news/world/asia/china-surveill...

kkarakk• 7 years ago

you can fool gait recognition simply by putting a stone in your shoe,gait recognition is no where as sophisticated as it claims to be.

basetop• 7 years ago

What about their OS, Office Suite, DBs, etc? Do they turn down sales of those on human rights concerns too? Will Microsoft stop taking "telemetry" data with its software?

Would they reject any sales to saudi arabia, china, israel or even the US military if human rights concerns arise?

If this is part of a genuine ethical paradigm shift within the company, then I commend them for it. But if this is just a one-time PR move, then my opinion of Microsoft has dropped considerably.

_cs2017_• 7 years ago

Microsoft said they refused to provide the tech due to the bias against minorities caused by the training data.

How difficult is it to fix this bias? For example, the model can be told to only produce a match when a confidence level is higher than a certain threshold. Then the threshold can be increased as needed on those subsets of faces where training data is lacking. Would that work?

Also, why not build more diverse training data if this is a pervasive problem? It is not free, but neither is it cost prohibitive for someone like Microsoft.

grumpy-cowboy• 7 years ago

Microsoft turned down facial-recognition sales on PR concerns.

xrd3• 7 years ago

This seems positive.

Yet, this article sounded like a PR piece from MS. Is it just me?

Has Reuters had issues in the past accepting work from journalists who are paid flaks?

Is my Spidey sense off here?

xrd3• 7 years ago

The journalist Joseph Menn looks like he has stellar integrity as a long time writer (despite hiding Beto's involvement with Dead Cow cult) so it must just be me.

laughinghan3• 7 years ago

despite hiding Beto's involvement with Dead Cow cult

Huh? Isn't Menn's Reuters article the one that broke that news?

I'd also question how much the public interest would be damaged if that information hadn't come to light.

ei8htyfi5e• 7 years ago

Are there just PR departments behind the scenes leaking things like this following the other bad press received by other tech firms? Anybody have experience with this world? How far out can you predict media coverage I wonder.

Vinnl• 7 years ago

I think the best part of this is that they're talking publicly about this, and are explaining our reasoning. We shouldn't be dependent on Microsoft making the ethical judgment correctly every time. Instead, (potential) customers themselves should learn about the potential downsides, lest they simply go to a competitor to purchase the same flawed technology anyway.

MattSayar• 7 years ago

My first impression: This is a good thing overall. But then I think, well some other bidder will do it. And what if they do a worse job? Does that lead to detaining the wrong person? Is that better?

Microsoft turned down facial-recognition sales on human rights concerns

Related Stories