America is turning against facial-recognition software

https://www.economist.com/united-states/2019/05/25/america-is-turning-against-facial-recognition-software

by pseudolus • 7 years ago

269 138 7 years ago

xena4• 7 years ago

Good. This kind of software is a mirror to the data fed into it. People on the edges of society (that the people making the mirror might either intentionally or accidentally ignore) will get fucked over tremendously by this technology. We should put a complete ban on it. Even if it means gimping my photo organization programs.

diafygi• 7 years ago

Just so I have good arguments when talking to someone about this specific point, can you or someone else please go into more detail on why this will happen?

donohoe• 7 years ago

Not specific to facial recognition but covers it and the deeper issues. It is a thorough yet accessible primer on the many ways bias can creep into these systems:

https://parametric.press/issue-01/the-myth-of-the-impartial-...

paganel• 7 years ago

Not the OP by technology like this will explicitly make racial profiling legal again, because the power that be will hide behind the decisions taken by algorithms: “The computer told us to stop and search this black person’s car, it wasn’t us, the computer is smart, we were just following the computer’s orders”.

trevyn• 7 years ago

Along the lines of “everyone is breaking some law”, there could be a shift from “technically illegal but generally harmless and tolerated” to “illegal and mindlessly enforced by automated technology”.

An example being China’s automated jaywalking-shamer.

bryanrasmussen2• 7 years ago

There have been numerous issues where corpora of peoples faces have been insufficiently stocked with a particular racial group, in such cases people's faces might not register as faces and therefore be shut out of whatever the facial recognition is supposed to implement.

For example facial recognition at a high priced building, suddenly non-white people can't get in because you did not show a real face at the scanner. Sorry, it's a bug not a feature.

The same problem would of course apply to anyone with facial deformities etc.

The problem would be essentially of two forms - Automatic exclusion from a group by facial feature, or automatic inclusion in a group by facial feature.

Think of the automatic inclusion as advanced phrenology or other sort of woo, in this scenario company X sells the idea that they will have facial recognition of 'criminal types' based on whatever research they can pull out that criminals often correspond in appearance. Then they fill that corpus up with criminals which then is a nice self-selecting system of find the black/hispanic person and deny them the job, loan etc. because of likelihood for crime past the threshold set.

the reason why this would happen is the same reason why banks used to deny loans to people in certain zip codes, because by doing so they could be racially profiling and excluding but pretend not to do so.

lazyvoice5• 7 years ago

For example, let's assume you live in Brunei where homosexuality is a capital crime. Facial-recognition (especially in concert with other pattern-recognition software) drastically reduces the man-power necessary to identify individuals for enforcement.

If you're talking to someone who's more conservatively inclined I'd talk about how Iceland has managed to basically eliminate Down Syndrome. Prenatal pattern recognition has successfully convinced nearly all Icelandic parents who have a pregnancy where the child may have Down Syndrome to abort.

If your listener doesn't care about gay rights AND thinks any fetus with a higher likelihood of having Down Syndrome should be aborted, then they're probably exactly the sort of people working on or hoping to use these tools.

dsfyu404ed3• 7 years ago

I fully agree with your sentiment but I don't agree with a ban. Prohibition doesn't work. You should be able to use it but the legal risks and liability you open yourself up to by using it should be strong enough to deter its use except when there truly is no substitute. Asbestos comes to mind as an example. People (often on HN) often run in circles and scream about the sky falling because "OMG it's not banned" but the reality is that the court of public opinion has basically ruled it is not ok to use. That's how I want to see facial recognition tech and other tech that enables corporations and governments to stalk people at scale treated

rmrfrmrf3• 7 years ago

Prohibition doesn't work

You're appropriating sociological behaviors of the individual to maxims at the corporate level, which is ridiculous. Corporations need to be regulated, not coddled in some kind of convoluted, scaled-up, harm-reduction policy.

xena4• 7 years ago

For instance, I am transgender. This kind of technology would allow them to identify "masculine" features of my face that I didn't ask for. This could cause me to be profiled as male, regardless of what my court order or personal preferences say.

This kind of technology is fundamentally dangerous to people who do not fit cleanly into the rigid caste-like boxes that are used to categorize people.

moate• 7 years ago

Counterpoint: How many cloned humans have we seen?

Prohibition works when the barrier to entry is higher. Any mook with some yeast and sugar can make alcohol. Pot grows just about anywhere (which is why they call it weed). But not a lot of people are privately developing nuclear bombs or cloning humans.

stared2• 7 years ago

As a person with prosopagnosia (vide http://web.archive.org/web/20080704154655/http://www.prosopa...), I would love to have personal software for recognizing people.

At the same time, I am aware of the dangers of people doing it for commercial purposes (on so many levels!).

jasonlotito• 7 years ago

It's things like this that always make me think twice about banning things. Many people look at something as useless, wasteful, or bad, but to other people, it really does help people with certain conditions. So when you outright declare something is dumb or should be banned, I generally find these people to generally be ignorant about all the technology does, and it's just a knee-jerk reaction.

This is not to say they don't have good reasons for not liking a technology. Rather, that tend to not know all sides for something.

JamesAdir• 7 years ago

check out this: https://www.kickstarter.com/projects/1070992105/orcam-myme1s...

toephu22• 7 years ago

You have never been a victim of a crime have you?

I'm pretty sure if you were, and the crime was caught on camera, your position on facial-recognition software would change.

ashelmire• 7 years ago

Or you'll just get harassed more as a black person[1].

Funny enough (actually the opposite of funny), the systems used by law enforcement are designed to create more false positives, which then leaves it up to law enforcement to decide which "leads" to follow up on. This will certainly be used to validate abusive behavior.

Additionally, facial recognition is used in some countries right now to silence political dissidents[2]. If you think that can't or won't happen in the US, I'd suggest you look at how activists have been treated in the past (beaten, arrested, and sometimes killed).

1. https://www.nytimes.com/2018/02/09/technology/facial-recogni...

2. https://www.eff.org/pages/face-recognition

Barrin92• 7 years ago

I'm pretty sure if you were, and the crime was caught on camera, your position on facial-recognition software would change.

yes, but likely in the wrong direction because I'd be angry and personally insecure. Decisions of liberty, privacy and law shouldn't be based on personal experiences or trauma.

I find this line of argument weird. Do you really think a personal decision about long term social issues is better than an impersonal one?

mr_toad• 7 years ago

We should put a complete ban on it.

Both my phone and my PS4 use facial recognition for login. I don’t want that banned.

My camera can be set up to take a photo when it detects someone in the frame. Very useful for selfies. Are you going to ban that too.

Are you going to ban Tensorflow. Keras? SciKit-learn?

If I hand code my own neural net, are you going to ban that as well?

vangelis• 7 years ago

I disagree with a complete ban, but we could ban wide-scale implementation of this. You are not a municipality.

grenoire5• 7 years ago

This has been debated extensively on HN (and is still being debated actively), but all I hope is that people realise sooner or later that the level of safety you can acquire in a society is inversely related to the level of privacy individuals can have.

What I personally hope for is that people achieve a balance between safety and privacy.

RoryH• 7 years ago

Nouvelle idea: maybe we should just lock people up who violate people's safety instead of taking away everyone privacy ;-)

jihadjihad• 7 years ago

I ask, safety from what? From citizens or the state? Your statement implies that the freest societies are the least "safe", and the most repressed are the "safest". While you might be able to hinder criminality on the part of citizens with total surveillance, the cost of giving anyone that kind of power is too high.

Relinquishing privacy puts a society on a path where all power accrues to the state, to the detriment of all citizens.

grenoire5• 7 years ago

I agree to most, but because solutions like facial recognition are simply easier to engage than educational reforms or welfare-based safety [sic] nets, this is not necessarily something we get to choose.

States will prefer the former to the latter and firms will provide, at which point I don't think that the definition of safety-from is necessary.

Neutrion• 7 years ago

when you talking about balance, you must associate a timeframe with it. the amount of safety you get by sacrificing a single unit of privacy can be changed through technological advancement.

SCdF• 7 years ago

all I hope is that people realise sooner or later that the level of safety you can acquire in a society is inversely related to the level of privacy individuals can have

What leads you to think that this is always true? Do you not think there are any other solutions to reducing crime other than increasing surveillance?

grenoire5• 7 years ago

I do, but at the same time I think that lower-hanging, yet not so easy-to-pick, fruits are ignored already. In that case, it becomes more feasible to target far out approaches that are simply easier to implement. Unfortunately, those also happen to be ones that compromise privacy.

Moru• 7 years ago

It's easy to use face recognision on someone that has nothing to hide. That does not make it useful to track someone that does have something to hide. Is it enough to put on a mustache and some glases when you walk around a corner to fool the system? A painted face on the t-shirt and a helmet on the head?

cheschire• 7 years ago

That premise assumes that one can control safety, but that might be a hard assertion to build shared understanding on because not everyone shares that assumption.

Perhaps you would have more luck trying to build a foundation on the relationship of risk management and privacy. Mitigation vs acceptance vs avoidance, etc.

pjmorris• 7 years ago

How are you measuring safety, and how are you measuring privacy, to make the 'inversely related' claim? What is your evidence for this claim, measured in the measures you are using?

lazyvoice5• 7 years ago

How does this differ from "If you have nothing to hide you have nothing to worry about."?

Surely it's not difficult for you to imagine fairly common scenarios where your equation is patently untrue. A battered spouse of LEO or a non gender-conforming individual in a theocratic society are only two really easy examples that come to mind.

If we assume that individuals who control or have access to surveillance mechanisms are universally magnanimous and actually capable of neutralizing threats to safety, then your negative relationship between safety and privacy might hold up to scrutiny. I have more than a bit of trouble seeing how that assumption is anything other than utopian.

grenoire5• 7 years ago

I don't mean to defend surveillance as a gateway to safety by any means, in fact I believe in the exact opposite. Unfortunately, it's going to be integrated into our society whether we like it or not, and after that point it's only a matter of time before we come to accept losing our privacy; having ignored all the exceptional "casualties" and enjoying the add scrutiny it yields to law enforcement.

arrozconcosas3• 7 years ago

"Nothing to hide" is the worst argument possible on the subject

stubish• 7 years ago

I don't think it is inversely related. At a point, you become less safe. I would not feel safe if every moment of my life was broadcast to everyone; I would be walking a razors edge, where I pay for every minor slip up. Life would be one long anxiety attack. Privacy means being able to relax, and absolutely required for mental and physical health. No privacy is inhuman.

dsfyu404ed3• 7 years ago

This is a meaningless statement without a threat profile.

Sure if everyone lives in a police state under the thumb of the government you're safer from petty crime but you're a heck of a lot less safe from government abuse.

Loughla4• 7 years ago

Look at the news cycle. Most stories are about how crime is everywhere and your family is in danger (nevermind that the local news channel in Northwest Iowa had to report from Alabama to find some blood and gore), meanwhile, the government is fighting a noble war against terrorism in some far-off country.

I don't generally subscribe to conspiracy theories, but I do genuinely believe we're (in the US at least) being groomed for some sort of repressive state apparatus using this type of software to 'keep us safe' in the next decade or two.

arrozconcosas3• 7 years ago

It is not zero-sum. You can increase safety while keeping privacy. Putting airbags in cars makes us safer without losing privacy (to any meaningful degree). Even if there is a relationship curve, you can shift the curve or jump between them.

soulofmischief• 7 years ago

Those who would give up essential Privacy, to purchase a little temporary Safety, deserve neither Privacy nor Safety.

EugeneOZ3• 7 years ago

I'm ready to sacrifice my extremely convenient way to unlock iPhone just to know this technology is banned worldwide.

This technology opens wide doors for surveillance, for finding, identifying and pursuing political opponents - not just their leaders, but every person who was brave enough to protest on the streets.

It opens wide doors also for cyber bullying, and this issue is a problem for every country, not only for dictatorships.

tobylane• 7 years ago

What would you expect to be achieved by banning it? Intelligence services will continue with it, which is my major concern over its current use.

EugeneOZ3• 7 years ago

It will be less widespread and fewer engineers will push it forward.

walshemj2• 7 years ago

I think the concern is more widespread use by LEO and corporates who have far less oversight - and will effect several orders of magnitude more people.

forgottenpass• 7 years ago

Jack Marks, who manages Panasonic’s public-safety products, called it “short-sighted and reactive”. The technology exists, he said; “the best thing you can do is help shape it.”

Not using it IS "shaping" it. Just not into a shape profitable for this dishonest huckster.

alfiedotwtf• 7 years ago

America is turning against facial recognition, until it's not. Total surveillance looks like it's inevitable.

Surveillance will come slowly, then suddenly

vfc12• 7 years ago

It's like nuclear power, technology is what we make of it.

Do we want to use it for criminal purposes only, effectively to almost make it impossible for someone to commit a violent crime and get away with it and make a much safer society?

I'm all for it, there are some nasty people out there, serial killers, pedophilians, drug cartels. This technology deployed at scale would make these crimes much harder to get away with.

As long as it's another way to ensure democratic laws are respected, I don't think that completely forbidding it makes sense.

I'm all for regulation, we don't want this technology to be abused.

coding123• 7 years ago

Ski mask. You might catch/convict a guy with it, but soon after a lot of people are just going to wear them all the time.

vfc12• 7 years ago

I don't know how effective is this technology to identify masked people, if at all.

But then again, how suspect is that to wear a ski mask? Everyone will notice.

bpyne• 7 years ago

Yup. About half the US needs them for 3 or more months per year. And then there's the rise of surgical masks to prevent the spread of viruses.

I also foresee a surge in wide-brimmed hats.

johaugum• 7 years ago

The same fundamentals used in facial recognition can be applied to 3D pose estimation for gait recognition. This allows identification of a person without biometric features like a face or iris [1].

Facial recognition is used as a catch-all term because it's easily understood, but the deep-learning behind it is much more widely applicable.

[1] https://arxiv.org/abs/1710.06512

yusi-san• 7 years ago

The thing is it depends of what you call 'criminal' on the moment, right ?

And when you know you're watched, you start self-censoring to adapt what should be the 'right thing' to avoid blame or judgment.

arrozconcosas3• 7 years ago

Too late

Photos are everywhere, the software will only get to be more accessible. You can't put this back in the box.

alexnewman• 7 years ago

Actually there's a proliferation of private facial recognition software being deployed across almost all large buildings in the united states. it's just behind the scenes and not being used by law enforcement. This is the standard for america, law enforcement gets barred from high tech, but private industry is free to use it. Eventually they sell it back to law enforcement.

briatx• 7 years ago

Earlier this year the Washington Post reported that many small police departments were abandoning body-worn-camera programmes because of the cost.
Although the cameras are cheap, officers can generate 15 gigabytes of video per shift; storage costs mount. Police unions often oppose body-worn cameras, fearing they imperil their members by giving superior officers licence to search them for punishable behaviour.
Other officers complain about the amount of time required to review and redact footage in response to public-information requests.

Sure, the Police themselves don't like the cameras turned on them. Suddenly it is "too expensive".

Then they will turn around and argue for ubiquitous CCTV camera installations.

PunchTornado• 7 years ago

would you want a body camera at your work? You wouldn't I'm sure. Don't blame them for looking for their interest. It's someone else's job to (politicians) to legislate the use of it.

majormajor• 7 years ago

Don't blame them for looking for their interest.

Blame everybody for only looking out for their own interest, from scammers to corrupt politicians to short-sighted investors to oil execs. Otherwise it's just status-quo infinite whack-a-mole and we see how well that's going.

antisthenes• 7 years ago

What a poor analogy. My job doesn't have anything to do with being the sole arbiter of violence and defusing dangerous situations at the trust and expense of taxpayers and local citizens.

If they don't want accountability as police officers, they can go find another job.

Don't blame them for looking for their interest.

I can, I will and I will argue that it is my moral obligation to ensure that the public servants are acting in the best interest of the public as a citizen. But thanks for your concern.

dylan604• 7 years ago

In my job, I'm not provided a set of lethal and non-lethal weapons. In my job, I am not given the ability to detain anyone. In my job, when someone aggravates me, I do not have issued weapons to use against the aggressor. In my job, I'm not allowed to use deadly force because I feel threatened. I blame them 100% for only looking out for themselves and not the citizens they are sworn to protect.

lohszvu• 7 years ago

It doesn't really matter if they have a body camera because the courts never convict them anyway

xenocyon• 7 years ago

That's not exactly correct. There are several factors here. A big one is that prosecutors (whose jobs depend on police cooperation) have been reluctant to go against police; this probably won't change with video evidence. However, another factor is that jurors are currently inclined to take the police officer's word for it in he-said-she-said type cases, and this may change if bodycam video evidence becomes more plentiful.

ssully• 7 years ago

Since 2004, Chicago Police Department paid out $662 million [1] in police misconduct settlements.

The optimist in me says that the cost of proper police reform, training, body cameras, video storage, and actually punishing bad cops would be much less then that.

[1] https://www.chicagobusiness.com/article/20160320/NEWS07/1603...

mindgam33• 7 years ago

I find this to be a really compelling counter-argument to the whole "accountability is too expensive" line. Even if proper tech + training cost only reduced rate of severe misconduct by 15%, that's over $100M in savings.

I'm sure implementing reform etc isn't cheap, but I seriously doubt it's on the order of $100M.

mindgam33• 7 years ago

Axon, which makes body-worn cameras and Tasers... is building a system for managing records...the firm’s chief information-security officer, says that “what we can do to help officers improve most isn’t the sexy stuff. It’s helping them be more efficient and spend more time on the street.”

Uh... what they could do to help officers improve most might be shipping a device that doesn't seem to mysteriously malfunction or be turned off during the most controversial use of force incidents.

The way bodycams are supposed to help is by forcing officers to be accountable for their actions. When police have plausible deniability due to poorly designed tech, all these expensive devices become basically worthless.

In my cynical moments I have wondered if making such bad tech is actually a selling point for Axon and its competitors. They're selling to police departments; the less likelihood bodycams have of embarrassing police, the more likely they are to make a sale. Check the box, move along. I try to have more faith in our institutions, but the abject failure of bodycams from a technology perspective makes me wonder.

I supposed it could also just be due to a few companies having a lock on an enterprise hardware market.

thatoneuser• 7 years ago

Huh so the cops think giving cops the ability to search for punishable behavior might be an issue...

linuxftw• 7 years ago

America isn't turning against anything. There was no shortage of people that bought the new iPhone with face scan technology.

I believe we live in a society where it's already possible to track the vast majority of people to a general area 100% of the time. Technology is progressing, soon it will be real-time tracking of targeted individuals 100% of the time. That will slowly expand to all individuals. Between cell phone tracking, license plate scanners, and facial recognition cameras, the system knows where you are.

johnrbent• 7 years ago

There is a lot of sensationalism (see title of article) and high, anxious emotions around facial recognition right now, which really is to be expected at this point. But it is muddying the rational conversations we should be having about it. Can Amazon lessen Rekognition's bias towards white people (ie. bright pixels)? Yes. Get over that. This article completely whiffed on an important point regarding police use of facial recognition (or maybe I missed it) and that is _police_ bias. We are talking about one of the most culturally and racially biased concentrations of power in the country. It's important to note that they will NOT be training the models or writing the algorithms that drive their tools. We have a chance to let some brilliant engineers create deliberately unbiased tools that can only improve the situation in America's police force. PLEASE do not let your fears win. It will be used for evil and it will be used for good. That is not something that should be banned. Sorry I started getting a little emotional.

justwalt• 7 years ago

I think the folks who are against facial recognition because they view it as “racist” are in the minority.

Loughla4• 7 years ago

The problem I have with your statement is that it seems to put 'brilliant engineers' on some kind of magical pedestal.

Engineers, much like police, are humans. Therefore they hold biases, they can be short-sighted, and they may not understand many of the long-term ramifications of their jobs.

All this does is move the power and bias up the chain from police to engineers while simultaneously making some very, very powerful decisions about what privacy is and what your rights are in society with 0 oversight from the actual society. In fact, police are (in theory) beholden to politicians and the voting public right now. If piss-poor decisions are made based on faulty software from software conglomerate A, who do we hold accountable?

mLuby• 7 years ago

Facial recognition just vastly increases the likelihood to locate a suspect. What happens after that, from detention to arrest through jail and trial remains unchanged, and is still conducted by humans in the criminal justice field.

notus• 7 years ago

Facial recognition is coming no matter what. There is no way people will sacrifice the conveniences that come along with it. Sure people will be against it when their only concept of its applications is to "catch bad guys", but when it begins to cut time out of their day they will have a change of heart.

mensetmanusman2• 7 years ago

This backlash is the silver lining to China’s shortsighted implementations of this technology to their world renowned police state.

toephu22• 7 years ago

This may be an unpopular opinion here but on the flip side have you been to China lately?

Street crime (including petty thievery) has dropped immensely.

Any city in China is much safer than Oakland.

mooseburger• 7 years ago

Hasn't that always been the case in China?

themacguffinman• 7 years ago

I don't think safety is the only thing that matters to people.

mensetmanusman2• 7 years ago

There is always a silver lining to authoritarianism.

snowwrestler• 7 years ago

I can't read the article without an Economist account, but I think the headline is a bit too broad. There are a lot of people who happily use facial recognition to unlock their iPhones every day, and I haven't seen much of a backlash against that.

Why not? Because it's isolated on the device, and can be turned off whenever you want. I think the real concern is the combination of facial recognition with vast centralized databases. Those big centralized databases are just begging to be exploited and abused, whether for commercial, authoritarian, criminal, or even creepy "LOVE-INT" stalking by the people running the databases.

I think facial recognition (and other forms of recognition) can have a bright future in society, but only if they conform to our expectations of privacy. Those expectations have been created over thousands of years based on how we interact with people. The closer that computer interaction can hew to existing social conventions, the smoother path it will have to adoption.

I don't mind if the waiter in a local restaurant recognizes me when I come in; in fact I like it! But I don't think I would like it very much if I knew he was calling 15 different data brokers to report on me every time he sees me.

tim333• 7 years ago

I can't read the article without an Economist account

try https://outline.com/BhW2BL

povertyworld3• 7 years ago

Getting discouraging to see facial recognition and machine learning getting conflated in every article's comments.

dwyerm• 7 years ago

Yes, but the thing is that facial recognition IS just video compression.

You take a video stream and compress it down into a timestamped stream of IDs. It's really lossy, but it's the same as OCR or Speech-to-Text -- it is a tool that allows us to better handle large streams of data.

As always, the tool isn't the problem. It's the use of the tool.

(But I think we all know that.)

this2shallPass• 7 years ago

Police are only one group who might use facial-recognition software. Body-worn-cameras can be used and not collect face data or use facial recognition technology, or their legal uses can be severely restricted and clearly defined (or outright banned, lots of options). Why conflate all these things. They overlap sometimes, not all the time.

America is turning against facial-recognition software

Related Stories