How the Police Use AI to Track and Identify You

https://thegradient.pub/how-the-police-use-ai-to-track-and-identify-you/

by kungfudoi • 6 years ago

217 107 6 years ago

andreyk• 6 years ago

Don't think it's cited in this piece, there is also this recent piece of news - "Controversial facial-recognition software used 30,000 times by LAPD in last decade, records show"

https://www.latimes.com/california/story/2020-09-21/lapd-con...

I guess it's reasonable for police to use this technology, but not without more transparency and accountability... Feels like this year that's becoming very clear.

moate3• 6 years ago

>I guess it's reasonable for police to use this technology...

Depends on your interpretation on the 4th amendment. I personally feel it's extremely unreasonable.

monocasa2• 6 years ago

You're not the only one. If you read the opinions in United States v. Jones, the supreme court makes a pretty big distinction between the kinds of surveillance a police officer could reasonably do on their own, versus the scaling up that technology allows, and how that might kick otherwise allowed surveillance without a warrant to requiring a warrant just about the sheer amount of information being acquired. And that's even with data that could be available from watching from public streets, if there were enough officers to watch everyone.

stanrivers• 6 years ago

China perfected this all for us and now the U.S. government can piggyback off of their practices... great.

Serious note - the below is such a scary concept. There is a line somewhere and this feels like you are starting to cross it... both place-based and person-based predictive policing sound ripe for profiling / instigating actions that police can use as a reason to arrest an otherwise harmless person.

"Predictive policing programs are another illustrative example showing how data, surveillance technology, and a system of automated policing work together to spy on, search, and, ultimately, control Americans who have not committed or been convicted of a crime.

Predictive policing is premised on the idea that historical data of crime, demographics, socioeconomics, and geography can be used to forecast future incidents. Knowing where crime is likely to occur again, police try to intervene beforehand and prevent it.

Broadly there are two kinds of “heat maps” produced by predictive policing models: place-based, which uses less data to try to avoid systemic pitfalls of relying on crime and demographic data and surges police into specific areas, and person-based, which tracks and creates a list of “high-risk” individuals by combining a person's criminal history with an analysis of their social network. "

jancsika2• 6 years ago

person-based, which tracks and creates a list of “high-risk” individuals by combining a person's criminal history with an analysis of their social network.

There is no way the software actually achieves this. If it did we would have already heard about a well-off target identified by the software, a target with the means to mount a loud defense against such tracking.

And it cannot be that the software accurately returns such results but LE simply ignores them if the target seems to be too powerful. In that case we would have already heard from a leaker about a murder from a perpetrator that such a system correctly identified but LE neglected to follow up on. (Think of the value to the company of such evidence!)

No, I'm guessing this system gives LE cover for the same flawed systems they've historically used to target and harass people who don't have the means to loudly mount a defense. Just take the same process that a court has ruled against as unconstitutional, use its data as input to your ML algo, and voila! You've probably got at at least one more decade before society figures out you're just appending the words "through a sophisticated AI system" to your outlawed policing.

moate3• 6 years ago

>There is no way the software actually achieves this. If it did we would have already heard about a well-off target identified by the software, a target with the means to mount a loud defense against such tracking.

I mean it's more likely that the system is going to informing you about likely targets for certain crime types (less murder which is frequently singular and more drug dealing since repeated). It's going to help create lists of suspects to pursue on a matter. If one of them is also a senator's son or a CEO, you'd likely go in softer than Joe Poorperson

monocasa2• 6 years ago

Why would we have heard about any of the failures? Part of the 'blue wall' is intended to hide policing failures from the general populace. Particularly when

* in the case of targeting a well off person who they decide to further investigate, they can simply use it as reason to look harder at someone, use parallel construction to build the case that the defense sees, and simply drop any cases where the defense happens to get too close. This is what they do with stingrays.

* in the case of targeting a well off person who they decide not to investigate, they can simply tell themselves that the person is a paragon of society and it must be one of the failures of the technology. Like when a boy escaped Jeffrey Dahmer's house heavily drugged, running down the street, bleeding out of his anus, begging anyone who would listen to not let Jeffrey take him back, the police got involved and gave him back since Jeffrey was such a paragon of the town. That cop later became head of the police union.

heavyset_go2• 6 years ago

I'm guessing this system gives LE cover for the same flawed systems they've historically used to target and harass people who don't have the means to loudly mount a defense.

This is exactly what happens. If we look at the evidence[1], law enforcement preys upon the poor and minorities who are not even committing crimes, but their predictive policing systems target them for harassment anyway.

[1] https://projects.tampabay.com/projects/2020/investigations/p...

Jon_Lowtek7• 6 years ago

Such software exists, it is as bad as you think it is and it produced well off targets that loudly complained. See the nsa skynet program that used map reduce on pakistans telecommunication metadata to identify possible terrorists and promptly identified Al Jazeera as a terror cell. As you correctly guessed LE did not go after them and did not bomb their office because, luckily, someone cross checked the automatons results. And then we did hear about it from a leaker.

There is far less known about such systems in police and fbi but what i read so far about the fusion centers that is, if not already achieved, at least a goal.

WarOnPrivacy• 6 years ago

With rare exceptions, police are strongly addictive personalities. There's no bit of power over the public that cops can use in a reliably responsible way.

Surveillance & control are drugs, ones that they are not capable of putting down.

tal8d4• 6 years ago

That is true of almost everybody, not just police. Petty tyrants are a dime a dozen, they just lack the ability to legally kidnap you. People are bad, so we need a government comprised of people are bad, so we... I already hear the cries of "But who will build the roads?!"

srtjstjsj6• 6 years ago

So we come together as a democratic society to hold each other accountable.

Animats2• 6 years ago

Can this be used against employers to detect wage theft? "Time shaving" should be easy to detect. If cell phone tracking, W-2 payments, and employer time records are in substantial disagreement, wage theft has been found.

moate3• 6 years ago

By whom? The government has shown time and time again they aren't interested in policing attacks against labor (see: the toothlessness of the NLRB, the gig economy, weakening of unions/organizing, etc) so it wouldn't be them.

The companies aren't going to do it because LOL wut?

So how are you going to get labor to do that? And if they do, who do they take it to (because again, NLRB is underfunded and relatively weak)

Animats2• 6 years ago

We'll have to see how pro-labor the next administration is.

Labor law enforcement could be run as a profit center, like drug enforcement, if the fines were higher.

klenwell• 6 years ago

I initially read this the other way: used by employers to detect wage theft.

Supervisor Drone: You are allotted two government-mandated 15-minute breaks a day, but our third-party right-to-work service detected you away from your work sector for an average of 16.8 minutes per break period over your past 3 shifts. You are hereby terminated and liable for service fees you agreed to as part of your original employment agreement.

aaron6952• 6 years ago

I know HN isn't very technical so this might be hard to understand, but this isn't AI.

It's just computer power.

All this was possible 40 years ago but it cost orders of magnitude more.

notsuoh• 6 years ago

It's true that AI is a bit of an overloaded term these days, but in the modern usage of AI you have all types of machine learning and neural networks, which this certainly is. The AI you're thinking of is usually referred to as AGI. But all that aside, in the end your comment is still a little off the mark because what is any flavor of AI but computer power.

aaron6952• 6 years ago

No one is using neural nets or machine learning I can see here.

AI is just computer power, hard disk space, big databases, fast networks, cheap cameras etc sums up the 'AI' community though.

Letting cheap hardware make it look like they can do something. I will agree to that.

geraldkleber11• 6 years ago

This is why we built Trix (trix.co).

It's a consumer-facing photo editing app that uses adversarial AI to manipulate your photos in such a way that companies like Clearview AI can't train facial recognition algorithms off your data. You can download the Android or iOS version at trix.co - we're in beta right now.

Would love to get anyone's feedback!

chance_state• 6 years ago

Do you have any data on how effective this technique is?

geraldkleber11• 6 years ago

Not yet - much of this research came out of U Chicago this summer and we just launched our beta. You can download at trix.co (iOS and Android).

polishdude202• 6 years ago

Hmm trying to download it on Android, I click the link and it starts to take me to google play but it just loads forever. Never gets to the page to download it

geraldkleber11• 6 years ago

Hmm, lemme do some bug squashing. Will reply when it's dealt with!

samename• 6 years ago

“We require that you sign up with a valid US phone number to verify your identity as a human for security purposes.”

How does collecting personal data improve security? Why is identity verification needed when the point of using the service is to avoid automated identification in the first place?

bigbubba• 6 years ago

Such a database would be a tasty treat for any unsavory company looking to purchase them.

_tulpa• 6 years ago

I really don't think you can solve technology with more technology...

geraldkleber11• 6 years ago

In this instance, we're well convinced we can.

godelski• 6 years ago

This is cool, do you have a technical blog post?

geraldkleber11• 6 years ago

No technical blog post yet, but have some faqs up that could help out!

https://www.trix.co/faq

vaccinator• 6 years ago

Sounds like a fight that can never be won

geraldkleber11• 6 years ago

We don't think that's the case!

searchableguy• 6 years ago

What if this works to increase bias against certain group of people because the facial recognition software isn't trained on them?

I am not sure technology can solve this completely.

geraldkleber11• 6 years ago

I very much appreciate that question, and having been in this space for a few years now, it's certainly one that's relevant to the tech as whole, but less so our app.

Our technology is simply indexed to a public data-set of 30k individuals and when our deep learning model scrambles the key-points on your photos to confuse the clearviews of the world it does in a random manner. The model truly is a black box in that way.

srtjstjsj6• 6 years ago

Do you have any evidence that your system works?

"Adversarial AI" doesn't really work against systems like Clearview that aren't using "AI" in the first place.

geraldkleber11• 6 years ago

You should check out the research that was done on this. The technology works - we're simply the first to really productize it in this manner.

https://sandlab.cs.uchicago.edu/fawkes/

Jon_Lowtek7• 6 years ago

Split your Privacy Policy into parts along technological borders like "site, crm, app" instead of one size fits all. Disclose what you are actually doing and restrict yourselve to that instead of trying to have as much wiggle room as possible. This sounds counter-intuitive because web-business standard is to be as consumer unfriendly as legally possible, but the privacy tool market likes higher standards.

After reading your policy and tos i think someone should check the app for facebook scraping, because the legal texts sure imply you are doing that:

"By granting Trix access to any Third-Party Accounts, you understand that Trix may access, .. any information .. that you have provided to and stored in such Third-Party Account (“SNS Content”) ... all SNS Content shall be considered to be your User Content"

Some great advice from the privacy policy:

"You should always review, and if necessary, adjust your privacy settings on third-party websites and services before linking or connecting them to Trix’s websites or Service."

Statements like this are red flags:

"We may collect metadata associated with User Content. Metadata typically consists of how, when, where and by whom a piece of User Content was collected and how that content has been formatted."
"Trix may transfer information that we collect about you, including personal information, to affiliated entities, or to other third parties" ... "for the purpose of providing the Service"

This is a typical blank.

Basically they reserve the right to do as they please with your data and all data they can access through services you link with their service.

toydog5343• 6 years ago

technical cofounder of trix here. we do not currently have social logins for the app to this point doesnt make much sense. but we probably can further modify the templates we used for our tos/ privacy policy to asuage your concerns :)

we absolutely do not (and would not) scrape data from your social media accounts. unfortunately, terms like this are standard for many tech companies today.

ape4• 6 years ago

And why do they scan the demonstrations asking for police accountability - right away that sounds like a conflict of interesting. You never hear about the AI scanning faces at places where actual criminals hang out - like maybe a pawn shop (possibly selling stolen goods).

tal8d4• 6 years ago

Because the "demonstrations" have devolved into riots where "asking" means firebombing. Combine that with local prosecutors refusing to pursue charges, and you've got a situation where the police a highly motivated to identify everybody in black bloc chic.

heavyset_go2• 6 years ago

If you want to see the horrors that result from law enforcement's use of AI and predictive policing, look no further than here[1].

It's several videos taken from body cameras of police officers harassing, assaulting and abducting people because they showed up in their system as being related to, or knowing, people who are suspected, and not convicted, of crimes.

The videos are from a story that made the front page on HN about a month ago[2], if you want more details.

[1] https://projects.tampabay.com/projects/2020/investigations/p...

[2] https://news.ycombinator.com/item?id=24363871

maze-le• 6 years ago

This incredibly disturbing. I grew up with family members telling me horror stories about state abuse like this, from more than 30 years past in a country that doesn't exist anymore: The German Democratic Republic. If this isn't classic Stasi Zersetzungstaktik, I don't know what it is. Aren't there any laws against abuse like this? How the hell is this legal?

kyuudou• 6 years ago

I'm glad you've pointed this out. Comparisons to GDR and the Stasi often get ridiculed for being excessive and hyperbolic when in fact what's currently implemented would've been the Stasi's dream.

If anyone needs a good weekend movie recommendation, Das Leben der Anderen[1] is excellent and quite relevant.

1: https://en.wikipedia.org/wiki/The_Lives_of_Others

srtjstjsj6• 6 years ago

It there any "AI" in the Tampa issue?

Or "predictive"?

The Tampa program is vindictive policing to harass undesirables into leaving town.

bitdestroyer• 6 years ago

Just a note, that's Pasco county which is north of Tampa. I believe you're referring to this story that broke a few weeks ago: https://projects.tampabay.com/projects/2020/investigations/p...

r73ydhdh• 6 years ago

It seems like in both of those cases the real problem was the pre-emptive actions taken by the officers. If someone is predicted to commit a crime it makes radically more sense (as far as avoiding acting on false positives) to surveil them in a clandestine fashion and simply allow them to commit the crime while you record the evidence. An arresting officer can be dispatched later to make the arrest without any need to risk escalating an in-progress crime.

blantonl• 6 years ago

It won't be long before we see more and more escalating instances of the population using traditional electronic and physical warfare techniques against law enforcement in these "battles." I predict we'll see someone in the general public shoot down a collection drone, or start to actively target these physical collection nodes installed on poles, posts, buildings etc.

There were already numerous documented cases of electronic warfare being deployed during protests at law enforcement - the jamming of their public safety 2-way radio systems.

Whenever there is an escalation of techniques against the "enemy" - the enemy fights back. There will be interesting developments in this space the next decade with regard to this. Law enforcement already needs advanced direction finding equipment - which is traditionally a military operation.

Jon_Lowtek7• 6 years ago

Last month, it was revealed the Department of Homeland Security authorized the domestic surveillance of protestors and journalists, training a system usually reserved for hunting terrorists overseas with drones on American citizens exercising their First Amendment rights.

wait wat

edit: the articles linked source doesn't actually say that but instead talks about “Baseball card” dossiers.

Pfhreak7• 6 years ago

This is where policing in America has been headed for at least a decade now. Everything is so polarized that people who would normally be opposed to this view those being tracked as their enemies, and thus fair targets. (For example, I'd imagine that right wing or libertarian types would normally consider surveillance to be a real bad thing. But if it gets deployed against George Floyd protestors... Ehhh... nbd)

When you hear politicians talking about being "tough on crime" or "law and order", this is exactly the program they are pushing for.

srtjstjsj6• 6 years ago

Please don't conflate conservative with libertarian. They are partially similar in economic policy but opposites in social policy.

sudosysgen7• 6 years ago

My History teacher once remarked that a hallmark of imperial collapse is the use of the tools of empire at home.

Gibbon1• 6 years ago

My iron rule of human relations. If you see someone do something shitty to someone else, stick around long enough and they'll do it to you to.

That's why watching the US's power elites callously using the nations resources to screw smaller hapless nation states scares me. Not to mention they keep grudges.

Pfhreak7• 6 years ago

I'm not going to say ACAB, but many who do believe that no matter how nice of an individual a cop might be, they are still working within an organization that pushes for privacy shredding technologies like this.

You can be a sweet, community oriented beat cop out there doing your best, but behind you stands surveillance, militarization of police, and the tragedy of prisons in America.

Maybe this isn't a generalization that's fair -- should/do we fault all FAANG employees because their parent companies are doing unethical things?

I don't know the answer, but articles like this definitely make me think a whole lot more about whether individual police can be ethical while operating in the way the departments do today.

ssklash• 6 years ago

I don't think you can separate the individual from the organization when the organization either fundamentally or in practice is engaged in negative/harmful activities. Policing is broken in the US, and though its original intention is good, the daily practice of it today is systemically flawed and no longer has the original intent as its goal.

The same applies to companies like Facebook and Google. They fundamentally exist to extract data from their users. They never had another legitimate mission that was corrupted over time, unlike the police. Apple, Netflix, Amazon at least have services that do not revolve around taking advantage of their users and manipulating their behavior.

Organizations are made up of individuals at the end of the day. If you are a cop or work at companies like Palantir, Google, Facebook, you are helping prop up organizations that actively do harm to many people. You may not be as responsible as cops that murder or decision makers at the top, but their guilt would not be possible without you. Please think long and hard about where you work and why you work there.

srtjstjsj6• 6 years ago

Are you honestly saying that FB and Google don't provide anything that users value?

chordalkeyboard5• 6 years ago

I agree with much of what you have said but you appear to be lumping all policing organizations together on the basis of some recent high-profile events. Many cops work in local departments serving communities across the US and we simply do not have data on how prolific (or not) these abuses are, nor do we have data on the natural background rate of human-on-human abuses with which to compare the current level of abuses with a hypothetical alternative (such as mafia rule).

This is akin to saying that all programmers are evil because of the actions of google et al.

kortilla6• 6 years ago

By this poisoned well logic, you are responsible for any increase in crime that occurs from a reduction in policing. Adults that work for an organization which is critical to every civilization to date (law enforcement) work to fix it from the inside. Only immature people take the unethical stance that there should be no law enforcement because there are flaws in the current structure. Even in Minneapolis, the only thing the council can agree on is that “defunding the police” doesn’t actually mean taking away the police funding. Only a small anarchist percentage of the population wants nobody to actually be in the police.

the daily practice of it today is systemically flawed and no longer has the original intent as its goal.

Care to say what the goal of the Boulder, CO police department is? How about the one for Waco? If the goal is not law enforcement, how does that square with the fact that people keep electing the heads of these orgs (Sheriff, Mayor, etc)?

kortilla6• 6 years ago

but behind you stands surveillance, militarization of police, and the tragedy of prisons in America.

Prisons are not related to police anymore than the citizens who vote the legislators in who pass the laws and the judges who sentence to prison for violating said laws. As a cop (beat or all of the way up to chief of police) you have no say over private vs public prisons, prison policies, or even prison sentencing because they are not the same organization.

Moreover, surveillance and the militarization of police are both policies of local police departments. You can make a completely ethical decision to join the sheriff’s department in a local town that has none of these capabilities nor wants them. The law enforcement of a locality is strictly a result of that city’s politics. There is no national police force nor standard in the US that makes sense to hold all enforcement officers accountable for.

Pfhreak7• 6 years ago

Prisons are absolutely related to police. The only way I'm aware of to get into prison is via an arrest by police.

So, every arrest a police officer makes, they need to consider, "knowing what I do about prison, is it ethical to arrest this person?"

Police militarization is supported at the federal level through grants of military equipment, and nationwide through the use of trainings that teach officers they are "warriors" or "sheep dogs".

And yes, police are fragmented from a logistical standpoint, but culturally they are very similar. Police unions make holding bad cops accountable nearly everywhere, prosecutors refuse to prosecute bad police because their jobs require a good relationship with police, the idea of the "thin blue line" permeates every police dept, etc. They might all have different funding and standards, but there are systemic nationwide problems.

avs733• 6 years ago

I'm usually not the 'reduce it to math' person but this seems like a perfect example. Facial recognition in policing fails at a mathematical level because of Bayes.

Let's say I have a database of 10,000 people's faces and I'm looking to match 1 accused criminal from a good quality image. Let's also say my algorithm is 99.99% accurate with 100% specificity (i.e., I always ID the bad guy as the bad guy).

So I get a true positive, I can correctly ID the guy. But in doing so I am 10 times as likely to get a false positive. at 99.99% accuracy. Scale it to 1,000,000 people and that likelihood of a false positive is 100x. Because you are looking for a small number of people as your database gets larger the value of a positive match (i.e., your positive predictive value) rapidly approaches zero.

Yes this is a simplification but the basic idea stands. These algorithms used at scale are flat out unacceptable because the concept of their use makes false positives more likely than true positives.

gorgoiler2• 6 years ago

Something feels a bit off about this article.

The content, the name of the website, the logo they use — all make me initially trust them as my kind of people. I want to read this without questioning who the author is. It has the look and feel of something I might align with, instinctively.

Alas, this is a dangerous thing in the world of bogus news.

The author and his editors then refer to Founder Fathers (not Founding) and “tying” police powers (instead of “tying the hands of the police” or “curtailing the powers of the police”.) People learning English as a second language are, of course, allowed to make mistakes, but written journalism about US public policy isn’t. That’s the point where one stops and does due diligence on who these people are. This is hard, because the The Gradient’s team don’t really introduce themselves very clearly.

It would be great to have just a little bit more information about what this site is before delving into this article more. The about page suggests it’s a student/college magazine:

https://thegradient.pub/about/

Somewhere between the end of the article and the bottom of the page is a mini bio on the author. They work in policy / politics, including working in DC. They also have a public bio on LinkedIn with more details. I don’t mean to cast unfair aspersions, but am I reading a nudge piece from a lobbying think tank?

I feel guilty for even asking. How did the internet make me so skeptical?

sudosysgen7• 6 years ago

Why wouldn't such a deeply political article be written by someone in politics, given the context certainly in consultance with people from the field?

And which kind of lobbying think tank is against giving more power to the police?

gorgoiler2• 6 years ago

I’m fine with think tank nudge pieces as long as they aren’t pretending to be news editorial.

On today’s internet, there is a strong requirement for being up front out one’s agenda otherwise how do we differentiate interesting opinion/blogging from the fake news bunkers?

In this case, a buried bio on a think piece I’ve never heard of wasn’t enough to suppress the “is this manipulative blogging dressed up to look like reputable journalism?” alarm bells.

(Even though, in fairness, this author does seem to be closer to the Bob Woodward end of the spectrum than the Ghanaian/Russian troll farm end.)

kortilla6• 6 years ago

And which kind of lobbying think tank is against giving more power to the police?

Any on the left in the US right now. Police enforcement has become a huge political issue for the November election. Anything to drive up more support for the notion that LE is corrupt/fundamentally broken is good for those aligned with a D sweep. This is not cynical, it’s just how the game is played in the lobbying/“think tank” world.

known• 6 years ago

AI is not immune to Garbage In = Garbage Out

How the Police Use AI to Track and Identify You

Related Stories