Algorithmic Justice Could Clear Convictions in California

https://www.artificiallawyer.com/2019/02/26/algorithmic-justice-could-clear-250000-convictions-in-california/

144 124 7 years ago

maitland• 7 years ago

The Digital Regime of Truth: From the Algorithmic Governmentality to a New Rule of Law - https://www.iainmaitland.com/pdf/Rouvroy-Stiegler.pdf

anton_tarasenko• 7 years ago

"Algorithmic justice" reminded me of a study where researchers predicted the risk of a crime better than judges:[1]

Millions of times each year, judges must decide where defendants will await trial—at home or in jail. By law, this decision hinges on the judge’s prediction of what the defendant would do if released. This is a promising machine learning application because it is a concrete prediction task for which there is a large volume of data available. Yet comparing the algorithm to the judge proves complicated. First, the data are themselves generated by prior judge decisions. We only observe crime outcomes for released defendants, not for those judges detained. This makes it hard to evaluate counterfactual decision rules based on algorithmic predictions. Second, judges may have a broader set of preferences than the single variable that the algorithm focuses on; for instance, judges may care about racial inequities or about specific crimes (such as violent crimes) rather than just overall crime risk. We deal with these problems using different econometric strategies, such as quasi-random assignment of cases to judges. Even accounting for these concerns, our results suggest potentially large welfare gains: a policy simulation shows crime can be reduced by up to 24.8% with no change in jailing rates, or jail populations can be reduced by 42.0% with no increase in crime rates.

[1] https://www.cs.cornell.edu/home/kleinber/w23180.pdf

_v7gu2• 7 years ago

Until the populace learns how to improve their chances of getting released and starts to game the system, introducing endogeneity.

(Also noticed the nice coincidence of a professor with user name klienber having a NBER Working Paper)

dagw• 7 years ago

Until the populace learns how to improve their chances of getting released

As long as that correlates with behavior we want to see from the populace; https://xkcd.com/810/

candiodari3• 7 years ago

Which is totally not possible with judges, right ?

I feel like a lot of arguments being made here fail the A vs B test. Any argument that purports to provide help with choosing Judges vs Algorithms needs to apply differently to Judges, and differently to Algorithms.

How about: with Judges we simply won't know (for sure) what influences them. Are they racist ? Who knows. Do they prefere to let people with jobs out (realistically: yes, but we don't know for sure). Do they ...

With algorithm we can literally test, by presenting them with artificial cases, lots of them, and see how they judge. With a judge, you can't.

gbrown• 7 years ago

The authors note that judges may care explicitly about racial bias, but based on a quick read they're making a really, really big mistake in the language they're using: they confuse arrests with crime. Arrests and convictions are simply a measurement mechanism for crime, which is known to have severe biases.

reader50002• 7 years ago

There is no convincing evidence that arrest rates "severely" overestimate offense rates; if anything it is just as likely arrest rates underestimate offense rates.

lamy20002• 7 years ago

Kleinberg uses arrests for violent crime because they are known to have substantially less bias/zero bias

will_brown• 7 years ago

By law, this decision hinges on the judge’s prediction of what the defendant would do if released.

That isn’t the legal standard for release pending trial.

Thus is may look like an algorithm “predicts” crime better than judges, but judges can’t withhold bail/bond because they “predict” a certain defendant will commit another crime (because that’s not exactly judges are doing when determining bond).

atemerev• 7 years ago

Judging... 97% complete.

lordnacho• 7 years ago

What's remarkable is the article uses the word "algorithm" while stripping it of its current near-magical connotation. Bravo.

worldsayshi• 7 years ago

https://www.codeforamerica.org/

Does anyone know of good examples of such initiatives in the EU?

poloniculmov• 7 years ago

See https://codeforall.org/

shireboy• 7 years ago

Maybe it’s naive, but I’ve often thought law itself should be spelled out in code. Obviously some things are human judgement. But take those as inputs into an unbiased machine that outputs fair sentencing, procurement etc. Still let meatbags review it at least for the time being. It could be a guidance system for judges and juries at least at first. Seems like it could help clear cases quicker and provide fairness.

wallace_f• 7 years ago

I dont know if law can be reduced to code, and it should ideally be intelligible and accessible to the average person, it seems?

Reminds me of this video, where a physicist argues math is not the language of the universe because it cannot convey meaning about many human concepts.(1)

Anyways, it's not like most people, or even apparently police officers, understand even basics about the law right now, anyways.(2)

1 - https://youtu.be/inPcQeYWVT8

2- https://youtu.be/4ap6Kmo69lQ

_red6• 7 years ago

I dont know if law can be reduced to code, and it should ideally be intelligible and accessible to the average person, it seems?

Pick One:

(a) There is only black and white - Laws are simple

(b) There are many shades of grey - Laws are complex

You can't have:

beefield2• 7 years ago

Yes, it is naive. All code has bugs. Some time ago some people though it would be good idea to encode contractual agreements with code. I guess it should be no sutprise that someone found a bug in the contract and got him/herself most of the money that was tied into the contract framework. If I recall correctly, we are talking about tens if not hundreds of millions of dollars of involuntary bug bounty in this particular case (google DAO hack of you have not heard)

Further, in my opinion there does not seem to be any solid logic what me and other humans think that is right or wrong, which makes it quite difficult to spell out preferred laws into code. (If someone disagrees, I am happy to hear one counterexample of moral axiom that holds always, without any exceptions whatsoever, realistic or unrealistic. Even for one single person)

0xBA5ED• 7 years ago

Loopholes are essentially bugs, and they're common. Is there a reason to believe another way of encoding laws would necessarily be more bug-prone than it currently is?

pault3• 7 years ago

I'm not saying you're right or wrong, but the hard fork after the DAO hack returned all of the money that was stolen.

candiodari3• 7 years ago

This happens with real contracts too. Even been in government contracting ? Same with large companies.

denzil_correa2• 7 years ago

Maybe it’s naive, but I’ve often thought law itself should be spelled out in code

Any formal system can either be consistent or complete - not both. Therefore, you will always need an oversight to check for most individual legal cases.

naasking6• 7 years ago

Your therefore doesn't follow. You haven't established that the law would even need to form a "complete" logic, which means it must be able to encode both addition and multiplication. There is little reason to think this is the case.

Furthermore, oversight wouldn't necessarily yield any further insight even if it were. Humans aren't extra-logical, we have similar limitations.

andrepd2• 7 years ago

That makes no sense. You're trying to apply a mathematical theorem to a real-world situation on the basis of some very thin analogy.

rayiner2• 7 years ago

Law as code solves a problem nobody has: application of unambiguous law to unambiguous facts. Even the simplest cases involve law that is unambiguous by design: what is a “unreasonable” search for fourth amendment purposes? All cases involve ambiguous facts. Was the eye witness in a position to see the murder? Is a witness lying? Every criminal charge and many civil claims involve an intent element. What was the defendant’s intent? Litigation is almost entirely concerned with these things, and codifying law addresses none of it.

It’s like applying code to coding. Yeah you can have IDEs that generate skeleton classes. Does that actually solve a problem anyone has, or create a real productivity boost compared to just using Emacs?

hobbyjogger• 7 years ago

Great comment. Here's a good page from CodeX: The Stanford Center for Legal Informatics[1].

"One technical problem with Computational Law, familiar to many individual with legal training, is due to the open texture of laws. Consider a municipal regulation stating "No vehicles in the park". On first blush this is fine, but it is really quite problematic. Just what constitutes a vehicle? Is a bicycle a vehicle? What about a skateboard? How about roller skates? What about a baby stroller? A horse? A repair vehicle? For that matter, what is the park? At what altitude does it end? If a helicopter hovers at 10 feet, is that a violation? What if it flies over at 100 feet?
The resolution of this problem is to limit the application of Computational Law to those cases where such issues can be externalized or marginalized. We allow human users to make judgments about such open texture concepts in entering data or we avoid regulatory applications where such concepts abound.
A different sort of challenge to Computational Law stems from the fact that not all legal reasoning is deductive. Edwina Rissland [Rissland et al.] notes that, "Law is not a matter of simply applying rules to facts via modus ponens"; and, when regarding the broad application of AI techniques to law, this is certainly true. The rules that apply to a real-world situation, as well as even the facts themselves, may be open to interpretation, and many legal decisions are made through case-based reasoning, bypassing explicit reasoning about laws and statutes. The general problem of open texture when interpreting rules, along with the parallel problem of running out of rules to apply when resolving terms, presents significant obstacles to implementable automated rule-based reasoning."

[1] https://law.stanford.edu/2016/01/13/michael-genesereths-comp...

watwut• 7 years ago

While I generally agree, yes ide with reasonable code generation is better then just using Emacs, unless you packed emacs with so many plugins that you turned it into yet another ide.

qwerty4561273• 7 years ago

Algorithmic justice can only be considered justice if the laws are scientifically proven to be just.

jobigoud• 7 years ago

Is that different from Human-Judge justice?

qwerty4561273• 7 years ago

Human-Judge justice is judgement and human judges are supposed to have moral authority to judge. Algorithms hardly are.

dominicr• 7 years ago

Also: Algorithmic justice can only be considered justice if the training data are proven to be just.

realradicalwash• 7 years ago

imv, "just" good training data could definitely make things more just.

that is, if you clean your training data properly and adjust for correlates of injustice, e.g. race. I could imagine two ways (I am sure there are more): 1) you only train it on data from one sub-population, e.g. whites / wealthy people / etc.. 2) you carefully correct wrong decisions and biases in your training data and don't even train on such factors as race.

and then, in applying the model, you don't specify those correlates, either.

this could be a small revolution.

YorkshireSeason• 7 years ago

Naturally, there is no agreement of what exactly justice means. A pragmatic definition of justice that I found rather helpful for navigating the world: a society is just if it gives me (my in-group, my tribe) somewhat more than the others.

That said, and generalising from the success of formalisation of other previously informal domains (e.g formalisation of fair elections lead to insights like Arrow's impossibility theorem), I expect that formalising concepts of justice will lead to major progress.

pault3• 7 years ago

I think the ambiguity with justice comes from the ambiguity of what exactly a crime is. What is or isn't a crime is largely a product of the society that defines it.

mschuster913• 7 years ago

A big part to help would be forbidding employers, landlords and banks to access or use conviction databases, with limited exceptions for security-sensitive employment such as banks, security guards, childcare/education and the likes.

Or, simply, do not make them "public" in the sense of "putting it on the Internet". By all means, make convictions and court documents public in the sense that one can go to the local library and do research there in person (to provide a natural scaling limit), but it should not be acceptable that there are "data mining" companies that get the name searches on Google polluted by conviction records or court documents as sensitive as one's financial worth during / after a divorce judgement process and then charge people extortion fees to "remove" the records from their site (only to reappear on another site, rinse and repeat).

In Germany, this is the norm. Drug testing by employers or checking their credit score is also not allowed, with highly limited exceptions. As a result, while we do have a problem with convicts being discriminated against after release, it doesn't even come close to the level of problems the US has.

Nasrudith2• 7 years ago

There is a fundamental flaw with the approach - the right to remember and the right to print/speak. You can keep a newspaper in your house. Why then is it permissible to stop someone from archiving the facts? Even if later proven false it is still useful data that say the New York Times featured an angry Trump full page editorial calling for the death penalty for them?

I don't disagree that undue judgement is an issue for rehabilitation but I have heavy doubts that forced amnesia is a good idea - especially given the repeat scammers who would exploit it.

mschuster913• 7 years ago

Why then is it permissible to stop someone from archiving the facts?

The "publication" laws were written in a different time - newspapers have, simply because they're physical objects, not searchable at the vast scale and speed that a quick Google search allows now. The laws simply have to be modernized, that's it.

especially given the repeat scammers who would exploit it.

These can be sued until they burn down to the ground. Making money off of people's misery in that way is as despicable as it gets in a civilized society.

astura2• 7 years ago

There's an absolutely huge distinction between "try to rewrite history so [thing] doesn't exist anymore" and "law preventing you from taking [thing] into account when making hiring/renting/whatever decisions."

It's not impossible, insurance companies have no problem with this, the insurance industry is highly regulated about specific things they can/can't consider while underwriting a policy, which varies by jurisdiction. They just don't ask about/look up factors that they aren't allowed to take into consideration while underwriting a policy, but aggressively ask about/look up factors they are allowed to take into consideration.

For example, in Massachusetts insurance companies are forbidden from considering credits score when setting premiums and making underwriting decisions, a practice which is extremely common in the industry where legal (people with high credit scores have less insurance claims). That doesn't mean Massachusetts is trying to rewrite history so that your poor credit doesn't exist, it still exists, and others are allowed to use it for other reasons (such as underwriting loans).

Saying "we forbid you to take into consideration conviction history when hiring" isn't anywhere near "forced amnesia."

jrochkind13• 7 years ago

Why is this a big deal? It matters because algorithmic justice, of a complex or simple variety, has come in for a lot of flack in the US, in part because of one truly major mess that was caused by the COMPAS system. So, it’s great to see algorithmic justice doing good.

That's because these don't seem like the same things at all to me. Lumping them both under "algorithmic justice" with an agenda to make the one look better doesn't change that. This thing in this story involves zero machine-learning, zero statistics at all, zero attempts at statistical prediction of likelyhoods of anything, or even any heuristics at all. Those are all the things that have come for a lot of flack. Looking things up in databases and using them to automatically fill out forms, indeed, has not come under a "lot of flack."

lallysingh• 7 years ago

The public doesn't care about those distinctions. So it's useful to cancel out a negative gut-reaction to encourage a more nuanced view.

rayiner2• 7 years ago

This isn’t algorithmic justice, just regular automation. So it’s dishonest to suggest it should cancel out the problems that have been created by algorithmic justice.

bilbo0s• 7 years ago

I don't know man?

Just Devil's Advocate here, but it I think if Trump did that, and then tried to say he was just trying to encourage a "nuanced" view, we'd all yell, "See! Politicians are always lying!"

hnaccy• 7 years ago

That seems manipulative.

jrochkind13• 7 years ago

You think the public is going to be opposed to _this_ sort of thing, because of what they've heard about the _actual_ statistical algorithmic things?

I seriously doubt it. I think the author was trying to do exactly the reverse, get people to _support_ the statistical algorithmic things because of this thing, which seems obviously a good idea.

It seems like (exagerrated of course) saying "Yeah, I know you don't like autonomous military robots killing people, but you like YOUR TOASTER, right? It also works on electric power, and turns itself off unattended when your toast is done. Maybe there's a role for Autonomous Killing Robots after all, they might even be able to toast your bread with their lasers!"

Most of the public has never even heard the term "algorithmic justice", of course. You're saying "the public doesn't care about those distinctions," but it's the OP author _suggesting_ we include software providing _decision-making_ based on statistical algorithms, along with simple rule-based automation, both under "algorithmic justice", _rather_ than making a distinction. The OP author encouraging us not to make distinctions. I think the public is smart enough to distinguish between software that is _making decisions_ and software that isn't, and it's the OP that is asking this distinction not be made. As the public is indeed just starting to learn how to think about and categorize these things, I think it's irresponsible to try to educate them _not_ to make distinctions, and I think it's being done for an agenda.

yarrel• 7 years ago

It might be, but using the benefits of a very different thing to launder the reputation of something that people have very legitimate concerns about isn't nuance - it's dishonesty.

crb002• 7 years ago

This issue is transparency. Iowa's "algorithmic" bail program has zero transparency. They have even violated Iowa's public records law and refused to let me inspect the software.

Shivetya• 7 years ago

what can be used to crawl through defendant's records to exonerate them can also be used for other purposes so care will always need to be taken with expanding processes such as this.

that out of the way, long term it would be best to standardize how this information is stored, categorized, and shared, so that exoneration in one locality can be more easily implemented in another.

dsfyu404ed3• 7 years ago

This technology has long been used to dig through the past of anyone and everyone that comes in contact with the legal system. When you get pulled over for a taillight out and the cop runs your driver's license your pot charge from the 80s comes up.

snidane• 7 years ago

Algorithmic justice is one of those problems where techies think that by applying algorithms and techniques du jour will somehow solve the problem. Unfortunately law is so complex nowadays that it is ridden with internal contradictions. Formalizing law would only reveal these. To change the system, a massive reduction of complexity of law would be required before any math could be applied on it with some success. But then if we simplified the law to make it comprehensible to mortals again, who would need any algorithms at all?

Relevant quote from Tony Hoare. s/software design/law/

There are two ways of constructing a software design: One way is to make it so simple that there are obviously no deficiencies, and the other way is to make it so complicated that there are no obvious deficiencies.

naasking6• 7 years ago

Formalizing law would only reveal these.

Which would identify easy targets for lawsuits against the government and cases which would overturn convictions and lead to the massive reduction in complexity that you seek.

hueving• 7 years ago

But then if we simplified the law to make it comprehensible to mortals again, who would need any algorithms at all?

Because automating simple systems is what software is great at. Why wouldn't you?

dsfyu404ed3• 7 years ago

Because it is likely to enable us to go further us down the path to 1984.

Fucking with people's lives is something that should be done by a human that can see the bigger picture, not a bunch of code daisy chained together.

Nasrudith2• 7 years ago

I don't think it is techies at this point - more hanger-ons marketers given the complete lack of requisite understanding or interest in it and pushing a solution based on how good it sounds instead of how well it works.

Even by the standards of machine learning which has often been guilty of following spurious correlations. Anyone who actually worked with it would laugh like an artilleryman hearing Aristotole's impetus theory because it is so obviously disconnected from reality.

I have cynically theorized it is deliberate "bias laundering" ever since I heard of a defacto classist British algorithm which included /garden size/ (yard in American sense).

There is literally no legitimate reason to include it in a reoffense risk as a variable. Even if they thought it somehow had a rehabilitative effect then ownership shouldn't be a variable when they can just turn them into upstanding citizens through yardwork!

NoblePublius• 7 years ago

This is lovely until the same system and precedents are used in reverse (algo district attorney) or by banks to automate the foreclosure and seizure of thousands of homes. The banks did it before!

yboris• 7 years ago

Even simple linear models can outperform "experts" in numerous fields, especially in those where the feedback loop (for learning) is very long (as is the case in the criminal justice system).

An amazing paper that touches on this topic: In Praise of Epistemic Irresponsibility: How lazy and ignorant can you be? by Michael A. Bishop

You can find the PDF easily. Here's a ref: https://link.springer.com/article/10.1023/A:1005228312224

sametmax• 7 years ago

Algorithmic justice, the one thing that is worst that electronic voting. I didn't think it would be possible.

pnw_hazor6• 7 years ago

OCR + Data mining with automatic form generation.

It works here because the state is willing to accept the data/results at face value.

Kind of a stretch to call it algorithmic justice.

bertil2• 7 years ago

I would see it as a stretch to call it “machine learning” justice (ignoring the OCR) but it is automated with an (albeit simple) rule -- and that process is an algorithm.

Like for all machine-learning project, I would recommend ‘Crawl before you run’ and have that project encourage the Justice system to modernise and have basic data-handling processes in place. We’ll get to Orwellian pre-cog soon enough. Right now, let’s get basic things like not arresting homonyms or people who have committed acts that are not crimes anymore.

pnw_hazor6• 7 years ago

We could call it:

Filtration Justice

Greater than Justice

if time_since_last_conviction > n years AND past_crimes_committed NOT IN [some list of crimes] { generate form and email to prosecutor. }

The OCR is the most important part.

edit-to-add:

some misdemeanor convictions are barred from sealing/expungement. It is jurisdiction dependent but they usually are crimes, such as, DUIs, domestic violence assaults, sex crimes, and so on.

staticautomatic• 7 years ago

I really hope the term "Algorithmic Justice" doesn't catch on. It reminds me of the old saying that military justice is to justice as military music is to music.

anilakar• 7 years ago

Let me get this straight: Are there people who have been wrongly convicted and are currently locked up only because someone hasn't done their paperwork?

empath75• 7 years ago

There are tons of people in jail that shouldn't be in jail, for all kinds of reasons.

twic• 7 years ago

So far they have reviewed 43 years of eligible convictions, proactively dismissing and sealing 3,038 marijuana misdemeanours and reviewing, and recalling and re-sentencing up to 4,940 other felony marijuana convictions which were sentenced prior to Proposition 64’s passage in November 2016.

This sounds to me like it's quashing convictions for marijuana-related activity which is no longer a crime, and where sentences have already been served. So the convicts are no longer in prison, but they have a criminal record, which this project will wipe clean.

In a post from last year [1], a lawyer writes:

But here in California, there’s hardly anybody in prison for marijuana anymore because of the reforms in our system that have happened over the last decade.

[1] https://melmagazine.com/en-us/story/if-cannabis-becomes-lega...

bertil2• 7 years ago

“Has done the paperwork” is a vague concept but, yes, there are a lot of people who are in jail for crimes that do not exist anymore.

There are more people who would be in if someone cared enough to raise key exonerating circumstances, judicial practices change since their condemnation. It’s less “straightforward” but it’s a lot less controversial than you would think.

This is why people who can read law books are so valuable in prison.

crimsonalucard• 7 years ago

So does this algorithm generate a report that predicts the future? Like a minority report?

Razengan3• 7 years ago

Lawyers and doctors are probably the best candidates for replacement by AI, and those professions will lobby the hardest against their replacement, but ultimately, I would rather be governed by transparent, open-source, AI than humans.

ducttapecrown• 7 years ago

I have the impression that machine learning is incredibly not transparent. Maybe I'm conflating machine learning with AI. Could you explain your vision of transparent AI a little more?

Razengan3• 7 years ago

i.e. as long as it exposes all the facts/criteria it uses to arrive at each decision/verdict or reallocation of resources/manpower.

Of course, every rule or law would have to be initialized by humans in the first place.

Simplified example: "Murder is bad.", but after the AI observes enough cases where the victim attacked the killer first, it may present "But not when in self-defense." as an amendment, along with its reasoning, which again would have to be democratically approved by humans for it become a default condition.

It could even retroactively apply such new rules to all past cases, better than a human-run system, fixing wrongful convictions etc.

lurquer• 7 years ago

You may be right about doctors. Their subject matter is repairing or maintaining, essentially, a mechanism.

Lawyers are different. Their subject matter is the interaction between human beings. A lawyer, for instance, must persuade a judge or jury or, in the transactional setting, the opposing party. So long as humans are governed by humans, there will be a role for lawyers.

hippich• 7 years ago

Except sufficiently complex trained AI no longer transparent. (Open source part relates only to code running network, but network's decisions are much more opaque)

pnw_hazor6• 7 years ago

If you knew what lawyers do for a living you would probably realize that lawyers are probably one of the worst candidates for replacement by AI.

While a variety of smart or dumb machines are used today to assist lawyers, unless you are talking about Sci-Fi level AI, I can't see lawyers being replaced by AI. Okay, maybe a number of lawyers whose jobs are mostly secretarial in nature may be replaced by smarter tools, but not lawyers who counsel or advocate for clients.

It would only work in some kind of Sci-Fi dystopia where the legal system would be unrecognizable to those used to an adversarial based common law system, or the like.

NateEag2• 7 years ago

Speaking as one who was raised by a physician, I cower in horror at the thought of having a treatment plan designed and implemented by a neural network without human intervention.

We've seen that for a number of difficult problems, deep neural networks, given sufficient data and expert trainers, do a good job most of the time and occasionally come to mortifyingly wrong conclusions.

Run a 'sed s/mortifying/mortal/' on that previous sentence and think about it for a minute.

Razengan3• 7 years ago

Humans also make mistakes that are fatal for others, and we have been thinking about that for millennia.

Humans also subvert things in order to make more money.

Maybe not complete replacement, but where AI does most of the thinking, shows "This is what I found, and this is what I used, to decide this." and humans have the final approval on whether to go ahead with whatever it decided.

If someone has a disagreement, it would be submitted to the AI and make it re-evaluate everything while weighing it against the disagreement.

tareqak• 7 years ago

Here is the APNews article about dropping the pot convictions: https://apnews.com/1aeb5fed9e8746d8b120049e908af06c . They don't use the phrase "algorithmic justice", just technology.

KorematsuFred2• 7 years ago

Remember the time Google Image search started mistaking black people for Monkeys ? I am worried about Algorithmic judges for the same reason that minorities might get into trouble here.

KorematsuFred2• 7 years ago

I think I would prefer a system where Algorithms decide not to charge a person. Only those to be charged go to a real court.

Also, all plea bargains must be algorithmic so that narcissist DAs can't browbeat ordinary people into accepting guilt.

Algorithmic Justice Could Clear Convictions in California

Related Stories