Study finds Wikipedia influences judicial behavior

https://news.mit.edu/2022/study-finds-wikipedia-influences-judicial-behavior-0727

by drewem • 4 years ago

209 122 4 years ago

news.mit.edu

czl_my• 4 years ago

Related discussion: https://news.ycombinator.com/item?id=32253053

This is why in itself Wikipedia cannot be a trusted source but should be treated as purely, an extension of the media. They choose who they can cite and who they cannot.

nehalem• 4 years ago

Could you elaborate? I do see the issue about reliability, but I don’t see how this study is related? They added a random selection of cases and provided summaries without intentional bias. Except for a complete set, this is always the case for every database of cases.

exadrid• 4 years ago

Wikipedia has a very clear pro-science and pro-reality bias, and that's a good thing. You would not want news like OAN to be cited on their.

coffeeblack• 4 years ago

The problem is that the news sources that Wp trusts have gone completely bananas over the past decade.

headsoup2• 4 years ago

Depends if it's true though, right? Let's not dismiss everything because culture wars.

ars2• 4 years ago

You clearly don't edit much of Wikipedia if you think that. Their bias actually toward things that are in the news.

And since the news of today is only interested in the most extreme, that's the facts you'll find on Wikipedia.

Truthfully a much worse problem is that an average user cannot edit on Wikipedia. To edit on Wikipedia you need to have an obsessive personality, to constantly guard your edits and keep other editors from removing them.

And that bias is much worse than any other, you'll find a ton of information on matters that attract obsessive personalities, and much less on more normal subjects.

wongarsu• 4 years ago

Wikipedia certainly isn't infallible, and on some topics reading the talk page or comparing how the article reads in other languages can broaden your understanding or completely change it.

But if that disqualifies Wikipedia as a "trusted source", which sources deserve that distinction?

helsinkiandrew2• 4 years ago

This is alarming but I'm not sure the study actually finds that "Wikipedia influences judicial behavior" in the way most readers would understand that term - influencing decisions and sentence.

It finds that "Getting a public Wikipedia article increased a case’s citations by more than 20 percent." That could mean judges are citing cases with wikipedia entries rather than similar cases without and their judicial behaviour remains the same.

feet• 4 years ago

My guess is that law clerks are using wikipedia as an easy way to find info on cases, which isn't entirely surprising to me

ajross• 4 years ago

Exactly, Wikipedia is the first step for almost every expert professional in every profession. We all do it, and the world is better for it.

Also the causality of the argument in the linked article seems clearly backwards anyway: it's not that "Getting a wikipedia page for your case makes it more likely to be cited", it's "More important/citable cases are more likely to have a wikipedia page". Which is sort of an obvious point.

photochemsyn2• 4 years ago

I think law clerks and other legal professionals would be using a curated service (most likely LexisNexis) as their primary information source, not Wikipedia.

Wikipedia is quite unreliable on any major legal case, controversial issue, political personality, corporate entity etc., as the relevant wikipedia pages end up being heavily manipulated by agenda-driven editors who often erase or alter the most important information.

smrtinsert• 4 years ago

Why would it be alarming. Wikipedia is a fine entry into any topic. Seeing PragerU influencing the court would be alarming.

bacchusracine2• 4 years ago

Sir, I'm afraid your bias is showing...

helsinkiandrew2• 4 years ago

Wikipedia is a fine entry into any topic

It isn't, that's the point of the study - not all court decisions are on wikipedia and those that are, are getting cited more. Judges, Lawyers, Clerks have access to the original decisions and court transcripts in electronic form going back decades and longer. If they're relying on whats on wikipedia - they're either doing poor research or are too lazy to write their own summations.

roenxi• 4 years ago

It isn't that obvious that this is alarming. What are we alarmed about?

The point of judges is that they make judgements. The citing of other law is just to try and keep some consistency between the judgements to make them predictable. In theory they could even give out inconsistent judgements and that would also be workable, it just happens that it usually means something bad is happening because judges aren't naturally more pleasant people than the horrible Mass of Humans that causes so much grief. So we want judges to put in an effort at consistency and if they are consistently using Wikipedia as a reference then it is easier for the rest of us to guess what the system is about to do.

It is easy to influence judicial opinion on a case. There are even professionals hired to do it as a full time job! We call them lawyers.

phpisthebest2• 4 years ago

>What are we alarmed about?

While I think the study has some major deficiencies addressed in other comments, if it was true, the alarm for me would be that Wikipedia has clearly become politically partisan in many area's and topics, including some that would intersect the law

Wikipedia is hardly a "neutral" site of just facts

MichaelCollins2• 4 years ago

The part that concerns ('alarm' implies surprise) me is the confirmation that activist wikipedians have levers and knobs to play with that impact court decisions.

And if Wikipedia does, you can bet Google does too.

wisnoskij• 4 years ago

But that is what influencing is. By definition, these results show that cases given Wikipedia articles exert an otherwise unproportional influence on the legal landscape.

Not a particularly surprising result, but it shows how much of an effect such a little thing can have, and how big of an influence for example controlling what articles published vs not published could have over time.

jbirer2• 4 years ago

After watching a female judge sentence a child with autism for killing his rapist and childhood abuser, after he was acquited by a male judge years before, I kind of lost faith in all types of judicial process. AI needs to take over for such things.

jagged-chisel2• 4 years ago

This goes against double jeopardy. Do you have a reference?

jbirer2• 4 years ago

https://www.youtube.com/watch?v=X2NZtcfqjDs

Warning: it's very emotional

morelisp• 4 years ago

I believe "he was acquitted" refers to the abuser, not the killer.

aqme28• 4 years ago

An AI would be trained on human decisions, and unless we're careful it can easily have the same biases as human decision making.

istinetz• 4 years ago

at least some biases we can avoid. We can scrub the training data from mentioning gender or race. We can not include photos of people, to avoid the attractiveness halo effect.

Taywee• 4 years ago

You mean when she sentenced a 25 year old with autism of stabbing a bedridden post-stroke 74 year old to death? https://www.bbc.com/news/uk-england-london-62332824

You're really misrepresenting the case, and I doubt a good AI would rule favorably on those circumstances. The judicial system doesn't tend to go easy on stabbing defenseless people to death, even if you can argue that they were bad people or otherwise deserved it.

sgjohnson• 4 years ago

If I was bedridden for probably the rest of my life, I’d really appreciate if someone stabbed me to death.

martin_drapeau• 4 years ago

How did clerks/lawyers find those articles on Wikipedia? Through Google I reckon. I wonder how bad/good the search is for court judgements of their existing system.

Sounds like they need to put those court judgements online, or if they already are, do some SEO work so they rank. Wikipedia was a great way to do that.

delecti• 4 years ago

There are legal search engines which just about every lawyer should have access to. Resorting to google or wikipedia seems like a weird intersection of being comfortable with technology but unaware of any of the standard options.

laGrenouille• 4 years ago

LexisNexis, which does other things now but started in the legal space, offers a huge collection of legal opinions with a fairly good search and linking capabilities. Most clerks and law professionals would have access to it.

I think the benefit of Wikipedia is not access to materials so much as it is the succinct summarization of the legal opinions. Perhaps now NLP could help with this, but it's a very complicated problem to provide a summary of the important bits from a 100+ page legal document.

bdowling2• 4 years ago

... the succinct summarization of the legal opinions.

LexisNexis and Westlaw produce succinct summaries of legal opinions. That's the basis of their value, because the legal opinions themselves are not copyrighted. They also categorize everything about an opinion so that their database is searchable by area of law, etc.

pojzon• 4 years ago

Lets check how google search results influence judicial behaviour. Ups we already know google creates echo chambers.

jorgesborges• 4 years ago

Unsurprisingly, the increase was bigger for citations by lower courts... The researchers suspect this is showing that Wikipedia is used more by judges or clerks who have a heavier workload, for whom the convenience of Wikipedia offers a greater attraction.

This makes sense. I don't know anything about the legal profession but they must have their own search tools and databases? Is it not easily accessible or digitized for all levels? I don't get why it would be easier in the first place to search Google or Wikipedia.

_fat_santa• 4 years ago

If I had to bet, they have those systems but they are a pain to use. If I had to speculate, it would be a lot like those citation search sites you used in College, they simply didn’t have the usability of Google. You had to enter search terms very specifically, whereas with Google you just search for what you want and Google figures out how to structure the query best to give you the results you want.

I can totally see a scenario where a clerk needs to look something up on a Friday afternoon, the though of pulling up the proper search catalog pains them so they shortcut, googling the thing they are looking for a reading the citations on Wikipedia. If I was a law clerk I would 100% do this.

vharuck• 4 years ago

Not to mention that Wikipedia also acts as a filter for importance. Important cases are more likely to have articles. This is bad for exhaustive research, but great for "I need to write a summary by tomorrow."

RaCaS123• 4 years ago

Speaking from experience, you're right given its convenience. Though, I (and any responsible lawyer for that matter) will then also look up the case on WestLaw/Lexis given Wikipedia is very much an imperfect source.

gergov• 4 years ago

you would be surprised how the world operates outside of tech

it's scary excel sheets and a lot of ducttape

hell I wouldn't be surprised if the next round of innovation will be born out of tech people going into gov and realizing how much man hours there are to automate

stevage5• 4 years ago

That movement has been around a long time - see Code for America and its offshoots for example. It sometimes works, sometimes doesn't. It's easy to think that new tech will fix everything but you need to understand the bureaucracy the tech resides within.

photochemsyn2• 4 years ago

This study should have been controlled by asking whether courts had access to the curated legal databases (primarily LexisNexis, I imagine):

"...the increase was bigger for citations by lower courts — the High Court — and mostly absent for citations by appellate courts — the Supreme Court and Court of Appeal."

Secondly, is it possible that the study designers themselves used LexisNexis to research their Wikipedia articles? The study design was based on:

"...creating new legal Wikipedia articles to examine how they affect the legal decisions of judges. They set off by developing over 150 new Wikipedia articles on Irish Supreme Court decisions, written by law students. Half of these were randomly chosen to be uploaded online, where they could be used by judges, clerks, lawyers, and so on — the “treatment” group."

If these Wikipedia articles were based on LexisNexis research, and that same LexisNexis database was being used by judges and clerks, this would seem to introduce some major biases (making the study rather worthless in terms of measuring Wikipedia influence).

pushedx• 4 years ago

The 77 cases that were given a Wikipedia article saw a 17% increase in citations after publication.

One significant outcome of giving a case a Wikipedia article described in the paper but not in the article is that the case receives an infobox visible at the top of search results for most search engines.

whatshisface• 4 years ago

Maybe there's a rising tide of interest in certain cases that lifts both Wikipedians and jurists.

wellbehaved• 4 years ago

Everything is politicized and weaponized.

pyuser583• 4 years ago

A while ago the Oxford English Dictionary wrote a letter to the US Supreme Court asking them not use the OED as an authority.

They said the OED definitions are based on imperfect collections of common use, and aren’t “right” or “wrong”.

They also said the OED often uses SCOTUS decisions as a source, which creates an incestuous link.

kemitchell• 4 years ago

Really interesting.

I searched for this online, but couldn't find it. Might you have a link handy? Much appreciated!

notRobot• 4 years ago

Not exactly what you're looking for, but a super interesting article from 2013 from the Oxford University Press's Blog:

The highest dictionary in the land? https://blog.oup.com/2013/06/scotus-marriage-definition-dict...

t_mann• 4 years ago

Previous discussion on HN:

https://news.ycombinator.com/item?id=32253053

option• 4 years ago

in other news, reading books and attending judicial classes and conferences influences judicial behavior

DontchaKnowit• 4 years ago

Can we stop calling stuff like this a "study"? This is not science, this is essentially journalism.

I mean technically yes they are "studying" a subject but this is not a scientific experiment, but it will almost certainly be cited as such just because it's called a study.

6177c40f• 4 years ago

What about it makes it not science?

tediousdemise3• 4 years ago

Concluding that knowledge influences behavior isn't science. It's common sense.

gwern2• 4 years ago

This is definitely a study and science. It's actually a rather high quality experiment too, I was in awe of the way they carefully constructed the pairs of decisions to make them as identical as possible, and then got the same law student to write an article for each member of a pair before randomizing to ensure they were written as similarly as possible. Everyone knows that you should be blocking to maximize statistical power, but very few people ever go that far. (And that's probably a good part of why despite the relatively small _n_, they still get very clear causal effects.)

stevage5• 4 years ago

I still don't understand why they bothered writing the articles for the ones that weren't being uploaded. I'm sure there's a reason, it just wasn't obvious to me.

edgyquant6• 4 years ago

This is a statistical study, what are you talking about?

dukeofdoom• 4 years ago

Judges are biased, especially in political cases. Look at the joke of trial in Alex Jone's trial right now to see this. He's literally not allowed to plead his innocence to the jury. De facto ruling based on accusation of not turning over all the evidence.

Biased judges like this, love circular reasoning, to find justifications. Hence why Wikipedia is useful. Keep in mind wikipedia change the definition of a recession recently, than when challanged changed the definition of definition. Just last week.

tediousdemise3• 4 years ago

Knowledge influences behavior? No way, I don't believe it.

Say I want to learn how to do a backflip. I do a quick search, find an instructional video, and nope the fuck out because of how dangerous and difficult it seems to perform. A prime example of knowledge that influenced my behavior.

Pointless study since this is just common sense. So much time and money is wasted in science just so that people can make a name for themselves or get a promotion.

stevage5• 4 years ago

To me, the fact that Wikipedia influences law is surprising. I would not have expected judges or their clerks to rely on Wikipedia or a resource.

TechBro8615• 4 years ago

It doesn’t seem unreasonable to extrapolate this finding to other areas of society where a small percentage of people write their opinions and then a much larger percent of people read them, eventually shaping aggregate opinion independently of any opinion held predominantly by “lurkers.”

How many elections have been shaped by 4chan, Reddit, and Facebook comments? There have been a few studies on this, eg in 2016 [0] and 2018 [1] which found results that probably will not shock you.

Personally, this might be confirmation/proximity bias, but I believe the effect of small internet communities is severely underestimated by the public and political consultants. This is an area that needs a lot more study, not so that we can stop it, but so we can raise public awareness of how memes spread and give people the tools to think critically without falling into the self-reinforcing cognitive traps that arise from filtering information with poor heuristics.

[0] https://arxiv.org/abs/1610.03452

[1] https://arxiv.org/abs/1805.12512v1

tediousdemise3• 4 years ago

a small percentage of people write their opinions and then a much larger percent of people read them, eventually shaping aggregate opinion independently of any opinion held predominantly by “lurkers.”

ELI5 brainwashing

asdff• 4 years ago

This is a repost from three days ago (190pts, 107 comments):

https://news.ycombinator.com/item?id=32253053

mikotodomo• 4 years ago

Wikipedia is an amazing wonder of the internet. I could spend ten hours a day reading deep into topics and related topics I only learned because of the initial curiosity I had. But that's beside the point. There is such thorough coverage of all topics of every field so I would indeed expect that judges are using it to look things up because of how convenient it is.

mellosouls• 4 years ago

The problem is that in certain subjects (especially those with a potential ethical dimension like social and cultural pages) there is significant bias towards the left (I say that as a left-leaning person).

That's not healthy at all when being used as a resource without awareness and significant caution.

jrochkind1• 4 years ago

On HN front page 4 days ago, with 190 comments: https://news.ycombinator.com/item?id=32253053

mch822• 4 years ago

Curious what people think about the ethics of this study?

The researchers describe running an experiment on the public without opt-in consent. They also describe allowing an algorithm to do the editing. That raises questions about the kind of edits the algorithm made, how the edits influenced case outcomes, and if/how the research team will clean up Wikipedia now that they’ve concluded the study.

Are there professional guidelines that suggest how experiments like this should be conducted? Do any regulations exist to protect people who get exposed to information as a result of an experiment like this?

wheelinsupial• 4 years ago

There is a paragraph on this in the publicly accessible pre-print [1] of the article.

"Approved by the respective ethics boards of MIT and Maynooth University, the experiment amounted to a friendly stress-test of the potential vulnerability of judicial legal reasoning to the limitations of reliance on Wikipedia, notably, its ad hoc topic coverage and unknown author/editorship. The experiment featured Wikipedia entries authored by faculty and by law students under faculty supervision, who each had access, through their university library, to all the relevant primary and secondary legal materials available to judges and their clerks. This assurance of accuracy and of informed analysis in the content of the entries—though short of that offered by a specialist textbook—indicates that judges or lawyers would be unlikely to be misled by what they might read. However, as the authorship of Wikipedia articles is opaque, this fact would not be known to any legal professional when using them. From the users’ perspective, there was no particular reason to imagine that the creators of the relevant entry had any legal expertise—or even that they lacked an ulterior agenda."

I'm curious where I missed the part about the algorithm did the editing? I don't see any matches for "algo" when I control + f the article. "Algo" doesn't appear in the article either. The excerpted paragraph says who wrote the entries and it wasn't an algorithm.

[1] https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4174200

mch822• 4 years ago

Thanks. You summarized the paper better than the article this discussion thread is about. On closer reading, CSAIL had people write the articles and used natural language processing to detect “linguistic fingerprints of the Wikipedia articles that they’d created”.

kazinator• 4 years ago

Is that necessarily bad?

A best-case-scenario example of influenced judicial behavior is that a judge is about to make a bad decision, but then is informed in some way by an excellent, well-referenced Wikipedia article.

Study finds Wikipedia influences judicial behavior

Related Stories