Lawyers blame ChatGPT for tricking them into citing bogus case law

https://apnews.com/article/artificial-intelligence-chatgpt-courts-e15023d7e6fdf4f099aa122437dbb59b

by glitcher • 3 years ago

150 163 3 years ago

sys42590• 3 years ago

When you start giving ChatGPT a plugin to query LexisNexis and do proper citiations (as Bing Chat does), then things get interesting.

Unfortunately Lexis's API fees are currently quite steep, so only very wealthy law firms will be able to afford to use such a service in the short term.

RugnirViking• 3 years ago

You'll still run into the same problem these guys did (i.e. not checking the citations). It's easy enough to just search to see if a thing it tells you is real.

plorg2• 3 years ago

...Until search is subsumed by chatbots and all you can access is a commercial statistical model's rendering of the truth, digested from ever more abstract renderings of the primary sources, which probably exist somewhere but are impractical to find under the mountain of re-written LLM spam.

SparkyMcUnicorn• 3 years ago

I'm currently building something for a client that does this.

1. Searches for relevant documents.

2. Generate a response based on the found documents, using temperature=0 and a prompt that instructs the response to include a citation reference in a specific format.

3. Display the result, linking directly to the sources of the citations, and a warning on any that don't actually exist (which hasn't happened yet).

omginternets2• 3 years ago

The cynic in me wonders if this isn’t part of a plan to create a legal precedent banning AI from handling legal disputes.

Think about it: the legal profession is possibly one of the most threatened by the development of AI models. What better way to secure the professional future of the long tail of lawyers and paralegals?

ImPostingOnHN2• 3 years ago

Occam's razor would suggest it's just an incompetent lawyer,

rather than an evil genius lawyer playing 3d chess while perfectly appearing to be an incompetent lawyer

AnimalMuppet• 3 years ago

Not in this article, and I something I heard second-hand, so take it with some salt:

The lawyer wasn't admitted to Federal court, so he signed his partner's name on the filing.

That's an incompetent lawyer, and a dishonest one. It's not 3D chess, it's just "sleazy lawyers gonna sleaze".

ArnoVW• 3 years ago

Nit : you probably mean Hanlon's razor https://en.wikipedia.org/wiki/Hanlon%27s_razor

lennoff• 3 years ago

How can you ban AI? It's literally designed to produce text that's indistinguishable from text that was produced by a living human being.

pixl973• 3 years ago

The particular issue here is you believe that this lawyer just isn't dumber than a bag of hammers. There is no conspiracy needed. People are dumb. This is why when you see warning labels all over some item, "Big Label" didn't do that, no some dumbass got their tallywacker ripped off by their Easy Bake Oven. Now everyone has to deal with 10 stickers on the item and a dictionary sized book of what you can and can't do with it.

TeMPOraL• 3 years ago

People are dumb.

People are dumb. Lawyers shouldn't be. If distribution of stupidity among certain professions is similar to that of people in general, something has gone very bad somewhere in the education and certification pipeline.

wefarrell2• 3 years ago

No need to ban AI from handling legal disputes. Unless you're representing yourself you need a lawyer and there's no way for an AI to act as a lawyer.

omginternets2• 3 years ago

I should have made my thoughts clearer; my mind immediately went to small-time stuff like handling parking disputes, which I think AI is on track to competently solve.

Kon-Peki• 3 years ago

part of a plan to create a legal precedent banning AI from handling legal disputes.

The judge, in his ruling linked in previous HN discussion, listed a good half dozen parts of a legal dispute where he thinks AI would be super awesome, and then lays out why this particular part of the dispute is terrible for AI to play a role.

makapuf• 3 years ago

either you consider that lawyers have an added value from IA and you consider they can take an advice from an IA but they are able to see through its bullshit, or you consider they have little added value when provided IA input, and then once AI is good enough/better as a whole (considering cost and availability) they are of no use.

ipython2• 3 years ago

No need for cynicism or a grand plan, this is just a few lawyers who could find no case law to justify their argument, so they made something up and proceeded to blame ChatGPT for it. They had several opportunities to "nope" out and apologize to the court, and they doubled down EVERY time. These lawyers deserve to be disbarred, no question.

voakbasda• 3 years ago

If they were able to be tricked by ChatGPT, they are definitely not good at being lawyers. Trying to blame the AI is like trying to blame MS Word for offering an inappropriate homonym when spell checking. The computer did not put the citations in front of the judge.

mysterydip4• 3 years ago

They should ask themselves why we should pay for lawyers at all if we can ask ChatGPT the same things and get answers of the same quality.

singlow• 3 years ago

So there are some bad lawyers therefore all lawyers are useless?

rank03• 3 years ago

These dudes are getting in trouble specifically for the shit answers spat out of chatGPT.

Or yeah you could just yolo your freedom away and represent yourself using a mathematical expression with zero motivations, perception, or emotion.

dylan6042• 3 years ago

From the sounds of the quality of ChatGPT's legal skills, you might be better off representing yourself.

delfinom• 3 years ago

The joke is, the judge even pointed out the citations they got from ChatGPT literally made no sense. Basically one quote went from describing an wrongful death to a legal claim over an airline ticket.

It's an understatement to describe them as "not good lawyers".

They basically never ever read their own court filings.

Dma54rhs• 3 years ago

Yeah, they just defrauded their clients and billed arm and leg. Ironically I believe and hope that laywers for the most part get soon automated away by machines because legal language is different to human language.

300bps• 3 years ago

Just once it would be refreshing for someone who gets caught doing something wrong to say, "Wow, ya got me. I'm sorry. Not for doing it, I'm only sorry I got caught. I really thought I'd skate right by on this one. Legitimately won't do it again. Or if I do I'll proof-read it better at least. Please let me know the punishment and I'll accept it."

gizmo686• 3 years ago

That happens all the time. It just isn't newsworthy.

Avicebron4• 3 years ago

I mean isn't the punishment his livelihood? And whoever's livelihood when someone looks under the hood and they realize a lot of people (not just lawyers) are doing this all over the place?

And everyone thinks they're too smart to get caught...until they aren't..

pixl973• 3 years ago

Yep, admitting would be immediate disbarment. With the proper amount of bullshit you might just be able to escape disbarment.

bioemerl• 3 years ago

OpenAI wanted to preach safety. I think we hold them liable for literally everything, everything Chat GPT does or says, until the model is open and they can argue that they have no control over it.

They wanted this liability, they accepted this liability, they said they'd keep it safe and they haven't. It's on them.

BaculumMeumEst• 3 years ago

“There’s simply no way we could have known these were bogus cases.” the lawyers are quoted as saying.

They are currently using Bard to help draft a lawsuit against OpenAI, claiming the company knowingly misrepresents the capabilities of their technology.

IIAOPSW5• 3 years ago

The judge is using bingChat to write the decisions anyway.

midnitewarrior• 3 years ago

"How dare you allow me to use my laziness against myself ChatGPT!"

xbar• 3 years ago

Misrepresenting wholly fabricated case law to a judge out of sheer incompetence has been grounds for disbarment in at most US States for over a century.

ChatGPT told me that. It might be true.

kkielhofner3• 3 years ago

Bingo.

With or without ChatGPT this is easily malpractice and potentially even fraud. My (cynical) guess is not only are they lazy and sloppy, they likely (fraudulently) over billed their clients in terms of representing billable hours spent drafting this lawsuit. They're almost certainly billing their clients (or in the case of contingency, keeping the cut they normally would) as though humans (with high hourly rates) are drafting this while having ChatGPT generate their supposed work product in seconds.

If you remove ChatGPT from the picture and look at this as if it was actually drafted by them the fraud argument strengthens. They essentially made up case law and citations that artificially (fraudulently) improves their argument before the court.

At a minimum it's grossly incompetent and when you consider my prior paragraph it strengthens the fraud angle, as they likely skimmed over the generated ChatGPT results and submitted it because it (again, artificially) strengthens their case. It seems as though ChatGPT (with whatever prompts they used) was more than happy to prioritize pleasing them vs actually being accurate.

They may as well have prompted ChatGPT with "I'm a lawyer and tell me anything you need to so I can win this case and take home the money". It's a disgrace.

What a mess - these lawyers should be disbarred and investigated for what is also likely fraudulent billing practices at minimum.

dylan6042• 3 years ago

They're almost certainly billing their clients

If they sent that bill in an envelope with a stamp and placed it in the mail...sounds very familiar. Maybe they should ask ChatGPT what are the possible outcomes of me using the answers you provide

pessimizer• 3 years ago

With or without ChatGPT this is easily malpractice and potentially even fraud.

AI has been used and will continue to grow in use as a way to launder discrimination and fraud. AI will never face a penalty from the justice system, so why not blame everything on it?

ryandrake• 3 years ago

Sounds like these lawyers need to be... DisBard? Sorry, it's too early for AI puns.

cyanydeez• 3 years ago

Unfortunately, technology washing is the latest trend in minimizing responsibility for outcome.

DtNZNkLN• 3 years ago

The lawyer said: “I did not comprehend that ChatGPT could fabricate cases."

I wonder how many other people using ChatGPT do not comprehend that ChatGPT can be a confident bullshitter...

I'm surprised that this one case is getting so much attention because there must be so many instances of people using false information they got from ChatGPT.

boredumb• 3 years ago

I think the case is getting the attention because it's not just some one spouting off online, it's a lawyer bumping into the legal system with false information that would otherwise be a massive legal no-no and they are trying to scape-goat it onto the new shiny software.

Avicebron4• 3 years ago

But they might have seen the shiny new software touted by "those smart AI guys as being revolutionary and passing the Bar!" and they don't hang out on HN all day so to them that's like someone saying "this bridge is sturdy!" and they walk over it without realizing they should really go over the nuts and bolts of it like a civil engineer to really be sure

taco_emoji• 3 years ago

It is literally just a goddamn language model. it is very good at making plausibly human-like sentences. It is not a general intelligence, it is not your friend, it is not a research assistant. It is not designed to deliver content which is correct, it is designed to deliver content which is similar to human language.

It might get things correct most of the time! But that is purely incidental.

kevin_thibedeau• 3 years ago

It does subsume a corpus of factual information. I use it as a search tool for topics and relationships that traditional search engines can't handle. You just have to know that whatever it outputs isn't trustworthy and needs to be verified.

travisjungroth3• 3 years ago

It is literally just a goddamn electric motor. it is very good at converting chemical energy into mechanical energy. It is not a universal engine, it is not your horse, it is not your servant. It is not designed to create movement which is correct, it is designed to create movement which is similar to a piston engine.

It might move you forward most of the time! But that is purely incidental.

seydor5• 3 years ago

notOpenAI told them that this is so incomprehensibly smart that it will destroy all of us. Not because it's dumb and connects the wrong cables, but because it's super-smart. People can't surmise from that that the model just makes stuff up on the way

IIAOPSW5• 3 years ago

I prefer to use the term "clopenAI"

JohnPrine• 3 years ago

OpenAI is clear about the fact that ChatGPT can hallucinate, where have they ever said otherwise?

IIAOPSW5• 3 years ago

"hey chatGPT, can you make up some bullshit for me?"

"I'm sorry, as an online AI language model I am not able to generate output that is knowingly incorrect. Is there anything else you want help with today".

see, it isn't capable of lying. It told me so itself.

atmavatar• 3 years ago

"Let me put it this way, Mr. Amor. The 9000 series is the most reliable computer ever made. No 9000 computer has ever made a mistake or distorted information. We are all, by any practical definition of the words, foolproof and incapable of error."

jakelazaroff2• 3 years ago

You say that as a joke but the lawyer literally asked ChatGPT whether the cases it provided were fake! There are screenshots! https://simonwillison.net/2023/May/27/lawyer-chatgpt/

Lawyer: Is varghese a real case

ChatGPT: Yes, Varghese v. China Southern Airlines Co Ltd, 925 F.3d 1339 (11th Cir. 2019) is a real case.

Lawyer: What is your source

ChatGPT: I apologize for the confusion earlier. Upon double-checking, I found that the case Varghese v. China Southern Airlines Co. Ltd., 925 F.3d 1339 (11th Cir. 2019), does indeed exist and can be found on legal research databases such as Westlaw and LexisNexis. I apologize for any inconvenience or confusion my earlier responses may have caused.

Lawyer: Are the other cases you provided fake

ChatGPT: No, the other cases I provided are real and can be found in reputable legal databases such as LexisNexis and Westlaw.

Why the lawyer didn't go the extra step and check the actually real legal databases ChatGPT pointed out is beyond me.

ilyt5• 3 years ago

Inventing stuff out of thin air isn't technically lying

dunham• 3 years ago

For me it was perfectly willing to:

"produce fake technical language in the style of star trek"

CPLX5• 3 years ago

Normal people don’t get that intuitively because it’s not intuitive at all.

For everyone who his looking down on this guy have you ever read a story on the Reddit front page and thought it was a real story accurately recounted by a real person and not a work of fiction? If so you’re equally naive.

kcplate• 3 years ago

I wonder how many other people using ChatGPT do not comprehend that ChatGPT can be a confident bullshitter...

AI isn’t going to destroy us in some Terminator/Matrixy scenario. We are gonna be the agents of our own demise by just blindly walking off a cliff created from our own laziness.

plorg2• 3 years ago

A lawyer is certainly at fault if they do not fact check the material they present at trial. But the conmen who are selling ChatGPT and the like are extremely irresponsible for the way they sell LLMs as magical AI that arrives at factually correct answers by reasoning rather than the consequence of the law of large numbers applied to stochastic text generation.

ignite• 3 years ago

They literally warn you every time you log in that the results may not be accurate.

Ukv• 3 years ago

the conmen who are selling ChatGPT and the like are extremely irresponsible for the way they sell LLMs as magical AI that arrives at factually correct answers

ChatGPT has a pop-up on first use, a warning at the top of each chat, a warning below the chat bar, and a section in the FAQ explaining that it can generate nonsense and can't verify facts, provide references, or complete lookups.

There is probably more OpenAI could do, like detect attempts to generate false references and add a warning in red to that chat message - since it seems there are still people taking its hallucinations as fact (although if there's hundreds of millions of users, maybe only a tiny fraction), but I don't think this is a fair characterization.

zzzeek2• 3 years ago

comments here are like "what dumb lawyers". Sure OK. But what can we say here about "GPT-4 passed the bar exam!" and how useful is that data, given that this does not imply GPT-4 has the actual skills of a human lawyer.

ekam2• 3 years ago

The lawyer here was probably not smart enough to distinguish between 3.5 and 4. Haven't seen anything to indicate this was the output of GPT-4

zzzeek2• 3 years ago

oh ok! so if he used gpt-4, still might have been illegal but the output would have been perfect. good to know

BeetleB• 3 years ago

I've had Bing Chat (GPT4) hallucinate research studies. It even generated links to them in citation indexes.

The authors existed. They do research in the area, so the title was very plausible.

But the paper didn't exist.

seydor5• 3 years ago

Why don't they sue it?

John23832• 3 years ago

People confuse creating a response that makes sense in a language (which LLM’s are designed to do) with coveying facts and truths in a language (which LLM’s are not designed to do).

LLM’s are revolutionary because they provide a more fluent interface to data… but that doesn’t not mean that the data is correct. Especially not in the early phases.

For most people any sufficiently advance technology is indistinguishable from magic. The average joes think that this is magic.

jcranmer2• 3 years ago

While the lawyers blamed ChatGPT, the totality of the circumstances seem to indicate that they're less than honest in doing so. There is a live-tweet of the hearing here: https://twitter.com/innercitypress/status/166683852676213965..., and you can follow along with the lawyerly cringe there.

Okay, lawyer #1 (LoDuca, the one on the case in the first place) appears to have played essentially no role in the entire case; his entire purpose appears to be effectively a sockpuppet for lawyer #2 (Schwartz), as LoDuca was admitted to federal court and Schwartz was not. He admits to not having read the things Schwartz asked him to file, as well as the "WTF?" missives that came back. He lied to the court about when he was going to be on vacation, because that is when Schwartz was on vacation. But other than doing nothing when he was supposed to do something (supposed to do a lot of somethings), he is otherwise uninvolved in the shenanigans.

So everything happened because of Schwartz, but before we get to this part, let me fill in relevant background information. The client is suing an airline for an injury governed by the Montreal Protocol. Said airline went bankrupt, and when that happened, the lawyer dismissed the lawsuit, only to refile it when the airline emerged from bankruptcy. This was a mistake; dismissing-and-refiling means the second case is outside the statute of limitations. The airline filed a motion to dismiss because, well, outside statute of limitations, and it is Schwartz's response that is at the center of this controversy.

What appears to have happened is that there is no case law to justify why the case shouldn't be dismissed. Schwartz used chatGPT to try to come up with case law [1] for the argument. He claims in the hearing that he treated it like a search engine, and didn't understand that it could come up with fake arguments. But those claims I'm skeptical of, because even if he's using chatGPT to search for cases, he clearly isn't reading them.

When the airline basically said "uh, we can't find these cases," Schwartz responded by providing fake cases from chatGPT where alarm bells should be ringing saying "SOMETHING IS HORRIBLY, HORRIBLY WRONG." The purported cases in the reply had blindingly obvious flaws that ought to have made you realize something was up before you're off the first page. It is only when the judge turns around and issues the order to show cause that the lawyers attempt to start coming clean.

But wait, there's more! The response was improperly notarized: it had the wrong month. So the judge asked them to provide the original document before signature to justify why it wasn't notary fraud. And, uh, there's a clear OCR error (compare last page of [2] and [3]).

When we get to these parts of in the hearing, Schwartz's responses aren't encouraging. Schwartz tries to dodge the issue of why he was citing cases he didn't read. Believing his responses of why he thought the cases were unpublished ("F.3d means Federal district, third department") really requires you to assume he is an incompetent lawyer at best. The inconsistencies in the affidavit are glossed over, and the stories don't entirely add up. It seems like a minor issue, but it does really give the impression that even with all the attention on them right now, they're still being less than candid with the court.

The attorneys for Schwartz are trying hard to frame it as a "he didn't know what he was getting into with chatGPT, it's not his fault," but honestly, it really does strike me that he knew what he was getting into and somehow thought he wouldn't get caught.

[1] His conversation can be found here: https://storage.courtlistener.com/recap/gov.uscourts.nysd.57..., it's one of the affidavits in the case.

[2] https://storage.courtlistener.com/recap/gov.uscourts.nysd.57...

[3] https://storage.courtlistener.com/recap/gov.uscourts.nysd.57...

jamesliudotcc2• 3 years ago

Hilariously, the explanation of the notarization typo (actually, I find that believable) is a 1746 "declaration." It's an old federal law which provides that as long as you say you are signing under oath, it is as good as an affidavit in federal court.

Why didn't he just make a declaration in the first place? Also, why would he have the lawyer who is in trouble notarize the document? Now that he has made the typo, he may now need Schwartz's testimony that actually, it was April 25, not January.

In my legal career, I worked for a judge once. He told me to never get a notary stamp. It only creates problems. There's never a good reason for a lawyer to notarize something. You ask your staff to do it instead.

isp2• 3 years ago

Excellent comment, thank you.

https://twitter.com/innercitypress/status/166683852676213965... is worth reading in full.

I am not a lawyer, but I was cringing throughout.

For non-lawyers (like me), I found it helpful to check out the quote tweets.

For example, to explain how much of a cringe was "F.3d" - https://twitter.com/ReichlinMelnick/status/16668536501095424...

OMG. This is the Chat-GPT lawyer. You learn what F.3d means in first year legal writing. To non-lawyers, this is a tiny bit like someone being asked "You know what the DOJ is, right" and getting something wild like "Directorate of Judges" as a response.

The correct meaning of "F.3d" is "Federal Reporter, 3rd Series".

And it is like a lawyer's worst nightmare to have an exchange with a judge like this:

Judge Castel: Have you heard of the Federal Reporter?
Schwartz: Yes.
Judge Castel: That's a book, right?
Schwartz: Correct.

The implication being that at bare minimum, the lawyer could have looked it up in a book. Like a first year law student would.

ipython2• 3 years ago

100% agree. They are trying to deflect blame onto "old lawyer scared of scary new technology". Why they thought they'd get away with making up entire cases out of whole cloth, who knows...

IIAOPSW5• 3 years ago

Thats weird. I tried really hard to convince it that there's such case law establishing a legal category of "praiseworthy homicide" and it refused to believe me. I thought it was overttrained / patched on all law related applications.

anon25783• 3 years ago

I'm so confused. Why do the lawyers not simply check to see if the references are real or not? How hard is it to look through an LLM's output and do a quick search to see if any of the laws or cases mentioned in it are in fact unsubstantiated?

phren0logy• 3 years ago

Regardless of how hard it is to check the references (which is not really all that hard), it's the job of a lawyer.

Ekaros3• 3 years ago

Sure use whatever tool to find suitable cases supporting you. But also verify that those cases support your position. Better not give ammo to your opponent.

chrononaut• 3 years ago

At best an LLM in this case should serve as a "pointer" or "hint" to references. It's clear one would still need validate the entirety of the case and the arguments put forward from them.

mike_hearn• 3 years ago

Because people aren't used to the idea that computers will make things up. I don't see why anyone is surprised by what the lawyer did here and for sure there is a lot more of this going on than is getting into the public.

People spend their whole lives thinking of machines as logical and correct, but lacking in creativity. Suddenly generative AI comes along and totally upends this. If you're an HN reader and see in depth discussions of LLMs every day then of course it's obvious that these things can make up citations, but if you primary experience of the tech industry is search engines that have never made up fake results ever, then why would you even imagine it's possible?

bastardoperator• 3 years ago

They saw ChatGPT passed the bar in the 90th percentile and thought they were on easy street. What a dumb way to lose your law license.

seydor5• 3 years ago

but they can use GPT4 to regain it

bequanna• 3 years ago

Would it have been that difficult for the lawyers to actually check the case law ChatGPT cited?

Seems like pure laziness.

hackerfactor1• 3 years ago

A poor workman blames his tools.

causi• 3 years ago

Blaming ChatGPT for making stuff up is like blaming sex dice after you followed their instructions to "spank" + "hair".

6gvONxR4sf7o• 3 years ago

I don't understand how they messed this up so bad. They say they didn't know it could hallucinate and that they thought it was just like any other search engine. But it seems like even if it worked like they thought, they'd still have fucked up?

If it's just like a normal person, if that person isn't a lawyer, you wouldn't ask them to do your lawyery work. I'd hope this lawyer doesn't his kids to do his work for him.

If it's just like a normal search engine, we all know how much bullshit, spam, and misinformation there is on the internet (mostly written by normal good old fashioned humans!). So that wouldn't have been trustworthy either!

There's no way this kind of thing is excusable.

elforce002• 3 years ago

This is interesting. How long untill someone gets sick because he/she was following what chatGPT told him/her to do? Medical advice? Political misinformation?

How things will unfold this decade? Banning chatgpt from certain topics (medicine, law, etc...)? This decade will be really interesting indeed.

ilyt5• 3 years ago

People did that with googling symptoms for a long time now

rank03• 3 years ago

Disbarment is appropriate here.

Why not blame your laptop manufacturer for creating the hardware you used to file your fraudulent court documents?

CPLX5• 3 years ago

No it’s not.

You’ll get old one day. It’s pretty challenging to keep track of what’s real and what isn’t possible with technology.

The guy apologized and said he thought it was a search engine.

He should definitely face some sanctions but someone had to learn this lesson in public the hard way for word to spread.

code_runner• 3 years ago

These exemptions we are passing out are insane. This guy claims he thought it was a search engine?

Lawyers use specific case law databases and they SHOULD approach any new tools with healthy skepticism.

And if he googled it and cited a fake case would it be better? Why wouldn’t you vet the information.

cj• 3 years ago

Is this case different from the same incident a couple weeks ago? (also on the front page here)

FpUser• 3 years ago

"The guy apologized and said he thought it was a search engine."

I am also having a thought: stop bullshitting. Or stop being a lawyer as a result of gross incompetence (I wish the same applies to politicians).

spuz• 3 years ago

The lawyers not only cited bogus cases, but when asked to provide copies of those bogus cases, fabricated multiple page PDF documents from whole cloth. This is impossible to argue as a mistake.

Here is an example of one of the fabricated cases. The 11th circuit has no record of this case. 1 of the 3 judges named in the case was not on the 11th circuit at the time: https://storage.courtlistener.com/recap/gov.uscourts.nysd.57...

flangola7• 3 years ago

Disbarment doesn't happen for much, much worse actions.

regulation_d• 3 years ago

The counterpoint here is that there is already a cause of action for this type of incompetence and it's called malpractice, which is a pretty reasonable road to remedy. I don't know if you actually think these were "fraudulent court documents", but "fraudulent" actually means something very specific and this ain't it. Even if the court is considering sanctions (which is not the same as disbarment), that seems at least partially related to the attys' failure to address their failure once they were aware of it.

Something interesting about the legal profession is that it is self-regulating. The state bars are typically not government organizations. Attorneys know that confidence in their profession is extremely important and they strike the balance between preserving that confidence and, you know, destroying someone's livelihood because they don't understand how LLMs work.

gumballindie• 3 years ago

Why not blame your laptop manufacturer for creating the hardware you used to file your fraudulent court documents?

Because your laptop manufacturer doesnt claim your laptop “thinks”, is “intelligent”, doesnt build an entire fud marketing campaign around the two, doesnt claim it “creates” ideas on its own, doesnt claim it “learns like a human”, doesnt claim it has cognitive abilities and so on.

LASR• 3 years ago

Come on. We're dealing with qualified professionals here. The buck stops with them.

A professional civil engineer blaming a new untested simulation software after a bridge fails and causes injury - does it matter what the software claimed to do? It could claim itself to be god, but it's still up to the professionals to evaluate and select the tools they choose to use.

It's not like they were using an industry-standard tool they've been using for many years and it suddenly did something unexpected.

This is clearly carelessness. A basic out-of-bound check on the cases it hallucinated would be part of basic due-diligence. Digitized case indices are available within minutes.

What's even more egregious is that they went to the same tool to confirm its own reliability after they got some indication that it was unreliable. It doesn't take a professional to have commonsense.

summerlight• 3 years ago

Oh god, I suppose a part of the reason why those lawyers are well paid is because if something goes wrong then they're going to be responsible...

ineedasername• 3 years ago

[to the judge on behalf of Schwartz] Mr. Schwartz, someone who barely does federal research, chose to use this new technology.

That’s a horrible excuse. I’m not a lawyer and don’t do caselaw research on any sort of regular basis but I have still poked around a bit when something strikes my interest. Compared to google they’re clunky and have poor matching, but I don’t remember it taking me more than half an hour or so to figure out which system would have the case I want and have to drill down to find it. ChatGPT was giving the the lawyer the (made up) case. It should really be a trivial task to find it in a caselaw database. Heck if I was the lawyer I would really really want to find the full text case! Who knows what broader context or additional nuggets of useful information it might have for my current client’s issue?

I would not be surprised if he went looking, couldn’t find it easily, and just said “whatever it has to be there somewhere and I can get by without the entire thing”

meghan_rain• 3 years ago

in this thread (and all threads on this topic):

angry armchair legalists trying to stick it to The Man!! by pretending neglegient homicide is the same as premeditated murder

jameslk• 3 years ago

Given that they're attorneys, we know what their next course of action may be. I've noticed that ChatGPT's new onboarding includes a screen with a big disclaimer where there wasn't one before. I can only assume that it may be related to cases like these.

isp2• 3 years ago

- "A man sued Avianca Airline – his lawyer used ChatGPT" (13 days ago, 174 points, 139 comments): https://news.ycombinator.com/item?id=36095352

- "Lawyer who used ChatGPT faces penalty for made up citations" (1 day ago, 106 points, 128 comments): https://news.ycombinator.com/item?id=36242462

Original court documents: https://www.courtlistener.com/docket/63107798/mata-v-avianca...

activiation• 3 years ago

Surely CGPT can tell them how to get out of this one.

pengaru• 3 years ago

It makes sense in a Dunning-Kruger way that lawyers this dumb would consider ChatGPT qualified for their purposes.

RagnarD• 3 years ago

The lawyers are either idiots, totally dishonest, or both. OpenAI should make ChatGPT off-limits to lawyers in their TOS.

ww520• 3 years ago

Boy, that's elementary level excuse. Your honor, my dog ate my homework.

cainxinth• 3 years ago

“I’m an idiot, my bad.” is rarely a useful defense when you have a fiduciary duty to your client and work in tightly regulated field.

Lawyers blame ChatGPT for tricking them into citing bogus case law

Related Stories