A man sued Avianca Airline – his lawyer used ChatGPT

https://www.nytimes.com/2023/05/27/nyregion/avianca-airline-lawsuit-chatgpt.html

by nlh • 3 years ago

174 138 3 years ago

nytimes.com

jaclaz• 3 years ago

Another thread where the poster linked to the relevant Court .pdf's:

https://news.ycombinator.com/item?id=36092914

Particularly relevant is the affidavit in which the lawyer tries to explain to the Court what happened:

https://storage.courtlistener.com/recap/gov.uscourts.nysd.57...

lionkor• 3 years ago

Hahaha that second link is brilliant.

"Hm, maybe I should double check ChatGPTs output... Hey, ChatGPT, does your output make sense?" - "Yeah, my output definitely makes sense". "Are you sure?" - "Yeah".

Well, then.

namaria2• 3 years ago

Mr. Schwartz said that he had never used ChatGPT, and “therefore was unaware of the possibility that its content could be false.”

Why did he think it was true, if he had never used it before then?

"I found it online" levels of competence here.

Szpadel• 3 years ago

archive.org link https://web.archive.org/web/20230527153017/https://www.nytim...

metadat• 3 years ago

https://archive.today/pnEhz

m348e912• 3 years ago

This problem will be solved with LawGPT which will be a generative pre-trained transformer trained on a vast array of legal cases, rulings, and briefings and designed with accuracy in mind and free from hallucinations.

It will cost $3,400/month for a legal firm the size of Levidow, Levidow & Oberman, P.C.

ChuckNorris89• 3 years ago

Still waiting for SaulGoodmanGPT

nostrademons• 3 years ago

Same thing, but paid in Monero.

seanhunter• 3 years ago

You don't want a criminal law AI... you want a criminal law AI.

fhd2• 3 years ago

Would probably lead to an uptick in tourism for Belize.

hammock• 3 years ago

Write an amicus brief for a Supreme Court case in the style of Saul Goodman

UNITED STATES SUPREME COURT

Saul Goodman's Legal Advocacy Clinic 123 Shady Lane Albuquerque, NM 87102 Phone: (555) 123-4567 Email: saulgoodman@lawyerextraordinaire.com

May 27, 2023

JUSTICES OF THE SUPREME COURT

Re: Amicus Curiae Brief - Case No. 12345

Dear Honorable Justices,

I, Saul Goodman, attorney-at-law and mastermind of legal brilliance, on behalf of the humble but feisty David Rosencrantz, submit this amicus curiae brief to the esteemed Supreme Court. This Court, known for its unflinching pursuit of justice, now has the opportunity to rectify a grave miscarriage of justice perpetrated by the lower courts.

SUMMARY OF ARGUMENT

The case before this honorable Court presents a classic tale of a downtrodden individual, David Rosencrantz, pitted against the formidable Goliath that is the United States government. Rosencrantz, a simple taxi driver trying to earn an honest buck in this dog-eat-dog world, has been unjustly accused of tax evasion. However, this Amicus respectfully submits that the government's case is nothing but smoke and mirrors, an elaborate scheme to crush the dreams of an honest, hardworking citizen.

ARGUMENT

I. Violation of Due Process

The lower courts, in their zealous pursuit of victory, trampled upon the sacred principles of due process. Mr. Rosencrantz was denied his fundamental right to a fair trial when the government's star witness, a notorious jailbird with a penchant for singing, was allowed to testify without any corroborating evidence. The government's reliance on the word of a jailhouse informant, who likely had ulterior motives, is an affront to the Constitution.

II. Reasonable Doubt

Ladies and gentlemen of the Court, this case is riddled with reasonable doubt like a cheap suit in a thrift store. The government's case rests on flimsy evidence and a convoluted web of speculation. The alleged tax discrepancies are based on questionable calculations and fuzzy math, all while conveniently ignoring legitimate deductions and expenses rightfully claimed by Mr. Rosencrantz. Reasonable doubt casts its long shadow on this entire proceeding, and it is the duty of this Court to bring it into the light.

III. Selective Prosecution

This Court has never turned a blind eye to injustice, and I implore you to scrutinize the government's motives. Mr. Rosencrantz, an unassuming member of society, has become the target of selective prosecution. Why, might you ask? It's because he dared to stand up against the establishment, questioning the powers that be. This case is not about taxes; it is about silencing dissent and punishing those who refuse to play by their rigged rules.

CONCLUSION

In conclusion, the government's case against David Rosencrantz reeks of a witch hunt, an attempt to crush the spirit of an ordinary citizen who dares to question authority. The violation of due process, the presence of reasonable doubt, and the specter of selective prosecution are grave injustices that cannot be ignored. This Court, armed with the sword of justice, must rise above the fog of deceit and deliver a resounding verdict in favor of Mr. Rosencrantz.

For these reasons, I, Saul Goodman, implore this Court to right the wrongs committed by the lower courts and restore faith in the sanctity of our legal system.

Respectfully submitted,

Saul Goodman, Esq. Attorney for Amicus Curiae Saul Goodman's Legal Advocacy Clinic

karaterobot• 3 years ago

It's not always clear to me why people downvote things on HN, but a lot of the time I find it's because they misunderstood them.

blibble• 3 years ago

and it'll still generate bullshit

the technology is not capable of knowing when it is wrong

babelfish2• 3 years ago

Don’t use GPT for fact finding. Use it for synthesis when you already have the facts.

_cenw• 3 years ago

Or use it for fact discovery if you're not well-versed in a field. But always check if it hallucinated reasonable sounding gibberish.

ttul• 3 years ago

Or use it to retrieve alleged facts from its own output, pass those alleged facts to another tool (like a legal search engine) for verification, and then get GPT to edit its output accordingly…

whatever1• 3 years ago

The thing is that if you are not an expert in the field you cannot tell gibberish from legit facts. Especially if the writing style and grammar are top-notch.

I know it is a bad bias, but we typically associate good and clear writing with legitimacy. Here we have chat GPT, that can do exactly that, but spit out complete bs.

croes2• 3 years ago

Don't take something as fact without at least checking sources.

Couldn't be too hard for court decisions.

Vespasian2• 3 years ago

Yes he is guaranteed to have access to some case library software.

Why he didn't verify the existence of these cases spit out by a tool he used for the first time is beyond me. It would have taken him a few minutes at most to discover the fake ones and skimming the bogus references (assuming some legal competence)

rhino369• 3 years ago

How do you do that? Is there a tutorial?

babelfish2• 3 years ago

Three years of law school?

entwife• 3 years ago

I'm only peripherally involved in the law field, and I am aware that ChatGPT and similar do not consistently provide correct legal citations. It also gets things wrong some of the time; the only way to tell is to be an expert or to look up the facts yourself. Don't use ChatGPT for legal writing, nor for anything else that requires accuracy.

mountainb• 3 years ago

When it's correct, it can be a good search aid. But for a lot of things, it is just incorrect with high levels of confidence.

You can also ask it Bluebook questions and it will often get the right answer. At other times, it will get the right answer but cite to the wrong rule (not that it matters that much).

Another issue is that it can cite to the correct case, but misunderstand what it is citing to. You can be really specific and ask something like "what is the x-factor test from Doe v. Doe" and it will get three factors correct and invent the other two.

The thing with law, though, is that there are often already many quick reference materials that have already been extensively published that will get you the answer you are looking for more quickly than you can get it through either search or a chat interface. Many state bar associations make available the equivalent of a "practice area in a box" full of checklists, templates, and other material geared towards making it possible for you to start working in that area almost immediately.

I have had it be useful in course correcting my research in an unfamiliar area of law. I was wasting a lot of time reading secondary sources and cases that were not relevant to my problem because I knew nothing about that area of law and my search queries were just leading me in unproductive directions. ChatGPT pointed me towards a more relevant case that opened up the rest of my research for me using conventional tools like Westlaw. It saved me a lot of time. But I did not use it at all for the final work product and never used it blind without looking at a source.

visarga• 3 years ago

That's right, generative model outputs are worthless unless checked by a human. You are using it right. For the moment I don't think there is any single domain where AI can work on its own, autonomy was reached in 0% of fields. That makes me think the removal of the human in the loop will take a long time. We are still safe, AI will be our sidekick.

It's crazy how AI seems to progress at incredible speed and yet we don't get closer to full autonomy anywhere. It's as if we discover new problems at the same speed we are solving them. Just 5 years ago nobody would think hallucinations will become a central issue in AI, we might discover other unknown unknowns that hide in our future.

im3w1l• 3 years ago

I've been thinking that it's funny how these AI tools are framed as assistants, but it seems they are actually the opposite. They are great at big picture stuff but sloppy when it comes to details. So the more logical division of labor is to make the human the assistant.

montroser• 3 years ago

Mr. Schwartz said that he had never used ChatGPT, and “therefore was unaware of the possibility that its content could be false.”
He had, he told Judge Castel, even asked the program to verify that the cases were real.

These two ideas are incompatible with each other. You can't claim that you didn't know to question the source, and then also that you questioned the source, even if it was done in the least effective possible manner.

NeoTar• 3 years ago

Perhaps logically incompatible, but within legal proceedings you are allowed to use what is called alternative pleading, or alternative defence. https://en.m.wikipedia.org/wiki/Alternative_pleading

To quote Richard "Racehorse" Haynes in the Wikipedia article:

"Say you sue me because you say my dog bit you. Well, now this is my defense: My dog doesn't bite. And second, in the alternative, my dog was tied up that night. And third, I don't believe you really got bit. And fourth, I don't have a dog."

So, here the defence is:

* I didn't believe the content could be false,

* Even if it is legally determined that I (beyond a reasonable doubt) knew the the content could be false, I asked the program to verify that the cases were real.

There are more details in the wikipedia article, but I believe this is legally valuable because a defendant is required to legally enter a defence and cannot easily change this.

codeflo4• 3 years ago

I don't even think it's logically incompatible. Isn't this just ((A → B) ∧ (¬A → B)) → B?

piaste2• 3 years ago

While I have no reason to doubt that it's valid to argue "A || (!A && B)", in this particular case B => !A.

When the lawyer brought evidence that he had tried to verify the information (B), that evidence itself automatically disproves his plead that he didn't think it could be false (A).

(Note that didn't have to necessarily be the case: for example, if he had claimed that it was his assistant who asked the verification question, rather than he himself.)

So, unless he made the second plead and brought the evidence at a latter time, shouldn't he have just skipped A altogether, and preserved his credibility? (And possibly avoided a perjury charge? IDK if in an American court both claims would be sworn statements)

shalalalaw• 3 years ago

We're a very tech forward law firm, and we're bullish on AI. The issue is that lawyers are traditionally tech illiterate, and they treat Gen AI like a search engine that puts results in narrative form. Realistically, I think AI generated motions and contracts are the future, and this instance will be looked at by every tech averse lawyer to try and stymie progress in the field. These lawyers deserve their sanctions for being so reckless with things they don't understand, but rather than take away that lawyers need to learn tech, lawyers will say tech is bad. I almost wish this was non-news so it wouldn't further push the legal industry into the past, but those clients were wronged and I guess people need to know what to beware of when hiring a lawyer.

Personally, we get really good results from using AI, it's already present in all of our processes, but we tell it what to generate, rather than rely on it to know better.

codeflo4• 3 years ago

It's not just lawyers who think that ChatGPT is a search engine. I've observed this many times in my vicinity, people from all walks of life think that Star Trek is here and computers now respond accurately to natural language queries. For non-techies, "just asking the computer" is so much more convenient than translating your question into traditional search queries.

So I guarantee you that stuff like this is happening daily across all industries. Depending on the profession, people will lose money or get hurt as a result of someone blindly trusting this technology. I can't prove it, but statistically, that's basically a certainty.

In my opinion, you can't overstate the importance of articles like this, which point out the limitations and highlight the dangers. I'm also against banning. But lay people need to be informed what ChatGPT is and is not, and OpenAI won't do it because they want to ride the hype train.

gpm• 3 years ago

In local-to-me politics we have a report on changing the admissions process for specialty programs in the public school system - which had a bunch of fake citations and people suspect was written with the "aid" of ChatGPT.

https://www.thestar.com/news/gta/2023/05/26/tdsb-fires-resea...

Xenoamorphous• 3 years ago

It's not just lawyers who think that ChatGPT is a search engine.

Let’s not forget that Google often puts incorrect information in their snippets/factboxes or whatever they call them.

cratermoon4• 3 years ago

Computer, what's the formula for transparent aluminum? Seriously, I got ChatGPT to spit out a scientific-seeming paper on the formula and manufacturing process for transparent aluminum. It did note that there's a real thing, aluminum oxynitride, which is the closest thing we have to the Star Trek material. It even wrote the following abstract, based on my prompt:

This scientific description provides an overview of the formula and manufacturing process of transparent aluminum, a material used in applications where both structural strength and transparency are required. Transparent aluminum finds extensive use in diverse fields, including public aquaria, where it allows for the display of large marine organisms. The description outlines the chemical composition, key properties, and the manufacturing steps involved in creating transparent aluminum.

Whether or not the six-step manufacturing process it came up with is correct or not, I haven't the expertise to say.

fennecfoxy4• 3 years ago

Idk it's pretty reliable even now if you chuck a vector db of "knowledge" in and inform the GPT in the overall prompt that it is must not go outside the bounds of knowledge provided with the user's query, and that if no hard knowledge is provided that it should respond along the lines of "I don't know" (or search the internet and parse results for them).

I imagine at some point this behaviour will be built in. People are treating GPTs like they're knowledge bases, but they're not. The fact that they can answer simple or complex questions correctly is only a byproduct of the transformer being taught how to string language together.

It's like an artist who learns to draw by making paintings of raccoons. You can't then ask them "what do raccoons like to eat?" or "what foods are poisonous to a raccoon?" just because they learnt to draw by painting raccoons. This is how people are treating GPTs atm. They believe in it because they ask the artist "what colour fur do raccoons have?" and because the artist can answer that correctly, they assume all other answers are factual.

ouid2• 3 years ago

ai generated contracts are the future.

Man, why do you guys get paid so much? Contracts need semantic correctness, ie the verification of the logical consequences of natural language. This is an AGI determining problem. At the point this exists, humans are basically obsolete as workers, and you don't have to worry about your law firm keeping up anymore.

hnfong2• 3 years ago

Let me get this straight, you're saying if AI can be a good lawyer AI can do anything humans can do?

So you're trying to say that lawyering is the hardest job in the world and only the smartest people can do it...?

vidarh2• 3 years ago

Contracts have lots of standard clauses or near standard variations of such, and as a consequence plenty of "dumb" template based generators already exists, and you may come across contracts that have just the occasional line manually written.

At least one company already integrates or is about to (not sure if it's in production yet) LLMs in theirs to do effectively smarter completions.

It won't be fully automated any time soon, but it will certainly eat into a lot of the simpler work.

drumhead• 3 years ago

You sound like an AI generated post.

Der_Einzige• 3 years ago

... retrieval augmented search is here today and is available in ChatGPT with plugins or integrations with vectorDB systems. A lot of AI systems are "search engines that give you narrative outputs"

Zafira2• 3 years ago

[L]awyers are traditionally tech illiterate

This is the main reason I think disbarment as the punishment in this specific instance may not be fair. There are people who are unaware of the limitations of these systems and the risk of these confabulations occurring.

While I don’t think disbarment is inappropriate, I would rather see the New York State Bar use this to require some better understanding of these emergent technologies or even better have all the State Bars start discussing some standardized training about this because it’s easy to see a person trying to treat this as LexisNexis.

NoMoreNicksLeft4• 3 years ago

Is disbarment about fairness? Is the primary goal of such proceedings to rehabilitate and apply a sort of justice?

Certainly, civil and criminal courts have those as their raison d'être. But I thought licensing boards had an entirely different purpose. If I surgeon was a good guy who genuinely wanted to help people and who didn't engage in any sort of malfeasance... but even so, he just kept slicing aortas open accidentally through incompetence, the board should say "aw shucks, he's had some bad luck but he really wants to heal people".

This is the same. The court system is replete with circumstances where a client does not get a second chance at pursuing justice. A lawyer that fucks that up, even if doing so in good faith, leaves them with zero remedies. This might have been a bullshit "Slippin' Jimmy" case this time, but the stakes could've easily been higher.

I don't think I want to live in a world where fairness plays any part in the decision by the bar on this matter.

ryandrake• 3 years ago

If your doctor asked ChatGPT to tell him how to remove your appendix, followed the directions, and subsequently removed a kidney instead, would you want him to lose his medical license?

nocoiner• 3 years ago

Out of curiosity, why do you think AI generated contracts are the future? Do you draw a distinction between contracts generated by AI and, say, contracts “generated” by first-year associates (i.e., using precedent to generate a first draft appropriate for the deal that’s then iterated by more experienced lawyers)?

Also, how is this incident going to push the legal industry further into the past? Do you think lawyers are going to, like, stop using email because of this?

SanderNL• 3 years ago

Lawyer uses completely new tech without so much as glancing over its properties and failure modes, also known as being completely incompetent.

Breaking news: the results are not fantastic.

dpflan• 3 years ago

Events like this are needed to show to society the flaws of the technology and possible misuse. They can then lead to regulations or guidelines, or even lawsuits that can further define how and when this technology can be used.

naet• 3 years ago

In a thread on Reddit I saw some people who were using ChatGPT as their personal lawyer around some tech issues, and the community seemed to have absolute confidence it was providing good advice. I tried to comment that I wouldn't trust GPT as a legal reference and got downvoted into the deep negatives.

I'm guessing we will see a lot more of these type of issues. ChatGPT is sometimes useful but sometimes full of misinformation and many people don't seem to be wary enough of the potential for error.

misnome• 3 years ago

Unpossible! I’m a 20x coder who spends hours a day writing regex, tried asking it a high-school level coding question, and it answered perfectly. I’m 100% sure that its just as accurate everywhere else.

wseqyrku2• 3 years ago

They should've used GPT-4.

wseqyrku2• 3 years ago

(The article does not say anything about this but I'd guess if they are as clueless as they claim to be by calling it "a source that has revealed itself to be unreliable", they might as well be using the free version without the knowledge of the latest version.)

codeflo4• 3 years ago

It's uncanny how precisely you've put your finger on what the actual issue is here.

hartator• 3 years ago

Judge Castel said in an order that he had been presented with “an unprecedented circumstance,” a legal submission replete with “bogus judicial decisions, with bogus quotes and bogus internal citations.” He ordered a hearing for June 8 to discuss potential sanctions.

Disbarment should be a no-brainer and a minimum.

Just “Southern China Airlines” should have raised eyebrows. This lawyer has shown obvious disrespect to the court, the court time, and to his client.

paul_f• 3 years ago

The lawyer should lose his license. Imagine they had turned in this brief and follow up and ChatGPT had not been involved. Instant disbarment. ChatGPT is not an excuse for a professional to produce nonsense in front of the court. Bye bye.

psychphysic• 3 years ago

Disbarment seems excessive to me.

I don't think this is indictiative of a wider issue. Or likely to be repeated substantially.

In terms of severity it's painfully foolish but then again ChatGPT is a totally new tool and a lot of people will be caught off guard.

I am stunned the lawyer didn't at least look up the case notes or even prepare a pocket brief if he believed they were real but hard to find

not_a_shill2• 3 years ago

If disbarment is the minimum, what's the maximum? I'm unfamiliar with other cases in which lesser offenses have led to disbarment.

bombcar3• 3 years ago

I wouldn’t be surprised if prison is on the table. Intentionally filing bullshit to the court can get you some penalties indeed.

eastbound2• 3 years ago

I suspect a lot of lawyers should be disbarred, GPT aside. They don’t always care about their customers. I’ve often been more expert than my lawyers in France, and I’ve seen at least one guy going to prison where the lawyer, publicly shamed by a dozen of youtubers for not actioning the various correct levers, told the excuse that he “only had an hour to review the case”.

What are we paying for, if the guy spends 4 months in prison before the faulty judgement being overruled, if the lawyer says he didn’t even work on the file.

When you hire a lawyer, you have no guarantee he will work for you.

LelouBil• 3 years ago

Do you have a name or links for the incident you mention ?

raverbashing• 3 years ago

The title of the made up case seems to be "China Southern Airlines", which is correct. But it is misquoted on Document 31 (Order to Show Cause) as China South Airlines

croes2• 3 years ago

Could be a clever method to make programs like ChatGPT untrustworthy for potential clients.

julienchastang• 3 years ago

Related article: "End of the Billable Hour? Law Firms Get On Board With Artificial Intelligence, Lawyers start to use GPT-4 technology to do legal research, draft documents and analyze contracts" [0]

Critical thinking skills are more important than ever in the age of AI. Used correctly, ChatGPT(4) can sometimes be a huge time saver, but you cannot believe all the bullshit it serves you.

[0] https://www.wsj.com/articles/end-of-the-billable-hour-law-fi...

isp2• 3 years ago

I've been following this from legal circles

Original court documents: https://www.courtlistener.com/docket/63107798/mata-v-avianca...

The lawyer didn't only cite "bogus" cases, but when challenged attached entire "bogus" case contents hallucinated by ChatGPT (attachments to #29 on link above)

In the second #32 affidavit, there are screenshots of ChatGPT itself! https://storage.courtlistener.com/recap/gov.uscourts.nysd.57...

A legendary example for the legal risks of hallucination in LLMs https://en.wikipedia.org/wiki/Hallucination_(artificial_inte...

_-____-_• 3 years ago

That the screenshots are from the mobile website for some reason makes this look even worse.

spondylosaurus• 3 years ago

It does. It's like this lawyer is charging you his hourly rate just to prompt ChatGPT while he's on the toilet.

asmithmd13• 3 years ago

Thanks for the great context. The lawyer should be disbarred. He doubled down when he was caught, and then blamed chatGPT. What do you bet he was trying to settle really quickly to make this all go away.

Here is the direct link to the chatGPT hallucination the lawyer filed in response to the judge's order to produce the actual text of the case: https://www.courtlistener.com/docket/63107798/29/1/mata-v-av...

BaseballPhysics• 3 years ago

Did he "double down" or did he genuinely not understand that ChatGPT was making stuff up the whole time?

isp2• 3 years ago

The above link wasn't the only hallucination(!)

The lawyer kept digging the hole deeper and deeper, and (as a non-expert) I agree that it seems that the lawyer is at serious risk of being disbarred.

Interesting documents are from #24 onwards:

- #24 (https://storage.courtlistener.com/recap/gov.uscourts.nysd.57...): "unable to locate most of the case law cited in Plaintiff’s Affirmation in Opposition, and the few cases which the undersigned has been able to locate do not stand for the propositions for which they are cited"

- #25 (https://storage.courtlistener.com/recap/gov.uscourts.nysd.57...) & #27 (https://storage.courtlistener.com/recap/gov.uscourts.nysd.57...): order to affix copies of cited cases

- #29: attached the cases - later revealed to be a mixture of made up (bogus) for some, vs irrelevant for others

- #30 (https://storage.courtlistener.com/recap/gov.uscourts.nysd.57...): "the authenticity of many of these cases is questionable" - polite legal speak for bogus. And "these cases do exist but submits that they address issues entirely unrelated to the principles for which Plaintiff cited them" - irrelevant. And a cutting aside that "(The Ehrlich and In re Air Crash Disaster cases are the only ones submitted in a conventional format.)" - drawing attention to the smoking gun for the bogus cases

- #31 (https://storage.courtlistener.com/recap/gov.uscourts.nysd.57...): an unhappy federal judge: "The Court is presented with an unprecedented circumstance. A submission filed by plaintiff’s counsel in opposition to a motion to dismiss is replete with citations to non-existent cases. ... Six of the submitted cases appear to be bogus judicial decisions with bogus quotes and bogus internal citations" ---- this PDF is worth reading in full, it is only 3 pages & excoriating

- #32 affidavits, including the ChatGPT screenshot

- #33 (https://storage.courtlistener.com/recap/gov.uscourts.nysd.57...): an even more unhappy judge: invitation for the lawyer & law firm to explain why they "ought not be sanctioned"

bandyaboot• 3 years ago

I’m quite amused that ChatGPT hallucinated a frivolous lawsuit brought by someone who was denied an exit row seat.

smsm42• 3 years ago

I wouldn't be surprised if it were based on a number of real lawsuits of the same exact nature.

Obscurity4340• 3 years ago

Wouldn't "confabulation" be a better term for this?

b1ue64• 3 years ago

https://archive.ph/tHuXv

squarefoot• 3 years ago

Could he sue his own lawyer now?

NelsonMinar• 3 years ago

OpenAI deserves some blame here. Their product UI does a very poor job conveying that it will just make things up when asked for specific information. And in the conversation there's no particular hint that the confidently stated "facts" are not in fact true. A large number of people get tripped up by this when first using ChatGPT. I did myself, it took me a day or two to realize the papers or web pages it was claiming existed didn't. It's pathological.

scrollaway• 3 years ago

This drawback is presented front and center every time you open ChatGPT. A lawyer of all people should have known better.

joshka• 3 years ago

@dang - the earlier article on reason.com is more thorough - perhaps merge these two posts?

https://news.ycombinator.com/item?id=36092509

https://reason.com/volokh/2023/05/27/a-lawyers-filing-is-rep...

acd• 3 years ago

I think it should be required to state if a written statement is written by a GPT.

"This text section is written by GPT provided by company XYZ"

Then humans would know that a particular part is written by an AI.

GaggiX• 3 years ago

Then humans would know that a particular part is written by an AI.

Humans would know if a part is written by an AI or not only if there is a way to enforce the rule.

Reason077• 3 years ago

"Estate of Durden v. KLM Royal Dutch Airlines"

Tyler Durden? LOL, you couldn't make this stuff up. Unless you're ChatGPT, obviously.

Am4TIfIsER0ppos• 3 years ago

I can't wait for chatgpt equivalent of citogenesis (https://xkcd.com/978/) to happen.

adql• 3 years ago

Can't bullshit people who specialize at calling the other side's bullshit.

Not that there can't be use for it,I'm sure it could work as great assisted search engine if it was made to cite sources.

simonw• 3 years ago

I tried pulling together a full timeline from the various documents, it's a fascinating story: https://simonwillison.net/2023/May/27/lawyer-chatgpt/

jhncls• 3 years ago

Even if the lawyer would have carefully checked and corrected ChatGPT's answers, isn't he going against his professional duty by submitting private information to a website that doesn't ensure confidentiality?

cratermoon4• 3 years ago

I was able to coax ChatGPT into inventing an entirely fake trend in cocktails and distilling, complete with stories about newly-opening bars and made-up concoctions (including recipes) supposedly created and served at these bars.

andrewmg• 3 years ago

Some context: any litigator will have access to Westlaw or Lexis-Nexis to look up and verify cited authorities like cases. It’s considered bad practice, at best, to cite authorities that one has not reviewed—for example, case citations drawn from a treatise or article.

As a practical matter, it is inconceivable to me that the attorney here, at least upon being ordered by the court to provide copies of the cases he cited, did not look them up in West or Lexis and see that they don’t exist. That he appears to have pressed on at that point, and asked ChatGPT to generate them—which would take some pointed prompting—was just digging his own hole. That, more than anything, may warrant professional discipline.

nightowl_games• 3 years ago

The precision with which you wrote this is refreshing. I'm guessing you're a lawyer?

There are interesting parallels between lawyering and programming.

I'm often surprised at how poorly many programmers write English.

I think I'd have been a good lawyer.

dpifke• 3 years ago

Previous discussion: https://news.ycombinator.com/item?id=36092509

nunez• 3 years ago

I guess it will take someone dying by following ChatGPTs "advice" to make people think twice about using these LLMs!

fennecfoxy4• 3 years ago

GPT4 hooked up to a legal DB and instructed that if no knowledge from the DB can be referenced to respond negatively would've solved this completely.

Even a hard link to rewrite the prompt from "use provided knowledge to answer the question" to "no knowledge could be found, inform the user of this fact in a way that keeps the conversation fluid. Recommend that they search other sources themselves." would be more reliable.

The problem, as always, isn't the tool. It's the tool using the tool.

krimbus• 3 years ago

Could we have certainty that an LLM would comply though?

I can only imagine this happening with external constraints such as parsing the output to perform a DB check, so I'm curious.

fennecfoxy4• 3 years ago

Hmmm, I suppose we could, if there was a focus on it.

However it's always interesting to watch how people react: * Traditional programming creates very strict functionality but is not flexible/is rigid. * Okay, so let's train models to act like biological life, which is super flexible but has a chance to make mistakes.

But then people complain about the latter making mistakes. Realistically we can combine the two to ensure that what a model is saying is based on fact, though.

Easiest way would be just for people to interact with an LLM to browse hard facts and then double check via sources. But relying on humans not to be lazy is just as good as relying on an ML model to act like traditional programming, so it's probably better to train our wild model with as many built-in constraints as possible and then decorate those with more traditionally programmed hard limits.

It's really hard though because of the output data. We're going from a binary output of 1/0 for traditional programming to a continuous output of 0-1 for models; if I ask it to summarise something, whether it's correct or not is not only based on whether it summarised based only on the original text, but also down to the reviewer's individual biases and wants/needs as to whether the summarisation is sufficient.

The continuous nature of an LLM's response does make it difficult to determine if it's sufficiently factually correct, though. Because if it's not repeating word for word, you need to be able to parse the output in order to check it...using another LLM (so compounding error...)

A man sued Avianca Airline – his lawyer used ChatGPT

Related Stories