Claude thinks it's a tumor, radiologists disagree

https://twitter.com/misha_saul/status/1771019329737462232

by baxtr • 2 years ago

19 16 2 years ago

twitter.com

infecto• 2 years ago

1) We already know LLMs when given the opportunity, will often make up information.

2) I am not sure its surprising that a general purpose model is not great at reading MRI scans.

striking• 2 years ago

It's not surprising to you or me, but think of someone less technologically proficient. They won't know the difference, and if the AI complies with their request in a convincing way, they may very well believe it.

bthrn• 2 years ago

For some reason I read this and then think of Arnold saying, “it’s not a toomah!”

DeepYogurt• 2 years ago

Really contextualizes those stories that read as "we don't know how the AI found the medical issue, but it did!".

prashp• 2 years ago

Claude isn't an approved medical device so why is willing to provide a diagnosis?

minimaxir• 2 years ago

Because it isn't RLHFed to do so. It's a good example why alignment sometimes isn't a bad thing.

ChatGPT does specifically disallow medical advice and generally it is RLHFed not to provide it: https://openai.com/policies/usage-policies

SkyPuncher• 2 years ago

My friends aren’t licensed healthcare providers, but they’re more than willing to provide diagnoses.

rsynnott2• 2 years ago

What does ‘willing’ mean here? To be clear, these things are not people, or anything resembling people. They do not have will.

GaggiX• 2 years ago

To be honest Claude vision is not that great, it's probably Claude’s biggest weakness compare to GPT-4.

olliej• 2 years ago

Yeah, there was a paper about AI excellence in diagnosing radiology images a few (5-10 now?) years ago, that turned out to be largely driven by accuracy skewed bias in the input data (IIRC it was using the existence of lab-style lighting, rulers, etc as its diagnosis).

Chat-GPT and all the LLMs are doing essentially the same "predict the next work in a sentence" to sound as plausibly real as possible, which is essentially the same gimmick, only we call it hallucination due to VC funded AI firms need to pretend LLMs understand anything.

Krasnol• 2 years ago

There are much better AI approaches to the problem like: https://mediaire.ai/en/mdbrain/

resource0x• 2 years ago

"We should stop training radiologists now. It's just completely obvious that within five years, deep learning is going to do better than radiologists"

G.Hinton, 2016

viraptor• 2 years ago

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6483404/

The error rate produced by a deep learning based methods dropped below that achieved by human observers in 2015 for the first time

Essentially we just need good implementations at this point. We're there, just not widely deployed.

rvz• 2 years ago

On some days, Claude (and other LLMs) will hallucinate on one input and produce garbage and in the next update it will then believe something else in the next unpredictably.

Perhaps using general purpose LLMs for safety critical use cases without humans isn't a good idea. This is why we have a 'human in the loop' giving the feedback to resolve the trust issue that LLMs lack.

Maybe we should trust LLMs like Claude to pilot a plane without any human pilots on board and raise VC money on the idea on replacing all pilots since we keep having these conversations on LLMs being compared with humans and countless VCs screaming over robots replacing humans in all occupations and 'accelerating everything' which could not be more wrong.

rsynnott2• 2 years ago

Bloody hell, this sort of thing is going to get people killed. The magic box is not a person, let alone a qualified doctor. Do not ask the magic box questions about anything important.

cheema33• 2 years ago

My girlfriend has had some medical issues for a few years and none of the 3-4 doctors she has seen have been able to diagnose her. That is not to say that no doctor can. Just not the ones she has seen.

I do not believe in the infallibility of medical professionals. And I have seen AI totally get it wrong every day. But, I am of the opinion that we are not very far from the point where AI will be better at diagnosing issues than an average doctor. That isn't a high bar. An average doctor isn't very good.

I think AI is almost there already for an average programmer. An average programmer isn't very good either.

Claude thinks it's a tumor, radiologists disagree

Related Stories