IBM’s Watson Now A Second-Year Med Student

http://blogs.forbes.com/bruceupbin/2011/05/25/ibms-watson-now-a-second-year-med-student/

by jmjerlecki • 15 years ago

117 74 15 years ago

blogs.forbes.com

ryanlchan• 15 years ago

Is this the beginning of companies using web 2.0 tech to solve 'big problems'? Watson might be the first step towards using the social graph and big data for things other than selling more widgets.

I'm keen to see how medical professionals start using this tool though. Professional pride and the immense amount of inertia present in medicine could hamper adoption. Worse yet, I can only imagine the level of legal liabilities using a system like this could incur. I'd love to see this adopted and think it could radically reduce costs and increase productivity in the medical field, but I'm going to resign to simple cautious optimism.

nradov10• 15 years ago

Clinical decision support systems don't really create any additional legal liability. They just present the user with possibilities. The physician is still legally responsible for making a diagnosis, prescribing a course of treatment, and carrying out medical procedures.

Jd• 15 years ago

Doctors virtually all use lightweight iphone or webapps (e.g. Epocrates) to handle drug information, etc. Like the rest of the world, they no longer memorize massive sets of evolving information, they simply walk around with their Iphone/device of choice and look it up.

The providers of such software now and in the future will of course employ teams of lawyers to craft appropriate statements like, "use this as a guide only and not as the basis for a diagnosis or prescription," which of course won't prevent doctors from using it as the basis, but it will keep a healthy sense of legal limbo which exists in many other fields.

Basically, I think this doesn't remove the doctor as the interface person, but it allows fewer doctors to see more patients and to streamline the process for many medical problems with (presumably) better outcomes on average. I'm usually not keen on the impersonal nature of modern medicine but what works, works.

yid• 15 years ago

There isn't too much "Web 2.0" tech behind building something Watson. Perhaps the use of ontologies and semantic web datasets, but in academic circles at least that's informally referred to "web 3.0" (that, too, is a bit of a misnomer because Cycorp has been building semantic ontologies for a long, long time).

robobenjie• 15 years ago

When contemplating what jobs were going to be replaced by robots, I never thought of doctors, but it now seems quite plausible, even probable.

(Not that doctors are going to be gone completely, just that their jobs will be forever changed the way that CNC has completely changed the job of a machinist)

c0riander2• 15 years ago

I wonder if doctors would become more of a service industry -- doing "patient care" (as customer service) instead of "disease care" now. Could be an improvement for the experience of receiving healthcare; definitely interesting to think about!

_delirium• 15 years ago

The doctors I know tend to think that it's at least 90% that way already (and has been for a while), at least if you're a primary-care doctor seeing patients in an office, rather than a surgeon or some kind of specialist. The typical patient isn't presenting with a rare disease worthy of a medical TV drama, and instead the main problems are patient-interaction ones: getting the right information out of them, picking treatments that the patients will be satisfied with and follow, figuring out what complicating lifestyle factors they might have, building trust with the patient and family members, etc. With the increasing influence of evidence-based medicine, where doctors are supposed to follow experimentally validated rubrics, the diagnosis itself is often semi-mechanical anyway even when done by a human.

Entaroadun3• 15 years ago

I'm sure most people wouldn't expect doctors to be replaceable. But look at the process to become an MD and think about the amount of pure memorization involved. Then it all makes sense.

rao• 15 years ago

Once a robot doctor is ready, imagine how easy would it be to create another one. Replicate the hardware, software and copy all the data. Compare this with the amount of time it would take to make a human doctor.

lgeek• 15 years ago

Computers are commonly used to assist in diagnosis. You might want to look up computer-assisted diagnosis[1]. If I am to believe my machine learning lecturers, their use is actually a bit more widespread than what the wikipedia page describes.

Anyway, I think it is likely to see a transition to computers diagnosing and doctors just checking the results; at least for some areas. If you have a diagnosis and patient information I guess it isn't that difficult to come up with a treatment. I think that doctors themselves basically memorize decision trees.

[1] http://en.wikipedia.org/wiki/Computer-aided_diagnosis

nradov10• 15 years ago

There is a saying in medical diagnosis: "When you hear hoofbeats behind you, don't expect to see a zebra". https://secure.wikimedia.org/wikipedia/en/wiki/Zebra_%28medi... Most patients have common problems that are fairly easy for experienced physicians to diagnose. So there is no point to entering a bunch of data into Watson just to get back the same answer you already know. But it could certainly be helpful for those rare edge cases where the physician can't figure out the "zebra", or where the patient hasn't responded as expected to treatment.

nostromo2• 15 years ago

I think this is backwards. (After thoroughly testing of course) I'd rather see Watson be the first-tier of diagnosis, backed up by a doctor when Watson isn't so sure of an answer, as opposed to Watson just helping befuddled doctors. This could really bring down medical costs at scale.

astrofinch2• 15 years ago

And also possibly make it harder for doctors to, say, recommend a more expensive procedure because their hospital gets paid more.

nradov10• 15 years ago

Well then who is going to do the clinical observation, coding, and data entry? That work doesn't get done for free and in most cases it would actually take the healthcare providers more time to do.

astrofinch2• 15 years ago

I know this might sound a tad heretical, but if Watson gets to be as good as or better than a doctor, do we really need doctors anymore?

The theoretical upper bound on Watson's ability is a lot higher than the theoretical upper bound on a doctor's, since it can make inferences based on a vastly larger data set.

carbocation2• 15 years ago

For emerging diseases, advanced malingering, etc, you definitely need real life actual breathing human beings. Otherwise, sure, physicians are just as theoretically replaceable as any information-as-a-service workers might be.

Edit: I didn't realize this was even a controversial statement to make. First, to be clear, when I'm in the hospital and say "doctor" I'm usually thinking non-surgeon, as I am here. Second, you will always need supervision of the machines, just not 1-to-1 supervision. Not every interaction brings up ethical questions, for example. Sometimes someone really just has a viral URI.

DocSavage2• 15 years ago

We'd still need surgeons until the robots are good enough. We'd need radiologists until they can add decent computer vision. Ditto for pathologists and other vision-intensive specialties. Orthopods are very hands-on and collect information through channels other than words. So there's a suite spot in the spectrum of doctors for something like Watson. I'd wonder about the ability to ask questions and ferret out socially-obvious problems like broken arms from abuse if Watson were a first provider.

Presumably you'd couple something like Watson with a world-savvy human physician assistant that acts as its eyes, ears, and touch.

itg• 15 years ago

For diagnosis maybe not, but we will still need surgeons, unless IBM comes up with a really good autonomous robot that can perform surgery.

lmkg• 15 years ago

Watson may be able to recommend you get surgery, but he won't be able to perform the surgery himself. He's only moving in on one aspect of medicine. Matching symptoms to diagnoses is the thing that machines are most clearly suited for in the entire medical profession, because of the vast volume of possible measurements and diagnoses, and because humans' various cognitive biases don't work well with the probabilistic nature of the work. That's still a rather narrow range of work. Humans will remain superior at performing medical work for quite a while. And until true Strong AI, humans will also remain superior for quite a large amount of gathering information that Watson requires, especially psychological measurements, and for making decisions that involve quality-of-life and other ethical considerations beyond just probability of success.

tl;dr Watson can, and will, exceed doctor's capability for a certain, somewhat narrow, range of their function. Doctors will still be necessary for the other things that they do.

derleth• 15 years ago

Can't sue a computer.

Legally, someone has to be responsible for the healthcare provided to the patient. The patient shoulders some of it, true, assuming they're basically competent adults and not severely impaired. (That's why powers of attorney exist, and why physicians are empowered to 'fire' non-compliant patients.) However, the MD is the one who has had the training and who is legally authorized to exercise medical judgement (as opposed to nursing judgement, and no, I'm not going to even try to get into where the arbitrary line between them falls), so the MD is the one who ultimately decides the treatment strategy, which may involve referrals to other MDs, who also must be legally capable of exercising medical judgement.

So, legally, there has to be someone to sanction (threaten with malpractice lawsuits and loss of licensure, among other things) and it's probably better for everyone involved if that person has personal contact with the patient.

blahblahblah• 15 years ago

Watson isn't a replacement for the physician. It's a physician's extender, a tool that allows a physician to "see" more patients in a day and provide diagnostic results in a more timely manner. The physician will still be the one responsible for the final judgment call.

BTW, it would have been nice if the journalist had bothered to consistently spell the doctor's name correctly (it's Siegel, not Siegal).

hncommenter13• 15 years ago

This is one of my favorite sayings, so I feel compelled to minorly correct it here. The full saying is "When you hear hoofbeats in Texas, think horses not zebras." The "in Texas" is a key piece of context, as there are some places where zebras might well be more common than horses. The point being that one should suspect and rule out the most common differential diagnoses first.

jurjenh• 15 years ago

I read your statement to assume that entering data into Watson is going to take longer than "entering" that same data into your GP. How did data entry for Jeopardy work?

This would mean that the main issue in adoption of Dr Watson (Hmmm, haven't I heard that name before?) would be the data entry - obviously a pain point in many digital services. If an effective answer is found, and I would suspect that this should soon happen, then the argument becomes more a matter of whether you prefer a subjective opinion from your GP (based on his/her current thought processes / focus) or an objective one from a digital process.

I think I'd prefer the digital process, once the main issues are ironed out.

cynest• 15 years ago

Iirc, Watson used voice input.

nradov10• 15 years ago

Don't hold your breath. This has been an area of active work by many of us for decades. We have some systems that work reasonably well for a few limited cases but the general case of clinical data entry won't be solved any time soon. It's a genuinely hard problem that those without experience in the domain tend to underestimate.

Doctors have only a few minutes for most patient encounters. They simply have no extra time to do data entry. You can delegate some of that work to clerical staff or allied health but there are limits to what they can do, and they don't work for free either.

robflynn• 15 years ago

This reminds me of some work I did a few years ago in the oil and gas industry. Not nearly as advanced as this, but we essentially read data from sensors in off shore oil rigs and ran it through our system. We had some pretty nifty pattern recognition stuff in place that would learn from previous data in order to become advanced enough to predict imminent sand breaches during drilling.

Given significant warning, it could save the drilling companies lots of money.

Katrina ended up severely damaging the rig we were using for testing and I'm not sure we ever went anywhere with it after that.

We did do some similar work for detecting trace metals in oil sample analysis records in order to recommend maintenance and detect possible future equipment failures.

Anyway, interesting stuff here. I'd love to play around with it.

edit Sorry about the double post. I think I got it removed. Connection was a bit flaky.

spc476• 15 years ago

It looks like Watson is shaping up to the 21st century version of Mycin (http://en.wikipedia.org/wiki/Mycin).

janesvilleseo• 15 years ago

I can see Watson being very helpful to Sergey in his pet project - http://www.nytimes.com/2009/03/12/business/12gene.html

PostOnce• 15 years ago

It seems it must be a tool in a tool chain, rather than a physician replacement. Tests must be run, and follow-up questions asked. Also, computers take everything literally, they don't say "Well, that's not what your face is telling me." or think the patient is lying, delusional, or misinformed. You can ask the patient on a scale of 1-10 of their certainty, but again, there are other problems.

"What are these red dots?" Also seems non-trivial to diagnose. They can be tested, or more questions can be asked of them, but this seems like an awful lot of work to be managed by a computer alone.

It's a good, or even great thing to research, but it seems far from implementation, and that's just from an engineering perspective, imagine the legislative hoops that will need to be leapt through!

sonoffett• 15 years ago

I could definitely see it not as a replacement for a physician, but as sort of a safeguard to check if the physician's decision regarding treatment for a particular set of symptoms matches up watson's previously learned data. Watson wouldn't preempt the physician, but rather raise an alarm if something looks out of the ordinary.

thaumaturgy• 15 years ago

> "What are these red dots?" Also seems non-trivial to diagnose. They can be tested, or more questions can be asked of them, but this seems like an awful lot of work to be managed by a computer alone.

I don't entirely disagree with you, but I think you might be underestimating the capabilities of smart programmers, e.g., some of the stuff at the beginning of http://daniellefong.com/2010/02/11/how-law-shapes-the-busine...

If algorithms discussed in that blog post were used in one system (even with pictures taken from an Android device possibly), and that system were to communicate results to Watson ... I can't think of a reason why that wouldn't work.

Other than, y'know, patents.

CobraKai2• 15 years ago

As a specialist, I have to say: This makes me very glad I'm not a primary care provider. The doctors of the future will be evaluated on bedside manner and interpersonal skills - not on diagnostic ability or knowledge, because they'll all have access to Watson. Which is scary because smooth talking idiots can get into medical school, but get rapidly weeded out in the real world. It scares me that stupidity can hide behind this computing beast. And there will be situations where Watson cannot help (emergencies) where a doctor and his smarts are all he has to go by.

If Watson makes a wrong diagnoses and a patient dies, who will be responsible? The doctor? IBM? All it would take is one or two incorrect diagnoses leading to large malpractice suits to really put the medical profession on edge about this. Granted, many incompetent practitioners exist out there making all kinds of dangerous decisions, but this would really stick out in the public eye.

arohner2• 15 years ago

You don't really have to get the diagnosis correct, you just have to get the error bars correct, and then disclose all of that.

"Watson believes the best treatment is Foo. It has an X% chance of working, with a confidence interval of Y. The second most likely diagnosis is Bar, with an expected loss of Z if the diagnosis is incorrect. Which would you like to do?"

Besides, several specialties are already essentially classifiers (radiologists, anesthesiologists). If machines can't beat humans at classifying cancer from lung scans, they're not far behind. By the time Watson is in production, I bet a computer will be replacing radiologists already.

wensing• 15 years ago

This is cool. When I was at IBM in Austin last week, one of the IBM-ers mentioned that one of the first areas they hoped to use Watson was primary care. Amazing. I think I'd rather go to Watson than most GP's.

recampbell• 15 years ago

More details: http://www.ibm.com/developerworks/industry/library/ind-watso...

sungam• 15 years ago

As a physician I doubt this will have much of an impact on the cost of health care. The limiting factor here is not technological but cultural. In the majority of patient encounters a diagnosis has either already been made or is obvious. In principle workers could be trained in the basic clinical skills required in a short period of time and several supervised by a single doctor. There are several reasons that this does not happen to a great extent: (i) Patient's prefer to be assessed by a fully qualified doctor (ii) Medicolegal - who is responsible if a mistake occurs? (iii) Trainee doctors on a per hour basis are often cheaper than employing other workers to perform the same tasks. (iv) For whatever reason Doctor's tend to see patients faster and make decisions more rapidly than other staff e.g. nurses performing the same tasks.

kds• 15 years ago

"Answering the question “What’s wrong with this patient given these set of symptoms and this family history?” turns out to be a natural fit for the computer."

Well, wake me up when the natural-fit question becomes "What’s wrong with this planet given these set of eco-symptoms and this social history?"

I'd really like to see what the answer might be.

A Post Scriptum Edit/Addition Below ;)

The IBM Watson system that recently won on Jeopardy is thoroughly probabilistic and statistical --Norvig (see news.ycombinator.com/item?id=2591154)

In a future far, far away when humanity will have gained experience with millions of planets, stars, and human colonies and space-castaway societies - maybe then - IBM Watson will be able to answer my question...

Submerging in cryogenic sleep... see you later... <yawning>

JulianMorrison• 15 years ago

Put it on a public website.

IBM’s Watson Now A Second-Year Med Student

Related Stories