AI tools have been built to catch Covid, but none of them helped

https://www.technologyreview.com/2021/07/30/1030329/machine-learning-ai-failed-covid-hospital-diagnosis-pandemic/

by gumby • 5 years ago

151 116 5 years ago

vitorsr• 5 years ago

“The models are so similar—they almost all use the same techniques with minor tweaks, the same inputs—and they all make the same mistakes,” says Wynants. “If all these people making new models instead tested models that were already available, maybe we’d have something that could really help in the clinic by now.”

Sounds about right.

briefcomment6• 5 years ago

I'm curious what people find is more likely given this evidence:

- 100s of models were done incorrectly, and none even remotely correctly

- there is no phenomenon to correctly model

Both seem extremely unlikely. Not sure what the alternatives are.

tablespoon4• 5 years ago

That's a false choice. There's also:

- Technology not suitable or adequate for this use case.

I mean, we've been to the "AI over-promises and under-delivers" rodeo before.

throwaway2048• 5 years ago

100s of models that all make the same wrong assumptions are going to be equally worthless.

You can make all the models about phlogiston you want, none of them are going to accurately explain fire.

new2994• 5 years ago

My guess would be that it's hard to get a clean well labeled dataset where case and control datasets are otherwise similar. Particularly, in a pandemic the people doing the data collection are otherwise occupied.

Another problem you have is when you do it right the results look worse than a biased dataset. So there's some incentive to continue working with a biased dataset.

webel0• 5 years ago

100s of DL groups with little previous experience in this area set out to train similar models at the outset of the pandemic. Many rushed out models. I’m not surprised that they ran into similar pitfalls.

roenxi• 5 years ago

there is no phenomenon to correctly model

COVID exists, it causes physical changes and affects the world outside the host. 100% it can be modelled somehow in theory.

That being said, there are harsh rules against collecting data (for example, it would be extremely illegal to create a sample set by taking a random group of people and infecting half of them with COVID on purpose). So I expect actually getting training data is a huge hill to climb. If most of the models used the same training data I assume there is a reason like that.

So in practice, I don't expect a model quickly.

jltsiren2• 5 years ago

As someone with a background in CS (but not AI/ML) and working in bioinformatics:

People with a background in methods like solving problems that are easy to solve with the methods they know. Those are rarely the problems people in biology/medicine want to solve. Some common pitfalls include:

* Asking the right questions.

* Framing the questions correctly.

* Finding computational problems that capture the essence of the questions and that can be solved efficiently.

* Interpreting the results.

* Avoiding systemic biases in rare but important edge cases.

Finding the right problems to solve takes time, and the first attempts will probably fail.

LeetHacks2• 5 years ago

Very early on in the pandemic one research group opensourced "COVID-Net", using the described flawed dataset.

Given the fact that at that time there was still quite a lot of "if we all help we can beat the pandemic" positiveness going around. An easily run-able piece of small code that promised "results" was available and the fact that a lot of organizations wanted positive PR, you get these insultingly low quality "research" practices.

I looked at a lot of the initial papers and models and most were forks of COVID-Net or used an already established model like ImageNet and retrained it with the bad dataset. Presto! Paper out PR happy. Very few questioned the dataset or the approach in general. But they loved bragging about it on LinkedIn.

toast0• 5 years ago

There's almost certainly something to model, you don't hear stories about hospitals filling up with patients and nobody knows if they're covid patients or how to deal with them; or at least, not anymore.

100s of wrong models doesn't sound extremely unlikely to me. It's really easy to train a model and test that it works on some dataset and then find it doesn't work on a different one. Setting up the inputs for training is difficult because of data privacy, there's intense pressure to publish, models have no expectation of explanability, lots of reasons for a wrong model that looks promising.

I've worked adjacent to people trying to solve problems with machine learning and it's rough going. When things work, ok nice, but when they don't, it's much harder to tweak than something based on heuristics (which, of course, don't always apply either).

beerandt4• 5 years ago

This is similar to people wanting to volunteer after disasters, but get mad when they're told they'd be more in the way than helpful.

It's selfishness and narcissism masquerading as philanthropy. Instead of doing an actual useful task, they want credit for the exciting/pioneering front line work.

It was the same thing with the ventilator shortage. How many people publicly declared they were "inventing" a new, easy to make ventilator? Design was never the problem, manufacturing was. But few were interested in manufacturing extant (and already FDA approved) designs.

afurculita• 5 years ago

Yep, that's because they didn't use a blockchain. That would have solved everything.

cjohnson318• 5 years ago

Not sure why this was downvoted.

mrlanderson• 5 years ago

bcuz a lot of hacker news readers have no sense of humor or sarcasm detection

tablespoon4• 5 years ago

Yep, that's because they didn't use a blockchain. That would have solved everything.

I just bought the COVID-19 NFT and burned it. Pandemic over.

new2994• 5 years ago

Many unwittingly used a data set that contained chest scans of children who did not have covid as their examples of what non-covid cases looked like. But as a result, the AIs learned to identify kids, not covid.
Driggs’s group trained its own model using a data set that contained a mix of scans taken when patients were lying down and standing up. Because patients scanned while lying down were more likely to be seriously ill, the AI learned wrongly to predict serious covid risk from a person’s position.
Errors like these seem obvious in hindsight. They can also be fixed by adjusting the models, if researchers are aware of them.

I wouldn't trust any method of fixing errors in data collection. You need to redo data collection and perform a proper case-control study. You need to select your cases and controls such that the demographics look similar.

In general, your goal is not to pick cases out against a background of the general population. It's to pick cases out from an at risk group.

And if there's any difference in procedure used for data collection between cases and controls, your data is compromised. Researchers often suggest correcting the dataset (cropping out artifacts etc.). In my opinion this doesn't really work... artifacts (for example in image data) can often cause subtile global differences in illumination which ML will pick up on but not be obvious to someone inspecting the images.

The only solution when you have such artifacts (and only one that would be acceptable to me if I was doing DD, or evaluating the research) is to redo data collection in the context of a more reasonable case-control study.

LeetHacks2• 5 years ago

If properly done, that approach would most likely generate a model that works really well and detects COVID for the right reasons. Put it out in the wild and it breaks down again because all radiologist would have to use the exact same method of scanning patients. Practice shows that scan parameters, position, annotations vastly differ across hospital, scanner and radiologist.

A client of mine wanted me to look into this dataset and help creating a model for detecting COVID. One of the doctors they where working with wanted to be able to submit pictures they took (from their point and shoot camera) of the CT scan. Good luck putting that variation in your dataset.

new2994• 5 years ago

It's a first step I guess. At least if you can get the classification to work with controlled data collection you have reasonable evidence that the approach can work.

If all other parameters area randomized between case and control, this is also fine. I'd guess you can also add illumination and other artifacts to the datasets to make the training robust to this.

But ultimately in the extreme case (like wanting to submit images from a point and shoot camera) I suspect you'll have a hard time building a system that's robust to that... or robust enough to be used in a diagnostic context...

I personally don't think I'd be comfortable working without a standard configuration used by the radiologists and a controlled protocol.

forcry5• 5 years ago

I wouldn't trust any method of fixing errors in data collection. You need to redo data collection and perform a proper case-control study.

Climate research would like to have a word with you.

new2994• 5 years ago

It seems like a very different scenario as we don't have control Earths that we do case-control studies on...

sg47• 5 years ago

AI tools cannot catch covid. Only humans can.

briefcomment6• 5 years ago

Don't forget bats.

js8• 5 years ago

Don't forget that nature bats last.

gunshai• 5 years ago

What is happening in this thread? "branch covidians"... What the hell is that?!

nradov7• 5 years ago

What ever happened to using AI to diagnose infections based on cough sounds? It seemed like a promising approach.

https://news.mit.edu/2020/covid-19-cough-cellphone-detection...

aaaxyz• 5 years ago

By the time you experience symptoms you've already been contagious for a few days. It's like trying to detect fires by looking for piles of ash.

nradov7• 5 years ago

Did you even read the article? It's about detecting cases without clinical symptoms yet.

bumby2• 5 years ago

I vaguely remember reading more about this elsewhere and I believe they only used COVID positive patients. It would be interesting if there are follow-up studies with mixed training data if true.

nradov7• 5 years ago

No the researchers tested using both positive and negative patients.

dontreact5• 5 years ago

The silly thing here is that it’s very easy to train models nowadays, so the medical AI space is very noisy.

With the right data and the right application, this technology can definitely be helpful.

However, it is very hard to sort out signal from noise since any reasonably competent undergrad can throw a convnet at some images and get an AUC.

Actually, I have even seen a high school science fair where they did a medical computer vision application.

I think this speaks more to the accessibility and ease of use of the technology that is out there than it does to the inherent limitations of the tools we have.

I am reasonably confident that there is a group out there that was more careful and has something that might have been useful, but it’s just impossible for clinicians to know who to trust.

I feel that both uncritical, un-nuanced hype (Radiologists will be replaced in 5 years!) and uncritical, un-nuanced closed mindedness (AI will never be clinically useful) is harmful for tackling the biggest problem the field faces right now: trust.

Who to trust. When to trust.

mattnewport• 5 years ago

I am reasonably confident that there is a group out there that was more careful and has something that might have been useful

What is that confidence based on? It seems like the researchers here made a reasonably thorough effort to find "something that might have been useful" and only turned up two that were even worth further investigation:

"She and her colleagues have looked at 232 algorithms for diagnosing patients or predicting how sick those with the disease might get. They found that none of them were fit for clinical use. Just two have been singled out as being promising enough for future testing."

So are you reasonably confident that these researchers just didn't look hard enough to find the good stuff?

Grimm1• 5 years ago

Why not? Plenty of researchers aren’t thorough. I’m not saying that’s the case here but it’s entirely plausible from what I know about academia and wouldn’t be a crazy claim to make.

dontreact5• 5 years ago

I think the issue is not with the training step but with the validation step. The only way to build enough trust is to carefully run a prospective clinical trial, and before that you probably need enough retrospective evidence to get some sort of FDA approval.

These processes take a lot of time and it’s not surprising that a year and a half after COVID-19 became widespread in the US, we still don’t have a fully validated model.

iamstupidsimple2• 5 years ago

I'm not sure if this has been attempted, so please do correct me, but I would very much like to see AI models designed to assist rather than give coarse classifications like "X disease detected". The best example I can think of would be creating bounding boxes around specific tells, or listing specific markers that a clinician would be looking for themselves, leaving the final decision up to them.

dontreact5• 5 years ago

Nearly all FDA approvals for medical imaging diagnostic software are for programs that assist. It’s been done for decades.

JohnJamesRambo• 5 years ago

Should have built AI tools to educate the ignorant populace and encourage them to get a vaccine. I know there have to be tons of adboys and girls out there whose only job is to persuade people to do something that could have knocked this out of the park. I see we are still at about 50% vaccination and Walmart is begging people to come get a free vaccine. I’m sorry my tone is negative but this is really getting me frustrated and making me lose hope in the future of humanity really. The stupid is winning. It’s the Asimov quote coming true. :( We’ve got to take real action against this.

"Anti-intellectualism has been a constant thread winding its way through our political and cultural life, nurtured by the false notion that democracy means that 'my ignorance is just as good as your knowledge."

jauer• 5 years ago

(also frustrated) re: adboys and girls, the problem (at least in the US) is that vaccines are political, so persuading people to get them is seen as dangerously close to subverting democratic society. Same thing as taking action to counter foreign influence campaigns.

see also: the leaks to project veritas about Facebook's counter vaccine hesitancy program.

kQq9oHeAz6wLLS• 5 years ago

Well, back when Harris and Biden both said they were hesitant to get a vaccine produced under Trump, they kinda set the stage for the dumpster fire we have today.

travisgriggs• 5 years ago

/c/s

So maybe the real mistake was making it free. IF a company could make a few bucks for every vaccine administered, we would have seen the ad industry being its full might to bear to harvest as many people getting it as possible.

grumblenum• 5 years ago

I don't think there is an absence of software with the purpose of convincing the public to consume the right pharmaceutical products. I would say every owner of major internet property has adopted some kind of feature or process to promote them. Indeed, it has become quite the fashion statement to signal one's zeal for pop-science and one's contempt for the "deniers." I'm sure there's a market for 'Wear. The. Mask.' bumper stickers.

I would agree that stupid is winning, but I probably agree more with Kant on what Enlightenment means.

forcry5• 5 years ago

The stupid is winning.

Calling someone stupid just because they do not share your blind trust in a bunch of people is kind of malicious.

kook_throwaway• 5 years ago

Blind trust in professional liars (politicians) and criminals (didn't Pfizer and J&J get the two biggest criminal fines ever assessed?).

Here's an interesting read from 2010 about how they were having former execs put in power in Canada. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2875889/

But I guess I'm the stupid one.

trhway4• 5 years ago

to educate the ignorant populace

ok, lets educate. And before educating others lets educate ourselves.

https://www.cnbc.com/2021/07/30/cdc-study-shows-74percent-of...

"About three-fourths of people infected in a Massachusetts Covid-19 outbreak were fully vaccinated, according to new data published Friday by the CDC.

The new data, published in the U.S. agency’s Morbidity and Mortality Weekly Report, also found that fully vaccinated people who get infected carry as much of the virus in their nose as unvaccinated people. "

And another point related to education. Couple days ago a White House official stated :"We're dealing with an evolving virus. There is no precedent, and there is no playbook". Note, it isn't the White House of the previous buffoon, it is supposedly educated current White House. No wonder that being led by the people like this the current pandemic response isn't that much better than the ones of the previous centuries.

doovd• 5 years ago

The vaccine doesn't stop you from getting covid. It reduces the effects so you're less likely to need hospitalization. So saying x% of people with covid are vaccinated doesn't mean much. There's some education for you.

Jweb_Guru2• 5 years ago

What's neglected in that article is that essentially the whole town is vaccinated:

https://www.masslive.com/capecod/2021/07/how-provincetown-ac...

Despite virtually everyone in the town being vaccinated, 25% of the cases in the outbreak were in the remaining handful of unvaccinated people! As more areas reach close to 100% vaccination, we will start to see weird quirks of statistics like this more often, but if your takeaway here is that it's really easy for vaccinated people to catch and spread COVID, you're very wrong. Breakthrough infections are prevented by vaccines like Pfizer with about 99% efficacy, and without a breakthrough infection there is no transmission. If pretty much everyone were vaccinated, the odds of transmission would go to zero very rapidly.

oezi4• 5 years ago

I guess you are trolling. And is really sad that such an one-sided argument as yours even exists.

First, you are missing that this is a Delta out-break. Delta has been found to be able to evade the vaccines more than other variants. This is something new and something we didn't know just 2-3 months ago.

Next, without stating how many percent of the population where vaccinated in MA what does three-fourths imply?

Governments world-wide are struggling with how to respond to Delta. Because it is more contagious and even for vaccinated people there is a risk of hospitalizations, there is a struggle what to do. Lift all restrictions (as England has done)? Stay rather strict (as in France/Germany where there are still public mask mandates)?

The reality is that there is no playbook.

parineum4• 5 years ago

I've been telling people that the way to get people to take the vaccine would be to give Trump credit for operation lightspeed (which I think he deserves but that's irrelevant) and start referring to the vaccine as Trump's Vaccine.

I'm okay with handing Trump a win if it can help save lives and end this pandemic.

azinman2• 5 years ago

Totally agree. I’ve been thinking the same, and it’s odd to me that he didn’t go that route to begin with. You’d think he’d be all over it.

snovv_crash2• 5 years ago

The vast majority of anti-vaxxers I know are more on the Green side. The Trump supporters got the vaccine as soon as they could.

TomSwirly2• 5 years ago

give Trump credit for operation lightspeed

Your idea wouldn't work. MAGAs hate liberals, and whatever liberals say appears to them like the voices of the grown-ups in the Peanuts cartoons. Maybe they pull out a few phrases to parody.

We've had over a year of near-continuous propaganda from the Republicans. Masks threaten your freedom. COVID is just the flu. No one dies from it, they put it down on every death certificate. Vaccinations kill you, not COVID. Vaccinations spread COVID, that's the new one.

Giving Trump credit for it would change nothing, even if the MAGAs heard it, and they wouldn't.

which I think he deserves

It astonishes me that after years of watching Trump lie and lie and lie and lie, rambling and incoherent, often contradicting himself from day to day, watched eighteen months of him lying about COVID including that first key year where he did everything he could as President to prevent an effective response to COVID, that people somehow believe that he's secretly a competent administrator who "gets things done".

What Trump "deserves" is to spend the rest of his life in jail. His deliberate mishandling of COVID resulted in hundreds of thousands of needless American deaths: https://www.thelancet.com/journals/lancet/article/PIIS0140-6...

criticaltinker• 5 years ago

I understand your frustration, but the condescending remarks are not conducive to wholesome scientific discussion.

Diversity is the way nature builds robust and balanced ecosystems - AI tools that reinforce ideological homogeneity could be catastrophic, for reasons I consider self-evident.

Returning to the topic at hand, and as a counterpoint to your claims, there are legitimate second and third order effects to consider regarding vaccines and the evolutionary dynamics of SARS-CoV-2. Don't take my word for it - here is peer-reviewed research from the brightest minds and top institutions in the world, raising serious concerns about approaching eradication of the virus purely through mRNA vaccines in their current form [1].

I cannot emphasize enough that the spirit of science is to question assumptions and rigorously prove hypothesis.

[1] Risk of rapid evolutionary escape from biomedical interventions targeting SARS-CoV-2 spike protein (April 2021) https://pubmed.ncbi.nlm.nih.gov/33909660/

blub• 5 years ago

Social diversity of opinion is also to be expected. No matter what the leadership decides, there will be people that criticize it and there will be people that are strongly against it.

The reasons almost don’t matter. If the leadership doesn’t have at least two plans to deal with this expected situation, then they have failed.

TomSwirly2• 5 years ago

I cannot emphasize enough that the spirit of science is to question assumptions and rigorously prove hypothesis.

But that's not what we're getting. Instead, we're getting a large number of people who haven't spent ten minutes bothering to learn anything about the science posting obvious bullshit.

How is anyone supposed to get actual science done if nearly all the "questioning" is repetitive worthless lies by people who have absolutely no interest in science at all and are simply screaming obvious lies at the top of their voices?

A question I ask a lot of the vaccine deniers on my Facebook page is, "Can you give a quick explanation in your own words why COVID-19 is so good at getting past the human immune system?"

No one has even tried to answer.

edmundsauto• 5 years ago

I’m not sure if adboys includes FAANGs, but mqny of those companies have spent billions of dollars to do just what you said.

There is an argument they weren’t as effective as they could have been. Is that what you mean?

wesleywt• 5 years ago

As a biologist who started working in ML, I see exactly the same issues in the field I am working in. I am spending most of my time fixing bad while others just use the public datasets as is. Maybe I am wasting my time because they are in the top journals and I am not.

cweill• 5 years ago

The hardest part about ML is understanding the domain. I agree with Jeremy Howard on this, that it's easier to teach a domain expert to do ML than to teach an ML expert the domain. I think you are on the right path.

8note• 5 years ago

Does a PCR test count as AI?

It has some S shaped function to trigger

yobbo• 5 years ago

The tools that are inferring the family tree of mutations (for example nextstrain.org) have been helpful. That's also a form of AI.

aaccount• 5 years ago

Most people working in the AI field are completely ignorant to the underlying mechanisms. They just hack together some code from github and feed some data in. Just because you get an output doesn't mean its even remotely accurate.

Anyone who does understand the underlying mechanisms are probably still trying to clean the data sets.

29athrowaway• 5 years ago

Superspreaders do not care about compliance or COVID prevention tools.

Superspreaders (antimaskers, antilockdown, antidistancing, antivaxxers) are the people keeping COVID alive.

Leparamour2• 5 years ago

Superspreaders (antimaskers, antilockdown, antidistancing, antivaxxers) are the people keeping COVID alive.

It's not as black and white as you try to paint it.

I'm not living in the US but I've encountered a sizable number of people who dropped all caution and safety measures as soon as they got vaccinated even though they still can catch the virus and spread the virus with the added bonus of favoring the formation of immune escaping mutations.

nradov7• 5 years ago

SARS-CoV-2 is now endemic in the worldwide human population, just like several other coronaviruses. It will never go away regardless of what people do. Fortunately the vaccines can limit the death toll.

disabled• 5 years ago

I guess the AI caught COVID-19.

ahD5zae7• 5 years ago

Yeah, I guess it would be cool to do an X-ray or CT scan of every patient, preferably in repeatable and controlled conditions similar to the AI training set. On the other hand there supposedly were very good results with training dogs to identify covid patients by smell alone. Which sounds faster, cheaper and probably even less error prone (judging by this article). I guess it doesn't scale well and there's not much money to be made, so there's that.

I think it may be the case with AI researchers where if the only tool you have is a hammer then every problem starts looking like a nail /jk

nradov7• 5 years ago

X-rays and CT scans involve some expense and radiation exposure. It makes no sense to subject every patient to such imaging studies. We have cheaper and safer tests to diagnose most cases of COVID-19.

sgent• 5 years ago

We have good tools for diagnosing COVID-19, and have had them from almost the start. If your looking for medical AI problems, start with the 12-lead EKG which fits on one piece of 8x10 piece of paper, is 2D, and current formulaic algorithms misread about 50% of the time when compared to a paramedic or nurse. Much less a physician, cardiologist, or electrophysiologist.

If this algorithm was better it would absolutely save lives and make everyday life easier for clinicians.

AI tools have been built to catch Covid, but none of them helped

Related Stories