Theranos Ran Tests Despite Quality Problems

http://www.wsj.com/articles/theranos-ran-tests-despite-quality-problems-1457399479

by Asparagirl • 10 years ago

145 52 10 years ago

Animats• 10 years ago

At what point here do people go to jail? The FDA can and has done that for faking lab tests.[1][2]

[1] http://www.nytimes.com/1983/10/22/us/3-ex-officials-of-major... [2] http://pubs.acs.org/subscribe/archive/tcaw/10/i11/html/11reg...

ageofwant5• 10 years ago

When they have been found guilty in a court of law ?

function_seven2• 10 years ago

Ha. Your answer was technically correct but completely useless. Implicit in Animats' question was when that whole process happens.

HarryHirsch3• 10 years ago

But who has standing to sue? Whoever suffered a stroke because their warfarin dose was too high or too low is dead, they cannot sue any longer. Whose dose was accidentally correct or whoever was lucky enough not to suffer a stroke did not suffer harm, so they can't bring suit either.

And Holmes and the board believed in good faith that their prothrombin test was accurate. No mens rea, no culpability. No prison for anyone.

seehafer2• 10 years ago

At what point here do people go to jail?

Generally the FDA will give companies several outs to fix problems, they only bring out the handcuffs after several ratcheting steps (one of which includes bringing in a US Marshal and a team from the FDA, at your expense, to be at your business every day and tell you how to run your company).

madaxe_again• 10 years ago

At no point. They go into politics, charges if any are ever bought will just sort of fade away.

These are rich powerful people, they don't go to jail.

Asparagirl• 10 years ago

Key takeaways from this new article, for people blocked by the WSJ paywall:

"That letter cited “deficient practices” in five categories. In one of them, hematology, the problems found by inspectors posed “immediate jeopardy to patient health and safety,” regulators said.

The test that inspectors found Theranos kept doing despite the erratic quality-control results was a hematology test that measures how long it takes blood to clot, the people familiar with the matter said.

The test is an important part of the treatment regimen for patients who are at risk of strokes or have blood disorders...

People who have seen the Theranos inspection report said it seems to indicate the company ignored quality-control results for the blood-clotting test that fell short of its own criteria.

The results produced in the quality-control checks repeatedly deviated from the lab’s typical result by more than two standard deviations, those people said...

At Theranos, there were seven quality-control failures in a single day, said a person familiar with the report...

Dr. Hamill said any doctors who received prothrombin time test results from the Theranos lab during the six-month period shouldn’t rely on them and should have their patients retested as soon as possible. “Those results are not worth anything,” he said...

The federal inspectors also concluded that the Theranos lab in California used expired reagents, or substances added to blood samples to elicit chemical reactions, for some prothrombin time tests, according to the people familiar with the inspection report.

Those people said the report also found that the lab’s director didn’t have responsibility for the lab’s quality-control program and the lab’s quality-control manager was unqualified..."

EDITED TO ADD: Actually, the really key sentence is this one: "Theranos spokeswoman Brooke Buchanan said: “The PT/INR issues identified by CMS related to tests run on conventional equipment using venipuncture samples.” A CMS spokeswoman declined to comment." Because that means that these quality control problems were not due to Theranos' "revolutionary" finger-prick tests, they were from regular old venipuncture tests running on a standard machine that were giving wildly incorrect results, expired reagents, no QC in the lab, etc. Bad data is worse than no data. If they couldn't get old-fashioned labs right, no wonder they were having trouble with the finger-prick stuff.

cordite• 10 years ago

That is rather scary. Quality control is a serious requirement to any competent lab. With physicians making decisions by results labs transmit, there is an inherent need of verifiable trustworthiness to the results.

A patient may be subjected to life altering medication or treatments, which is not a trivial responsibility that can be shrugged off with erratic processes or lack thereof.

A lab director's responsibility includes ownership of quality processes, reviewing abnormal results after the fact to see if proper processes were followed (such as communicating with the respective provider), reviewing financial success, and much more. To shrug off quality control and operate as if reputable is not just unethical, but incompetent. Such operations should cease immediately.

Disclaimer: I have experience in the medical lab industry.

TazeTSchnitzel• 10 years ago

It makes me wonder if Theranos's inaccurate results have killed anyone. It's not unlikely.

daveguy2• 10 years ago

Does anyone know if Theranos conducting tests outside of California? I thought I heard something about Nevada and maybe one other state?

seehafer2• 10 years ago

Arizona.

jonwachob91• 10 years ago

stevebmark2• 10 years ago

I've never seen so much scrutiny over a company before. Everyone is screaming "death to Theranos" at every opportunity. How much of that is purely balking over the high valuation? I have a feeling if they didn't raise so much money then journalists wouldn't be swarming this at all.

All of the critiques I see don't seem even close to show stoppers. No public peer reviewed studies? Some lab updates to match FDA regulations? Using some "unauthorized" third party equipment? This is a new company, why should these be the doomsday callings?

I have no idea if Theranos will succeed, but I can't shake the feeling that this media response is inflated due to the high valuation, and journalists are falling victim to the hype as much as the public, whereas this is a fairly normal health startup otherwise. It probably was a mistake to accept such a high valuation, so I don't think Theranos is totally innocent here.

It's also strange that a paywall article made it to the front page? I wouldn't guess there's a big overlap of WSJ subscribers and HN readers (lol conspiracy).

bronson• 10 years ago

And how much of that is due to Theranos being a medical company? You need to play a clean game when lives are at stake.

saalweachter• 10 years ago

Derailing, that's my biggest ":-/" with the Silicon Valley start-up "move fast and break things" culture.

If we as software engineers want to work on problems that matter, and not just dating apps and ad networks, we can't have a work ethic that encourages us to ignore troublesome laws and regulations and launch half-baked MVPs with a plan to maybe fix problems as they arise in future iterations. Sure, work smart, automate all the boring shit, but your automation has to be correct. Sure, focus on a MVP to get off the ground -- but make sure your product is viable, and correct across the minimum set of features.

Even when we are working quickly, sparsely, we should be laying good foundations for the future. We shouldn't be lazy conmen selling poisonous snake-oil.

Bud• 10 years ago

Completely wrong. How far down do you have to read in this article to find Theranos' valuation?

All the way. It's not there. At all.

Nobody really gives a fuck about the valuation except valuation wonks. But a lot of people care a lot about this woman being a complete charlatan who is endangering public health.

BinaryIdiot• 10 years ago

How much of that is purely balking over the high valuation?[...]this is a fairly normal health startup otherwise

Absolutely not. Start-ups are supposed to fail early and fail often but when you're in the medical space you simply can't act entirely like a technology start-up. People's lives depend on many of these tests; you can't just go "well let's ignore the FDA and we'll take the slap on the wrist" like in other industries. Here they might deliver inaccurate test results that are then used to treat someone for a medical issue.

That can be deadly. Yeah many people are going crazy from the valuation and secrecy but there is plenty of legitimate scrutiny here especially if you read this article (I'm assuming you didn't because of your very generic Theranos comment versus addressing the issues in the article itself which sound grossly negligent, IMHO).

sbose78• 10 years ago

True, if a startup in the medical didn't have the resources for quality control they should not have started up in the first place.

msabalau• 10 years ago

Let's see: Due to quality control “Those results are not worth anything,” Yet doctors were relying on them to adjust warfarin levels, where "Too much warfarin can cause fatal bleeding, while too little can leave patients vulnerable to clots and strokes, according to medical experts."

But the attention is because people can't stand high valuations?

detaro• 10 years ago

A company offering medical services that pose "immediate jeopardy to patient health and safety" deserves scrutiny. If they knew that their results weren't good and didn't act on that information, why should they ever be trusted in the future?

stevebmark2• 10 years ago

This illustrates my point. How does holding a drop of someone's blood in a lab pose "immediate" jeopardy to patients? They're in the trial phase, right? These tests are not yet being used to determine actual medical outcomes. What's wrong with trial and error in the trial phase? It's like PCI compliance. Who cares if your company doesn't pass PCI certification the first time (no one? You're given a long amount of time to fix PCI compliance issues).

manigandham• 10 years ago

As the other comments have stated - this isn't about valuation. This is a medical company that can, and probably has already, endangered patients' lives through its failings.

There is a reason why certain industries are scrutinized so closely because of the effect they can have on people. Being in those industries, even as a startup, means you must follow the rules.

Or are you fine with your doctor using some "unauthorized" random equipment the next time you need critical tests or surgery?

daveguy2• 10 years ago

Or are you fine with your doctor using some "unauthorized" random equipment the next time you need critical tests or surgery?

Hm. In that case I guess we would have to ask the question -- how legitimate does the eBay site look?

kelseyfrancis• 10 years ago

They've literally recklessly endangered human lives. This isn't a game.

alistairSH2• 10 years ago

This is a new company, why should these be the doomsday callings?

Theranos is NOT a new company. They've been around for more than a decade.

This isn't the latest and greatest dating app. This is medical tests that have a very real impact on the treatment provided to patients.

ghshephard4• 10 years ago

The key takeway I have from all this is if you are relying on Blood Tests for data that is critical to your health (as opposed to informative, educational, or personal interests) - it's probably a good idea, if you can afford it, to get results from three completely separate diagnostic systems - kind of like time with ntpd.

nradov3• 10 years ago

Not practical for most patients in the real world. Insurers won't pay for redundant lab tests. Getting blood drawn multiple times is painful and logistically difficult. For patients with clotting problems just getting blood drawn presents some risk. And there often won't be 3 independent labs in the patient's area.

ghshephard4• 10 years ago

You could reduce your accuracy by doing three tests of the same blood sample (of course, exposing yourself to the risk that the blood was taken incorrectly, was stored in an incorrect vessel, was transported incorrectly, etc...).

In terms of practicality/Insurance - this would mostly be a function of how absolutely certain we are that the tests are accurate, and how damaging incorrect results would be. If we are 99.99% certain the values are correct, and incorrectly results wouldn't result in irreparable harm - then yes, not worth the investment in redundant lab tests.

But, if there is, say, a 5% chance I might die on an incorrect test result that has a 1% chance of happening - and that test is performed 12 times/year - then you can pretty quickly calculate the net-present-value of my life versus cost of test. (I.E. If I think my Life is worth $10mm, and the test normally costs $500 each time, or $6000/year, and I need to have this test done for the next 10 years, (assuming the value of my life doesn't drop over time, to make this simpler) - then each time I get the test there is a 1% chance of failure * 5% chance of death = 0.05% chance of dying as a result of each test.

The odds of me dying then, in the next 120 test = 100%-(chance-of-not-dying-120-times-in-a-row) = 1-.9995 ^ 120) = 5.825% chance of dying in this scenario using tests that cost $60,000. If I could reduce that to 1% (4.825% reduction) chance by spending an extra $120,000 (perform the test in two extra labs) - then my my investment of $120k would return 4.825%*10mm = $482,500.

You can work the numbers, assumptions, etc.. however appropriate - but for some combination of test costs, frequency, potential harm, and value assigned to life - there is quantifiable value that justifies performing the test multiple times (and, likewise, clearly circumstances in which, even if the test results are more likely to be inaccurate, not worth performing thee test).

Not having access to multiple labs is a real problem though.

Cass• 10 years ago

If you're taking Warfarin, you have to get your INR checked every two to three weeks. It's really not practical to do that at three different labs every time. Also, this isn't some exotic edge case that only applies to a few people, but an extremely common medication. We're talking about huge numbers of patients here. If all their tests had to be triple-checked, the costs would be enormous.

ghshephard4• 10 years ago

See https://news.ycombinator.com/item?id=11243904 above as to why the cost issue is a risk assessment / return analysis. In certain scenarios in which potential harm is high from an incorrect diagnosis, it's worth the cost to perform the test completely independently.

Gatsky• 10 years ago

Not really. Normal labs that aren't aiming to be revolutionary ultra-profitable mega corporations will be fine.

Having a slightly more accurate result is useless, because there is far more biological variation in a person (from disease, the vagaries of physiology, compliance, pharmacokinetics, diet, the time of day, mental state, the season etc etc) than technical variation in a lab test. You would be optimising the wrong thing.

The only example I can think of where someone would probably be harmed by this is if they were undergoing some kind of a procedure with a bleeding risk, and the Theranos test said their clotting was fine when it wasn't, and they had a significant haemorrhage during the procedure. Someone will probably launch a class action lawsuit anyway.

I think that the Theranos board will realise that they need to get rid of Holmes in the next 6 - 12 months. Even if the tech actually works, having her there has become a liability.

HarryHirsch3• 10 years ago

If none of the three tests is calibrated and traceable to a standard you are just as well off throwing darts at a board and getting your numbers out of that. Take personal responsibility for your health and your bank account by throwing dice!

It's really funny how people have stopped hating on the FDA when Theranos is mentioned. In any other context the FDA is still worse than Satan and the pox.

HeyLaughingBoy• 10 years ago

Not necessary. I worked in the medical device field for many years, writing software to control instruments that tested blood (serum).

The instruments can be trusted.

If a test is borderline, either the lab tech will manually retest it, or some instruments can be given business rules like "if result is on the threshold of good/bad, automatically rerun the test and provide both results to the physician."

We were well aware that the software we wrote affected real people's lives and acted accordingly. These idiots are only giving the rest of us a bad name.

yitchelle• 10 years ago

Just starting to read up on this. Does anyone know if any of Theranos's customers used the lab results for ongoing medical treatment and suffered as a result of it?

jorgecurio• 10 years ago

This is speculation but...the company was founded in 2003....13 years ago and no product out yet...I have an awful feeling Holmes felt desperate, took on more than she could chew in order to compensate for the decade she lost, and now all the shortcut taking is catching up to her....

It's starting to look very ugly for what started out as something overreaching, and I think she will fight even harder to keep the house of cards from falling but pretty much this unicorn will be the first to go in 2016.

return0• 10 years ago

This all sounds spectacular i just wonder why it's only WSJ that is doing all the reporting for them.

nkrisc2• 10 years ago

Well they certainly managed to "disrupt" something, like trustworthy lab results!

thrillgore• 10 years ago

I just want to know when the tell-all book will be released.

Theranos Ran Tests Despite Quality Problems

Related Stories