Scientists skeptical about Theranos blood test

http://www.businessinsider.com/science-of-elizabeth-holmes-theranos-2015-4#ixzz3YO784nGj

by walterbell • 11 years ago

89 92 11 years ago

Hey, can we test your blood? We are making blood tests more accessible. No, we won't tell you what we are doing with your blood or how we are doing it or even if it works. What do you mean how does this make blood tests more accessible? Look, don't worry, just trust us, after all we have Henry Kissinger on the board, so we must be on the level.

rustyconover• 11 years ago

Let us not disregard the intelligence value the NSA or another other government entity gathers by doing a low resolution DNA sequencing all of the blood that goes through these machines for future reference. This covert use wouldn't be outside the bounds of proper skepticism with the involvement of ex-DoD members on the board.

Without proper auditing and disclosure of development processes and the new implementation of the diagnostic tests on chips (rather than disposable test media) it raises the question of how can anyone be sure exactly what and what not the machines are testing and recording? Slot machines in Las Vegas have their code more closely audited and monitored than this technology.

This situation just yells out for government regulation, peer review and reverse engineering.

jacquesm15• 11 years ago

Let me get this straight: because you're afraid that the NSA gets the hand on their data you want government regulation?

That's both the government.

When you're giving blood to some entity and you want to safeguard what happens to the result regulation won't get that for you, but independent oversight, sample anonimization at the source (insofar as this is possible with a blood sample, your DNA is your identity), decentralization of test locations (preferably as close to the source as possible), proven sample destruction and no keeping of data post sample destruction would be the way to go. Government oversight pretty much guarantees government access, the opposite of what you wish to achieve.

It will be very hard to create a service like this without a loophole that can be used to sneak such data out.

The best way in which this could be done is by having the tests made so foolproof that they can be done directly in the hospitals without centralized laboratories.

As for the board, you could say something similar about DropBox but I don't see them giving three letter agencies access to your account either without proper process and a court order.

Anything less could sink the company and given the amounts of money at stake here you can bet their competitors would love to do just that.

And who is to say that their competitors are not already acting in a similar manner anyway, they are after all processing about 100x more volume today.

_up• 11 years ago

Microsoft, Google, Apple are all NSA PRISM program participants! And nothing happened to them. Why should that be different with Dropbox? And isn't Dropbox using Amazon to store its data. It would probably be sufficient, if Amazon participates.

bhayden• 11 years ago

I don't see them giving three letter agencies access to your account either

And you wouldn't if they were served a NSL, which is very likely to have happened.

stefantalpalaru• 11 years ago

This company being a front for something defense-related makes more sense than naive investors throwing money at an idea with nothing to back it up for more than 10 years.

jacquesm15• 11 years ago

What stops the skeptics from doing their own tests by walking into a service center and having their blood analyzed for all tests possible and doing the same with their blood for conventional tests? Or an even large scale test?

If their concern is genuine (rather than a way to get a look under the hood of a competitor for the current sellers of such services) there are multiple ways in which they could get their verification without inside information on how Theranos does its thing.

darkmighty3• 11 years ago

I think the demand is the company itself should be doing those tests and publishing them. I can't say I see a reason they didn't.

I'm not sure how available the tests are yet, but if it's available really anyone could test it. However, it requires a sample of a large diverse population to get statistical significance and that isn't cheap, which is why most academics probably aren't interested in doing it.

XorNot5• 11 years ago

No the problem is no one can get their hands on a sufficient quantity of the tests in order to run such an analysis. This is not a cheap or easy exercise to do rigorously, and would require a fair amount of staffing to carry out properly.

The company being non-cooperative is it's right - it's just not good science.

jacquesm15• 11 years ago

They don't need to prove it works, they only need to prove that it doesn't. That's related to how defenders need to close all the holes but hackers only need to find one to get in, proving that it doesn't work for a subset of the tests is enough to put a substantial hole in the boat.

Proving that it works is a much harder problem that probably only Theranos is willing to finance.

sowhatquestion• 11 years ago

Exactly. This is what I was hoping for from an investigative piece on Theranos. Even an n=1 sample where the author had his or her blood tests done with Theranos and a competitor would be worth more than all the speculation in this article.

jacquesm15• 11 years ago

Especially if such a test gave different results than the accepted competitors test.

streptomycin3• 11 years ago

I don't think an n=1 test would be useful. Depending on people's preconceived biases, they would just say "see this confirms what I thought!" or "lol n=1 is meaningless". To do something conclusive, you'd need to do a real trial with n>>1. That costs way more money than Business Insider (or most other venues) is going to spend on an article.

cloudwalking• 11 years ago

You can't actually walk into a service center and order tests. You need a doctor to order them. This is one thing Theranos is trying to change: enable and simplify access to your own medical data.

mhb• 11 years ago

Are you referring to blood tests, in general, or Theranos tests? I don't know about Theranos, but, in most places (in the US), you can order blood tests without a doctor's orders. See: https://www.directlabs.com/

mbreese• 11 years ago

Any such comparison wouldn't work for just an N=1. You'd need many more people before you could start to draw a conclusion about the Theranos test versus a conventional test. Just an individual doing this wouldn't be anything more than anecdotal at best.

I'm assuming they've done this study, but have refused to publish it for some reason. But this is the real question from the article - where is this study? You don't have to disclose the full technique for the test in order to publish the comparison.

jjoonathan• 11 years ago

Agreed, a n=1 test wouldn't reveal the most insidious kind of failure: guessing to cover up crummy data (err, I mean "leveraging machine learning techniques to maximize accuracy"). I doubt they're that evil, but it would be nice to know for sure.

I'm assuming they've done this study, but have refused to publish it for some reason.

Because they're a bog-standard vertical integration play and if that fact comes to light before they are done using the hype train to raise money and grab market share then their competitive advantage will evaporate overnight. If they only release test results to interested parties under NDA it prevents public sector science from rebutting their claims to disruptive rather than incremental innovation, allowing the hype train to steam ahead unimpeded.

DannoHung• 11 years ago

This title seems kinda weird. Can a mod make it more about the fact that the test method is in question rather than that someone got rich because they invented it?

dang2• 11 years ago

Now that you point it out, I agree. We changed the title to make it more neutral.

jacquesm15• 11 years ago

I suspect the 'billionaire' was thrown in there as clickbait regardless, for some reason large numbers attract clicks.

XorNot5• 11 years ago

The composition of the board has always bothered me. It's not at all clear what business value a lot of those people bring. Maybe opening certain doors, but I would've thought in such an area, as the article suggests, you would want more medical expertise.

That should be a little bit of a red flag to anyone - since the history of scientific fraud is full of lone specialists who couldn't brook explaining the technical details to anyone else qualified to understand them.

Rapzid• 11 years ago

It seems pretty clear to me and with the lack of peer-review and technical details paints a pretty disturbing picture. When I saw those names an image formed of a company gearing up to bend the government to its business objectives through its board's political ties.

Now, I'm not sure if that's because:

A.) They don't have a game changer or it's going to come under fierce scrutiny by the larger scientific community so they need to try to mandate its use through legal means and using the military as a gateway and OUR military budget as an early source of funds.

B.) Even though they have the real deal, the nature of the political landscape in which medicine operates necessitates the back-room dealing that this board can deliver on.

cdcarter• 11 years ago

Your staff should be experts, your board should be ambassadors.

fsloth• 11 years ago

Medical industry as I understand it employs all possible methods to curb competition. These include heavy political lobbying. The Washington heavy weights make sure their competitors cannot find enough political ammunition to curb them unless there is strong fact based evidence against their method.

hackuser• 11 years ago

It's also interesting there are so many foreign policy heavyweights, and many in a certain range on the political spectrum. I'm not suggesting some conspiracy, I just wonder why.

minthd• 11 years ago

One big problem in the healthcare in africa, is the cost and complexity of blood tests. And we know lab-on-chip technology can fix that, but nobody is working on that, because of "business model" problems.

But that's one thing a successful theranos could do, and foreign policy is important at that.

pcrh3• 11 years ago

The VA is a big consumer of medical services.

learc838• 11 years ago

So the CEO dropped out of Standford at 19, started this company, and now she's a billionaire? I can see dropping out to found Facebook, but a biotech company?

What skills did she have that allowed her to do this, and where did she get the money to hire all the needed PhDs and get through FDA testing? How would she have even gotten access to the equipment and test subjects to prove that her idea was worth investing in outside of an academic setting? The whole thing sounds crazy.

Edit:

I read Elizabeth Holmes Wikipedia page and it says that when she was a sophomore at Stanford, her Chemical Engineering professor encouraged her to found the company. But I can't find anything about about any breakthroughs she made that warrant the millions in early investments. No one seems to want to talk about what exactly it is they're doing, and they're board of directors has a former Secretary of State, a former Secretary of Defense, and a former Senator. The fact that she's also been able to maintain 51% of this company is also astounding.

skrebbel• 11 years ago

Seriously? Just because you can't imagine it, it must be a scam?

learc838• 11 years ago

Yes, clearly that is exactly what I'm saying. Intelligent CEO == scam.

It takes more than intelligence to start a Biotech company. You need money, equipment, skill, and usually experience.

Medical breakthroughs usually require years of research and skills that are acquired working on a PhD. I'm not saying it's not possible for a 19 year old to make a world changing discovery, but when no one will say what exactly that world changing discovery is, I get a bit skeptical.

burritofanatic• 11 years ago

The New Yorker had a good article that covers her background. http://www.newyorker.com/magazine/2014/12/15/blood-simpler

vinhboy• 11 years ago

Although she can quote Jane Austen by heart, she no longer devotes time to novels or friends, doesn’t date, doesn’t own a television, and hasn’t taken a vacation in ten years. Her refrigerator is all but empty, as she eats most of her meals at the office. She is a vegan, and several times a day she drinks a pulverized concoction of cucumber, parsley, kale, spinach, romaine lettuce, and celery.

Every time I read about these ultra successful people, I just think they are freaking machines... and I don't mean that in a derogatory or mean way.

I am genuinely amazed people can operate like that. I know I can't.

hayksaakian• 11 years ago

they probably excluded all the boring and reasonable parts of the story

When she was 9, her family moved to Houston, where her father had taken up a job with Tenneco (Oil Company). Intrigued by their father's work in China, Elizabeth and her brother learned Mandarin Chinese at a young age.[5] She spent her teenage years in China, and while still in school, started a business selling C++ compilers to Chinese universities.[8][5]

http://en.wikipedia.org/wiki/Elizabeth_Holmes

jacquesm15• 11 years ago

Her story is quite comparable to Bill Gates. She's levelled up compared to her parents but they were by no means poor.

gadders• 11 years ago

I hope I'm proved wrong, but it does sound too good to be true.

josephpmay• 11 years ago

I'm kind of perplexed why Apple needs FDA approval for their watch to measure blood oxygen, yet Theranos can develop a completely novel blood testing technique whose results could impact major medical decisions without having to conduct a single study.

(note that I'm not saying that Theranos hasn't conducted studies, rather the FDA doesn't require any)

toomuchtodo• 11 years ago

FDA testing regulations loophole.

jacquesm15• 11 years ago

'device' versus 'service'

ealexhudson• 11 years ago

Which implies Theranos only uses standard devices for their service, then?

marze• 11 years ago

It is because the FDA's prime mission is to protect the drug manufacturers from any non-drug competition.

netrus• 11 years ago

A little off-topic:

The famous Forbes-List estimates her wealth to be 4.5b, because she owns half of a company that has raised 200m. If she is somehow reasonable with the finances of her company, it is likely that her actual bank-account (or real-estate etc) is well below 10m - which is a hugh difference to all the old-money people far below her in the list.

webreac• 11 years ago

many scientific revolutions have occurred after improvement of measure instruments. If we start to have a continuous measure of our blood, there may be revolutionary discoveries about health.

lotsofmangos6• 11 years ago

ADR scanning could probably do this non invasively, it can even pick up prions in blood - http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2008164/

learc838• 11 years ago

After a bit of digging I found another article that discusses the skepticism a bit more. The comments are worth reading as well.

http://pathologyblawg.com/pathology-news/new-yorker-theranos...

Gatsky2• 11 years ago

It is interesting to look at their test menu at: https://www.theranos.com/test-menu?ref=our_solution

You can see here that they have replicated most of the routine blood tests out there. Interestingly, they have also replicated some of the archaic and pretty useless blood tests like Rheumatoid Factor. They also offer Erythrocyte Sedimentation Rate (ESR). This is pertinent because ESR is actually a physical property of blood, it is performed by measuring the rate at which red blood cells fall to the bottom of a tube. Perhaps they have developed a microfluidic method to measure this, but it wouldn't be a 'chemical process'.

But at the end of the day, they are offering the same tests that are currently available. If someone said to me we can now do the same old blood tests faster and cheaper, I would say that this is going to have very little impact on people's health. None of these tests are truly useful in prevention or screening unselected patients.

It does really seem that Theranos is a case of packaging the existing blood diagnostics and trying to capture a large part of the market. Exciting for investors I guess, but for patients or healthcare workers probably not so much. I wish they were doing something like cheap, easy and reliable comprehensive circulating DNA analysis, or proteomics, or something with immune cell function profiling...

I hope with all this cash they have a program for giving this technology to places that can't afford the usual lab infrastructure, that would be a very good thing.

tel4• 11 years ago

If someone said to me we can now do the same old blood tests faster and cheaper, I would say that this is going to have very little impact on people's health. None of these tests are truly useful in prevention or screening unselected patients.

Are you kidding? I'm not completely familiar here, but I'd love to see a comparison of the current SOC for time between measurements of each of these endpoints and then a clinical judgement that genuinely nothing interesting happens in these levels at timeframes shorter than SOC.

A person I'm familiar with has a severe endocrine disorder which has eliminated nearly all of his natural hormone production. Instead, he must self-administer all hormones in proper dosages. In order to survive this way he has become vastly more familiar with his own blood panel than most human beings ever would dare. On insurance dime he takes far more regular blood tests than you usually can get access to and understands, even very roughly, how events in his life, diet, exercise, stress, weather, etc begin to affect his blood chemistry and, subsequently, his life. In his, of course anecdotal, experience these are things that he cannot converse with doctors about because they're not able to speak confidently so far outside of SOC. It's not necessarily surprising stuff, but there's comparatively little real-world evidence to give it a bite.

Theranos could make similar analyses achievable for far more people. It opens an entirely new branch of clinical research---blood chemistry fluctuation at the level of perhaps up to multiple times per day over months at a reasonable cost in both testing and human time (given adequate distribution---Walgreens or, even, eventually, a home edition). You can't pretend that Phizer and GSK, known partners, aren't all about that.

Exciting for investors I guess, but for patients or healthcare workers probably not so much.

Only if you assume SOC doesn't change under the influence of massively more available quantitative data.

streptomycin3• 11 years ago

A person I'm familiar with has a severe endocrine disorder

He said "unselected patients". The point is, some random guy with no preexisting condition might not actually benefit from being able to measure shit in his blood more easily. It could mostly lead to false positives, stress, and unnecessary costs. The only way to know for sure would be to do a study on it... but Theranos doesn't seem to like doing studies.

Making blood tests faster and cheaper would definitely be a good thing, but it's possibly not a completely transformative thing.

Gatsky2• 11 years ago

I am assuming that SOC won't change under the influence of massively more available quantitative data, or at least any gains would be both slight and only applicable to a very small number of people. I think it is probably a misconception that humans are simple enough that measuring something (on Theranos' current testing panel) which is produced in ten different ways in five different organs from millions of cells and then diluted in 7 litres of blood is going to tell you anything revolutionary.

Certainly happy to be proven wrong, but there is a risk of harming people (financially, psychologically and perhaps medically) if you convince them that relentless tracking of their serum sodium tells them something useful.

pcrh3• 11 years ago

I'm skeptical that there is anything massively different between the technology Theranos uses and that used by existing clinical diagnostic labs.

Their greatest claim, as least in the popular press, appears that they can do these tests on "a few drops of blood" (it appears to be about 0.5 ml from the image) as opposed to 10 ml or so. This claim is nothing special, as in fact almost all tests are eventually done on an aliquot no larger than a few microlitres that is withdrawn from the larger sample. As to speed of results, this would likely be linked to organizational efficiency rather than a technical innovation.

Further, there is no way all the tests shown on their website can be done with one 0.5 ml sample, as samples need to be collected into different anti-coagulants for different tests as some anticoagulants interfere with some tests but not other (this is why the doctor usually takes two or three samples into different tubes -- not because s/he needs 30 ml of blood).

They may however have improved automation, or a better run or cheaper service, in which case, good for them.

lotsofmangos6• 11 years ago

This gives a bit of perspective on their current system, it is far off something like a hand held scanner. It is from a blog article by a bio-chem due diligence company.

Whether it was due to the quantity of tests (7 tests) or the particular tests I ordered, I was quickly informed that I would be receiving a standard blood draw for my first visit. To my surprise, only 75-80% of patients receive the finger stick method, while the remaining 20-25% require a standard blood draw. The company’s goal to expand nationally relies upon the ability to integrate within the infrastructure of existing Walgreens’ drugstores, a feat made difficult by the need for a trained phlebotomist to serve a specific group of test orders. While the company currently offers more than 200 blood diagnostic tests, the internal assay development team is working to expand this menu to include over 1,000 of the most commonly ordered tests at their CLIA laboratory located in Palo Alto, CA.

In addition to my own blood draw, I stayed to observe the highly publicized finger stick, a staple of the start-up’s competitive appeal over established companies like LabCorp or Quest. It was here that the charm of Theranos became more obvious. Rather than a phlebotomist, a Walgreens pharmacy technician is easily able to perform the “nanotainer” blood collection steps, a testament to the simple and integrated workflow system developed by Theranos. Blood collection kits are provided to the technician, which include alcohol swabs, a warm compress, a finger stick, and a “nanotainer” (kept separately in a fridge). After scanning and labeling the sample tubes, the pharmacy tech begins the simple process as follows: apply warm compress to finger, clean the finger with alcohol swabs, massage blood towards the tip of the finger, stick the finger using one of 3 prick sizes (purple = small, pink = mid-size, blue = large), collect blood. The “nanotainer” which collects only a few drops of blood via capillary action, has continuously been an attractive marketing tool for Theranos, inflating the nano-sized tube with a larger than life persona. These samples are then stored in the fridge until a courier system can transport them to the CLIA certified lab in Palo Alto, where proprietary platforms allow the company to run traditional tests on 1/100th to 1/1000th of the ordinary volume. My results were emailed to my physician the following day.

http://decibio.com/blog/theranos-small-sample-big-opportunit...

Eridrus• 11 years ago

If someone said to me we can now do the same old blood tests faster and cheaper, I would say that this is going to have very little impact on people's health.

I think you vastly under-estimate the impact of cheaper prices. We're not all sitting around with computers in our pockets because they're drastically different, we're sitting around with them because they're cheap.

Obviously there is skepticism on the applications, this is why she is giving all these talks about what this would enable.

I don't have any particular insight about what this kind of price drop will produce (though cheaper STD testing can't be a bad thing), but I think massive reductions in price do fundamentally change how things get done.

al2o3cr• 11 years ago

The article seems very confused - if the process is "patented", then surely the details aren't a secret? That's kinda the exact POINT of the patent system...

streptomycin3• 11 years ago

I don't understand. The article talks about how there is nearly no publicaly available proof that these tests actually work, yet they're selling them in drug stores and hospitals all over the country? And all we know is that supposedly "Theranos' labs have been evaluated and deemed acceptable by drug companies, hospitals that work with them, Walgreens, and others"? Why is this legal?

Scientists skeptical about Theranos blood test

Related Stories