A Letter I Will Probably Send To The FDA on 23andMe

http://slatestarcodex.com/2013/11/26/a-letter-i-will-probably-send-to-the-fda/

by AndrewDucker • 13 years ago

49 38 13 years ago

slatestarcodex.com

iand• 13 years ago

Probably premature since it appears that 23andme ignored the FDA for six months.

waxy• 13 years ago

Is it just me who thinks this sounds more like an ad for 23andme?

AsymetricCom• 13 years ago

What kind of doctor would write such an irresponsible letter? Oh...

But the alternative is banning all information that anyone could possibly use to evaluate their own health.

stopped reading there.

timr3• 13 years ago

"There are several thousand papers about genetics, linking specific genes to everything from eye color to intelligence to chance of developing colon cancer. Some of these papers are no doubt correct, others will no doubt in time be found incorrect. However, I worry you want 23andMe to independently validate each of these results before informing customers about them."

As they should. If they can't show that the associations they're reporting are valid, what purpose does it serve to report them? That's the whole point. It's not like drug regulation at all, because most of this stuff isn't settled science (and hey, not for nothing...we do test the effectiveness of the antibiotics that get sold at your local pharmacy. A doctor should know that.)

Tell you what: send me your saliva and $80, and I'll write you up a really nice genetic report. It'll tell you all sorts of great stuff, and it'll be $20 cheaper than 23andMe. But don't try to force me to certify that I'm not making everything up -- that would be stifling my innovation! After all, I'm providing a cheap service. Disruption!

For a community that goes absolutely crazy with skepticism about every scientific study (i.e. "correlation! not cause!"), there sure seems to be a lot of credulity to go around on this issue.

lucraft• 13 years ago

23andMe can't possibly run independent replication studies of every genetic link they want to report on. Nor should they have to, any more than your doctor should have to personally run a study replication in order to inform you about the results of your blood test.

timr3• 13 years ago

You're just repeating the original argument, and it wasn't a good argument the first time.

First, 23andMe isn't working in the realm of established science -- most of this stuff is highly speculative, at best. Blood tests are not speculative precisely because they're regulated by the process that you're arguing 23andMe should be able to ignore.

Second, the "your doctor shouldn't have to run the study" argument is the reddest of herrings. Your doctor doesn't have to do that (anymore...) because the regulatory system has provided a network of providers who do it more reliably.

Third, your doctor really does have to know how to interpret the results of your blood test. She can do that because the science is well-established, and the results are directly correlated to certain clinical outcomes. For most of this genetic testing, there's no equivalent interpretation. That's why you need genetic counselors, who know enough to know when an association study is total garbage. Or if a "50% change in risk" is meaningless in a larger context.

In short, it isn't especially onerous to want 23andMe to report only those genetic variations that are based on clinically relevant data, for which their tests are accurate and precise. That's just the basic standard for medical testing.

Symmetry• 13 years ago

Is your genetic lab certified by the CLIA[1]? 23andme's is and I don't think anybody is arguing that they shouldn't be regulated as a genetics laboratory. It's the FDA's decision to regulate them as a medical device that people think is inappropriate.

[1]http://www.cms.gov/Regulations-and-Guidance/Legislation/CLIA...

timr3• 13 years ago

Read the letter from the FDA. That's not what they're doing at all. They're requiring a company that sells a direct-to-consumer medical test to certify the safety and validity of that test. I don't think anyone is concerned that they're cross-contaminating their samples or working in a filthy lab over at 23andMe -- they're concerned that the whole design of the test is broken.

Again, most of this stuff is sketchy science. You can do the microarray experiments in a proper (e.g. "CLIA certified") lab, and still have a useless, dangerously misleading test.

crusso4• 13 years ago

If they can't show that the associations they're reporting are valid, what purpose does it serve to report them?

Showing correlations and study information related to markers found in your genes doesn't serve a purpose?

Shouldn't that be up to people to decide with their $99?

Tell me that this information and their method of finding correlations isn't doesn't have huge potential benefits: https://www.23andme.com/about/factoids/

kohanz3• 13 years ago

Nobody is denying that there is potentially a lot of value in 23andMe's genetic reporting.

The point is, like all other medical tests before them, they need to go through the proper regulatory channels to ensure their safety and validity. It decidedly should not be "up to the people to decide" because we know that society at-large is not informed enough to make these decisions.

What makes 23andme different from all of the other medical tests and technology out there?!

jlanter• 13 years ago

Has it occurred to anyone else that 23andMe is possibly playing a PR game here? I find it hard to believe that they and their lawyers thought they could ignore the FDA for months. I never heard of the company before yesterday and now I'm seeing them everywhere.

dublinben• 13 years ago

They've been a well known company for several years. I don't think this kind of publicity helps them very much, since they're specifically banned from selling their product at the moment.

refurb3• 13 years ago

23andMe has two choices:

1) Stop making health claims about their service 2) Produce the evidence required to substantiate their health claims

The FDA has very high standards when it comes to health claims, see the Warning Letter for Cheerios: http://www.fda.gov/iceci/enforcementactions/warningletters/u...

Fiber can help lower cholesterol. Cheerios have fiber, thus why not make the claim that Cheerios can help lower your cholesterol?

The FDA is not big on "extrapolation". If B causes C and your product A causes B, you can't claim it also causes C.

How did General Mills respond? The simply modified their claims:

3 grams of soluble fiber daily from whole grain oat foods, like Cheerios and Honey Nut Cheerios, in a diet low in saturated fat and cholesterol, may reduce the risk of heart disease.

Zigurd3• 13 years ago

Processed food vendors get away with some pretty awful stuff, like Vitamin Water claiming their customers couldn't possibly confuse their product with something healthy.

refurb3• 13 years ago

The FDA can't be everywhere, but Coca-Cola (makers of Vitamin Water) got a Warning Letter from the FDA back in 2008 over their health claims for diet Coke!

http://www.fda.gov/ICECI/EnforcementActions/WarningLetters/2...

md2be• 13 years ago

23andMe didn't need to make medical statements to get most of the early adopters on board. Lets face it they made a dumb decision to do so.

masklinn• 13 years ago

And then they made a dumber decision by stonewalling the FDA.

apaprocki• 13 years ago

For the more technical customer of 23andMe, there probably isn't a need to do all of the interpretation themselves. There are tools which will take the raw data and query public databases to build together a similar (and sometimes more up-to-date picture). I don't think the FDA would want to shut down something like SNPedia.

But, there is a lack of quality tools available to query public genomic databases based upon raw data received from services like 23andMe. If anyone is a 23andMe customer, try out Promethease[1]. There is probably room for selling software to DNA service customers, but it probably isn't that big of a market (yet?).

An example, well-designed tool for viewing raw data files is 4Peaks[2].

[1] http://snpedia.com/index.php/Promethease

[2] http://nucleobytes.com/index.php/4peaks

tgb• 13 years ago

Having never heard of SNPedia before yesterday and still not being sure of it's correct pronunciation, I have no recourse but to say it as "tinpedia."

bronbron• 13 years ago

More likely SNP stands for single nucleotide polymorphism, usually pronounced "snip", so "snipedia" is probably more apt.

Or maybe I'm totally wrong.

moioci• 13 years ago

Dear Ms. Turnipseed, As you are no doubt aware, you recently assigned young Snidely a grade of "F" for the assigned paper on Richard I due some 6 months ago. I concede that he did not, in fact, produce such a paper and acknowledge that you offered to work with him outside of class on several occasions. But, as you must agree, he has attended class faithfully, has taken copious notes, and has even been in the library on more than one occasion. Clearly, his chosen topic, "Richard I: Lion Heart or Lily Liver," promises a worthwhile exploration of an important subject. I'm sure you will agree that, had he laid pen to paper (or digits to keyboard), he could have produced a more than adequate paper. Accordingly, I insist you assign him full marks forthwith.

Sincerely etc. etc, Snidely, Sr.

fedups• 13 years ago

However, I worry you want 23andMe to independently validate each of these results before informing customers about them. Worse still would be a demand that the company validate the results with its specific testing kit. It would be cost-prohibitive even for a single company like 23andMe. It would destroy any possibility of competition; even if one company did it, new startups would not be able to.

Looks like a case of the FDA doing what it does best--ensuring the task of medical innovation remains with big pharma, and leaving Americans stunned at the results.

tokenadult• 13 years ago

23andMe has plenty of money behind it through its investors. 23andMe has been trying to play the game of Big Supp (the "natural supplements" industry, which has billions of dollars of revenue) by trying to imply health claims that it can't back up with evidence. That's nice work if you can get it, but 23andMe doesn't have the same sweeping exemption from regulation that "natural supplement" makers have.

http://www.sciencebasedmedicine.org/new-concerns-about-the-s...

RankingMember• 13 years ago

You're a doctor without a primary care provider? Why/what?

Shinkei2• 13 years ago

Physician here. (Disclosure: I have no financial interest in this or any related company.) (Edited out some 'heated' remarks)

Medicine is NOT, I repeat, NOT 'hard' science. Most of the 'normal' lab values we use when reporting things like elevated white blood cell count, anemia, etc. are based on population studies with the average considered normal and setting extremes as 'abnormal.' You will find that different hospitals even have different values of abnormal. Not to mention, for cutting edge therapies (Oncology, I'm looking at you) there is no 'standard' of care and there is a high variability between physicians. My point is that, I could go on and on about how much the field of Medicine is subject to the vagaries of stuff like regional practice, academic pedigree ("Oh, we at Hopkins always do it this way, but Mass Gen does it that way."), as well as bias like anecdotal evidence (the plural of anecdote is NOT data!) or just plain mis- or dated information. If you are out of med school/training for only a couple years, the amount of new therapies, lab tests, etc. can be staggering and if physicians aren't learning about the new, then they quickly become dated.

So, what am I saying? The FDA is holding 23andMe to an unreasonable standard. If your doctor saw the same test results, they might tell you the exact same correlation (oh, you are predisposed to Alzheimer's or Parkinson's). Tests are incredibly complex... I mean we still don't have a perfect pregnancy test! Again, there is sensitivity, specificity, with false positives and false negatives. If you are unfamiliar with how inaccurate medical tests can be, please read up on some of those items. This is the best we've got for now. If people can order pregnancy and HIV tests over the internet, there's no reason they shouldn't be able to find out they 'might' have a predisposition to high blood pressure. I would argue 23andMe is not doing something so radical... as far as I'm concerned, pregnancy and HIV tests should NOT be over the counter if we are going to limit genetic screens because there is a LOT of work to be done in positives of either.

Unconvinced? Well consider this. Over the counter Tylenol products are responsible for about 500 deaths per year in the US. Do you believe that Tylenol should be prescription only just because a minority of consumers make mistakes? If not, then you must conclude that over the counter sales of 23andMe should be allowed unless the FDA can show it has ACTUALLY harmed people.

http://www.mercurynews.com/business/ci_23974801/tylenol-warn...

FireBeyond2• 13 years ago

You seem to completely miss the point, however - the FDA is asking (telling) 23andMe that they need to provide the documentation of the efficacy of their testing process. Accuracy and margin for errors, and NOT, as you seem to be implying, “forbidding the use of such a service unless accuracy can be guaranteed”.

Yes, 23andMe makes claims, and uses phrases like “likely”, “most often”, and so on. The FDA is asking to be provided the evidence that such statements are accurate, or how accurate they are.

Shinkei2• 13 years ago

Check their webpage:

https://www.23andme.com/health/all/

Under each item, there is an extensive explanation including the technical report regarding the exact details of what is being tested with citations.

This is the list of FDA approved 'genetic' tests:

http://www.fda.gov/MedicalDevices/ProductsandMedicalProcedur...

It's a sad mismatch between practice and regulation. Industry is WAY ahead of regulation. Very common tests performed by Clinical Geneticists are not included in this list. Yes, they will give you the disclaimer, but the data is there.

jaynos• 13 years ago

"Many of my colleagues do not know of the existence of personal genomics, and believe that genetic testing is a once-in-a-blue-moon procedure to be ordered only when there is high clinical suspicion for a rare disease such as cystic fibrosis."

If true, that is quite embarrassing to the medical community.

A Letter I Will Probably Send To The FDA on 23andMe

Related Stories