Is the FDA Too Conservative or Too Aggressive? Analysis of Clinical Trial Design

http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2641547

by nkurz • 11 years ago

63 33 11 years ago

papers.ssrn.com

kesor• 11 years ago

A family member of mine has tried to participate in a clinical trial. The trial also forces him to forgo any existing medication, basically making his illness intolerable. There is no indication in the trial if he is being served with real medication or a placebo, but his life has been extremely affected just by participation. Naturally he quit the trial. How would this statistical analysis address these common behaviors where ill people would just not participate in trials when certain illnesses are involved?

KingMob• 11 years ago

There's a whole statistical literature on survival bias. The basic equations all require "independent and identically distributed" data loss. As soon as that's not the case (e.g., more placebo patients drop out than treatment patients), you have to compensate for that.

That being said, I suspect this analysis is operating at a higher level where survivor bias isn't as relevant, because it's looking across many studies at once.

wcbeard10• 11 years ago

A good summary and commentary here

http://www.marginalrevolution.com/marginalrevolution/2015/08...

reasonattlm• 11 years ago

Also: http://fdareview.org/

"Medical drugs and devices cannot be marketed in the United States unless the U. S. Food and Drug Administration (FDA) grants specific approval. We argue that FDA control over drugs and devices has large and often overlooked costs that almost certainly exceed the benefits. We believe that FDA regulation of the medical industry has suppressed and delayed new drugs and devices, and has increased costs, with a net result of more morbidity and mortality. A large body of academic research has investigated the FDA and with unusual consensus has reached the same conclusion. Drawing on this body of research, we evaluate the costs and benefits of FDA policy. We also present a detailed history of the FDA, a review of the major plans for FDA reform, a glossary of terms, a collection of quotes from economists who have studied the FDA, and a reference section with many webbed links. A more detailed table of contents follows. We are happy to receive comments and criticisms."

kefka• 11 years ago

Yes, and bureaucracy never petitions for less bureaucracy. Only more. And we see this in multiple areas, and not just the FDA.

FDA bureaucracy equals more people dying because of slowness.

FCC bureaucracy equals regressive rules on radio transmitters and receivers, contrary to what the people in the field actually use/do.

FAA bureaucracy leads to idiotic laws on quadcopters/drones/personal UAVS that affect the landscape of the technology to the point where uploading a video to youtube is "against the law".

Patent law/USPTO bureaucracy leads to inane patents like XOR patent, "Slide to Unlock", fillet and round design patents (patent a 1/4 circle), One-click, and other patents that are glaringly obvious to anyone in the field. Yet, these enact heavy transaction fees; that is if the companies play along.

jcranmer• 11 years ago

I'm wary of relying on economists to assess the utility of the FDA. The alternative to the FDA they mention is basically self-policing. Except we already have a drug-like industry that relies entirely on self-policing: the "health supplements" industry, which the FDA is explicitly banned by legislation from regulating. And the precedent of the health supplements industry is extremely disconcerting--companies are basically legally lying through their teeth (it's not illegal if someone else, e.g., Dr. Oz, says it, cause they aren't advertising, they're exercising free speech), and they sometimes don't even bother to stuff their products with what they say the active ingredient is.

The problem with drugs is the placebo effect. Evidencing efficacy beyond the placebo effect requires large, controlled trials (the infamous Phase III clinical trial) that end up being correspondingly expensive; nothing less is sufficiently powerful. Because of placebos, you can sell expensive nothings, and it will appear to work sufficiently well that reputation wouldn't suffer.

cryoshon• 11 years ago

I've been on the science side of a number of FDA clinical trials for a few years now.

The FDA is too conservative with many metrics, and too aggressive with others. The clinical trial pipeline is tortuously slow for everyone involved-- the EU equivalent agency has done a lot to accelerate the process fruitfully. Eclipsing these FDA malregulations is the problem of transparency; if the FDA forced clinical trials to publish 100% of their data rather than only the positive results, we'd have a better medicine producing industry.

minthd3• 11 years ago

Is there any realistic possibility Americans could use drugs approved in Europe ?

dragonwriter• 11 years ago

Americans can obviously use drugs approved in Europe in Europe; drugs approved in Europe but not the USA cannot generally legally be imported, sold, or prescribed in the USA, however.

kirsebaer2• 11 years ago

Any American MD can ask the FDA for "expanded access" to use a medication not approved by the FDA. 99% of the time the FDA grants expanded-access requests.

However, a recent review in NEJM explained how it is time consuming and expensive to apply for expanded-access and often the pharmaceutical companies refuse to distribute the drugs even after the FDA grants expanded-access.

http://www.nejm.org/doi/full/10.1056/NEJMhle1409465

"Compassionate use" is a separate, but related concept with different rules than "expanded access". And in the EU expanded access is often called "named-patient".

vitd2• 11 years ago

Yes. My wife was given medication not FDA approved by her doctor. She had to order it from a pharmacy in Canada, but it was all legit. The (US) doctor wrote her a script and filled out some sort of paperwork for the government, as I understand it.

And no, her doctor isn't some quack who regularly prescribes unapproved medicine. Apparently this particular drug is approved in Canada and Europe, but the company that makes it is small enough that they haven't yet decided to apply for FDA approval because of the cost. (Or at least, that's what I've been told.)

astazangasta• 11 years ago

if the FDA forced clinical trials to publish 100% of their data rather than only the positive results, we'd have a better medicine producing industry.

I thought this was actually already the case with clinicaltrials.gov and it is apparently having a profound effect: http://www.nature.com/news/registered-clinical-trials-make-p...

DennisP2• 11 years ago

So the FDA is too conservative for life-threatening conditions, and not conservative enough for relatively mild conditions. That probably fits pretty well with most people's intuitive sense of things; it's interesting to see the math works out that way.

iskander• 11 years ago

My small sample size of experience with the FDA: they seem to be very receptive to novel therapies for terminal illness and it's surprisingly straight-forward to get a Phase I trial approved.

There's no static rule uniformly enforcing the initiation of trials, rather your IND (investigational new drug) application is reviewed by experts in the relevant field to determine whether (1) is there a plausible reason to believe your new therapy helps and (2) is it unlikely to kill or severely injure patients.

I can't think of a better way for things to work for late-stage cancer or other untreatable terminal illnesses.

transfire• 11 years ago

TriCorder X-Prize? FDA has done nothing but hobbled it's development. DNA screening for the masses? FDA shut down 23andMe. Terminally ill and willing to try an experimental drug? FDA says, "too bad" (and the Supreme Court agrees).

tim333• 11 years ago

They should have a principle of maximising the public good which I feel would allow some of that stuff.

kirsebaer2• 11 years ago

The FDA approves requests to use non-approved medications 99% of the time. When you hear about someone unable to access an experimental drug, it is almost always because the pharmaceutical company does not want to make the drug available early (for various reasons).

At the meeting, a senior FDA representative indicated the agency has approved over 99 percent of expanded access requests submitted via single patient or emergency INDs since 2009, suggesting the regulatory agency is not a major barrier to expanded access. As such, provided the access request is reasonably related to the potential benefits of the drug, the biopharmaceutical company is almost solely responsible for the decision and liability regarding whether to grant expanded access to an individual.

http://healthaffairs.org/blog/2014/07/31/individual-patient-...

rayiner2• 11 years ago

They didn't "shut down" 23andMe. They stopped them from marketing unproven health diagnoses derived from the DNA tests.

fedups2• 11 years ago

I think it went beyond a marketing ban, unless 'marketing' extends to selling and displaying health related information.

frostirosti• 11 years ago

There's so much conflict within the FDA that their opinion should carry no real weight.

mjfl• 11 years ago

A case study on FDA from my father, a cardiologist: A recent advancement in replacement heart valves reduces mortality rates from 10% in high risk patients (old and very unhealthy) to 0.9%, from 5% in medium risk patients to 0.4%. This studies were confirmed through clinical trials run by very well respected doctors. Nevertheless, the FDA still required their own set of trials to confirm the safety of the procedure. This was 5 years ago. The FDA first ran a set of trials, not on standard patients but only those that were deemed "inoperable". They spent a few years working that out and found that the device succeeded, but the approval was only given for patients that were inoperable. Then the FDA covered patients that were high risk, spent a few years and found again that the device was hugely beneficial in mortality rates and the device was approved for high risk patients. We are currently in the waiting for the FDA to approve the device for medium risk patients. In the mean time, my father had a medium risk patient who needed a valve replacement. He could not use the new device, so he sent the man to surgery for a standard valve replacement with 5% mortality risk. The man died on the table, becoming part of the 5%, and my father insists that he would have survived if they had been able to use the new procedure. In Germany there are less regulations, and so they have been using the new technique for 5 years with great success. Because of things like this, you'll commonly see medical procedures in Europe that are 5 years ahead of their counterparts in America, all because the FDA insists on using such plodding methods to verify safety.

chrisgd• 11 years ago

Couldn't that device be used off-label?

DennisP2• 11 years ago

I know an M.D. who travelled to Germany to get a medical implant (for himself) that's not yet approved in the U.S., even though it meant paying $90K out of pocket.

regressiveparty• 11 years ago

My experience with the FDA:

I'm "congenitally limb deficient" below the elbow. Grew up in the 90s using horrible myo electric prosthetic limbs that had 2 hour battery life and were in all practicality a glorified claw machine. I ditched that as soon as I realized they were actually a hindrance.

In 2006, there was a TED talk with Dean Kamen about a prosthetic arm developed by DARPA. It blew my mind and got me finally interested in wearing a prosthetic again. So much promise in a field that had been in a coma for the last 20 years.

That arm never happened. Why? Because it took the FDA 7 years to approve it. The project now is dead in water and the group responsible for building it is trying to literally give away their designs. Why did the FDA have to approve a prosthetic limb? Well, because it touches your skin. Yes, the silicon on the back had to be tested because of skin contact.

I'm now the lucky recipient of BeBionic 3 arm, which is great. Worst part about it? It uses radio frequency to control the hand and has a tendency to go haywire. I asked why they didn't just use Bluetooth and he said that would require another 2 years to get FAA approval. Ugh.

Anecdotal, I know.

Vexs• 11 years ago

See, this just seems overly investigative. If they were physically fusing the arm to your body, sure, that's a reasonable time for the FDA to investigate. But just because it is related to your body in any way doesn't warrant that sort of careful consideration. Similarily, if the radio was controlling a pacemaker, I wouldn't want it using bluetooth, I'd want something FDA approved. If it's an arm, I probably wouldn't want bluetooth (stupid pairing issues....) but something like the xbee format would be great, not some proprietary and less reliable RF protocol.

AnimalMuppet• 11 years ago

This is not an easy problem. If the FDA is too conservative, people will die who could have lived. If the FDA is too aggressive, a different group of people will die who could have lived. And no matter where the FDA is on the spectrum (insane extremes excluded), there are people in both groups who die. And there are people who cared for those in both groups, who scream about how the FDA is doing it all wrong, and who have diametrically opposed views on how the FDA should change.

Maybe the test should be whether the screaming is equally loud from both sides...

crusso• 11 years ago

Maybe if the FDA were only making recommendations then people could decide for themselves whether to go with the doctor who advising an FDA-approved treatment vs going with some other doctor who is advising something else?

minthd3• 11 years ago

Here's another example:

There's oragenics, which developed a vaccine against dental carries - a disease that affects 5 billion people. Something that i'm sure everybody would really like to have.

And this is why we don't have it ? In their own words:

" Regulatory Status:

We initiated our first Phase 1 clinical trial in April 2005, but we found it difficult to find subjects who fit the trial’s highly cautious inclusion and exclusion criteria, particularly with respect to the subjects’ lack of dentition. We concluded this trial early after enrolling only two of the 15 planned subjects. The FDA then recommended that we revise the protocol for the evaluation of ten healthy male subjects, ranging from 18 to 30 years old and with normal dentition, in an institutionalized setting. After we submitted additional information, the FDA issued a clinical hold letter in June 2007 for the proposed trial with the attenuated strain, citing the need for a plan with respect to serious adverse effects; a plan for the eradication of the attenuated strain in trial subjects’ offspring; and a required pregnancy test for female partners of subjects. We submitted additional protocols in response to the FDA’s concerns. In August 2007, the FDA issued a clinical hold letter with required revisions to the protocol for offspring of subjects. We submitted a response to the clinical hold letter in September 2007, and the FDA removed the clinical hold for our Phase 1 trial in the attenuated strain in October 2007.

While we commenced a Phase 1b clinical trial for SMaRT Replacement Therapy during the first quarter of 2011, the very restrictive study enrollment criteria required by the FDA made the enrollment of candidates meeting the restrictive criteria difficult. Due to the enrollment difficulty we encountered with our initial our Phase 1a clinical trial and now with our phase 1b clinical trial, we determined to discontinue pursuit of our Phase 1b clinical trial."

Such a shame.

http://www.oragenics.com/?q=cavity-prevention

Is the FDA Too Conservative or Too Aggressive? Analysis of Clinical Trial Design

Related Stories