Why are clinical trials so expensive? Tales from the beast’s belly

by wawayanda • 4 years ago

192 99 4 years ago

pfdietz3• 4 years ago

One thing not mentioned: people are paid to participate in clinical trials. For example, in a vaccine clinical trial I know about, participants are paid over $1000 (in dribs and drabs over the course of the trial, as a retention mechanism.)

With tens of thousands of participants, this cost adds up.

idealmedtech5• 4 years ago

Patient compensation is intentionally limited to prevent economic incentives from interfering with informed consent; pay people too much and they may participate in research that they would have passed on. Also, speaking from experience, patient compensation is a very small amount (5% or less) of overall cost. The vast majority goes to clinical staff costs, room costs, hospital overhead, labs etc.

pfdietz3• 4 years ago

That's not a vaccine trial though, right? I'm figuring those need more participants and have more retention problems, as well as less risk of informed consent problems.

boole1854• 4 years ago

Patient compensation is intentionally limited to prevent economic incentives from interfering with informed consent; pay people too much and they may participate in research that they would have passed on.

I'm not sure I understand the logic here. How does agreement suddenly become less informed or less consensual because the participant gets paid more? Sure, if they get paid more, they might be willing to accept more risk or do something more unpleasant. But that seems fair, not immoral. Why would it be more fair to say "yes, perhaps this trial is risky, but we only want participants who don't demand to be compensated much for the risk they take"? If anything, maybe that approach leads to less informed consent by filtering out the people who actually understand the risk and would have only agreed to participant in exchange for more compensation!

Should we also be morally offended any time an employer raises wages in order to attract more workers?

pinky14172• 4 years ago

That's true. Good Clinical Practice guidelines say that Institutional Review Boards must evaluate compensation is not too high otherwise it constitutes "undue influence".

...but I think that's a terrible idea. I understand that, to come up with a far-fetched example, high compensation could result in some weird edge cases where, for example, a Phase I trial with healthy volunteers and a non-preventive drug candidate (e.g. an antibiotic rather than a COVID vaccine), but you're confident that drug is going to kill 99% of the volunteers so you offer $10 million to each volunteer. A totally informed volunteer might view this as something akin to a life insurance policy that pays out to the family even upon suicide. I don't personally see an ethical problem with that so long as volunteers are really well-informed, but I could see how others and the law would be uncomfortable with the ethics of such a situation.

But aside from that far-fetched situation, it seems harmful to limit compensation... especially if it's true that patient compensation is a small part of overall cost. Perhaps if every trial doubled compensation to 10% of today's overall costs, absolute overall costs would decrease because less effort has to go into patient recruitment.

I do see one real-world issue with high compensation, but it has nothing to do with ethics. It has to do with statistical interpretation. If you start dangling too much money to participate in a trial, you may change the kind of people who participate in the trial. These individuals might not be representative of the actual users of the drug. The problem is similar to running a drug trial in the US versus Japan. The US has a pretty diverse population with respect to genetics. Japan, in contrast, is almost entirely ethnic Japanese. If you were to somehow succeed in using Japanese trial data to market an acne cream in the US, the same drug that cleared up Japanese faces might result in Venezuelans having their noses fall off!

However, I'm cautiously optimistic that higher compensation could make trials more representative. In my mind, I could see how African-Americans, who tend to participate in trials at a lower rate than their Caucasian fellow humans, would, if only for the shameful history of the Tuskegee Syphilis Study, require greater compensation to take the risk of participating in a trial.

NDizzle• 4 years ago

One point in there - data management. It varies in cost because on the upper end, that includes support for data change requests for the life of the trial. Zillions of people involved, human error, need paper trails and sign offs to fix most things.

bmj• 4 years ago

This is usually downstream of the CROs, or, at least contracted out. What's most interesting to me about the post is that the author doesn't mention any efficiencies gained by moving to electronic data collection. But, I also understand why it isn't mentioned -- I work for an EDC/eCOA provider, and our biggest competitor for new business isn't our competitors in the space, it is paper-based data collection.

orzig• 4 years ago

This is incredibly important if you want to understand healthcare. Every little thing is expensive in a way that is hard to imagine before you get into the field.

refurb3• 4 years ago

I worked in clinical development and the rough estimate of phase 3 clinical trial costs (this is a few years back) was $15,000 per patient per year.

Depending on the trial design it can be less (less frequent follow up, less invasive testing) or a lot more (say doing multiple MRIs or other expensive imaging).

It’s kind of mind blowing when you think about the really big trials for medications like cholesterol. Trials are often 30,000 or more patients, in two trials for 2-3 years.

You’re roughly talking about $2B or more just for phase 3 (ignoring all the earlier trials and preclinical work).

s1artibartfast8• 4 years ago

Another huge cost is comparator products. Some trials budgets I have seen spent 100-200+ million purchasing the drugs they are trying to outperform. This is for trials with <1000 patients.

iancmceachern• 4 years ago

I worked at a LVAD company (think artificial heart) that was doing a trial, they had to buy dozens (hundreds?) of their competitors LVADs, at $120k a piece

idealmedtech5• 4 years ago

There's a reason why we ended up putting our trials together essentially by trial and error! You learn a lot by writing your own clinical protocol, interacting with the FDA etc. It's an intimidating process at the outset, but if you can save $200k on a first-in-man study by not contracting a CRO, that's gonna extend your runway by a lot!

seidleroni2• 4 years ago

A fair point, but with the FDA responses taking up to 30 days you burn a lot of runway.

ska• 4 years ago

They have a 180 day review commitment. A 30 day delay on their side would be unusual and burn a big chunk of that clock.

s1artibartfast8• 4 years ago

That's why you have several Type B or Type C meetings during development to work out the kinks before you submit a protocol.

vibrio• 4 years ago

I don't completely disagree with your point, but for one's clinical regulatory strategy, perhaps "trial and error" isn't the best branding to appeal to investigators, patients, regulators or investors.

s1artibartfast8• 4 years ago

Are you under the impression that that's how they branded things?

idealmedtech5• 4 years ago

Perhaps I was being too self deprecating! We worked closely with the FDA through multiple QSub meetings to make sure it was up to snuff, and did a lot of legwork on the backend to make sure we understood what they were asking for. 3 or 4 revisions later, we had an approval!

adultSwim• 4 years ago

If you are worried about $200k, do you have enough money to bring your product to market?

idealmedtech5• 4 years ago

When we started putting the trial together, we were still self funded. In medical device, to get to series A you need human data, so by doing the trial protocol ourselves, we avoid the chicken and egg problem of needing money in order to raise money. Once you get human data, your valuation skyrockets.

seidleroni2• 4 years ago

As someone working at a company that has run several clinical trials, this really rings true, especially the segment on language translations.

turnsout2• 4 years ago

Great summary! And yet, for all their problems, CROs are sometimes still the fastest alternative. We did a clinical trial for a software platform a few years ago, and hoped to partner with a prestigious hospital system to run the trial.

The hospital partner was 100% on board—our protocol just had to pass their internal review board (IRB). IRB is a black box, and they could only give us a vague estimate of how long this would take. We had a huge team, and every week we sat around waiting cost about $200k.

Finally the client couldn't take the uncertainty and burn rate anymore, and they reached out to a CRO they had used in the past. The CRO's IRB had a fixed timeline (2 weeks?), and we were onboarding patients a short time later.

1auralynn• 4 years ago

Yeah the lesson with getting IRB approval if you want it done in less than 6 months is to always use an external IRB (if possible). Institutional IRBs are sloooooowwww.

MichaelZuo• 4 years ago

Why couldn’t the team be tasked on something else while waiting?

turnsout2• 4 years ago

We did keep working on the platform, but from the perspective of the client, we already had the MVP for the trial. So to them we were working on “nice to haves” for 200k/wk

BurningFrog• 4 years ago

Soap box: IRBs are one of many things deteriorating modern science, and in great need of reform.

tsbischof2• 4 years ago

Any particular issues? I am most opposed to the rigid 30-day cycles and the need to spend one just to get on the agenda. But in terms of the questions asked and the due diligence required, I have not found our IRBs to be onerous

xiphias2• 4 years ago

While there are lots of new biotech tools, most of the illnesses left to treat are caused by aging, when the human system falls apart from the interaction of old parts, therefore super hard to treat.

At this point I gave up waiting for a medical solution for my chronic illness, and have a better hope that I can live 15-20 years with it and rejuvenate my body after that time period.

olieidel2• 4 years ago

This is so, so interesting. Thanks so much for posting this!

The author already touched upon a few hypotheses why the industry has arrived at this state. There's one I'd like to highlight: The author, now having gone through this experience, would actually be in a perfect position to found their own CRO and implement all their efficiency learnings there. However, they probably look back upon this experience and think it was super painful and would prefer to do fun stuff like developing software instead.

That, for me, is the main reason. There are smart, structured, pragmatic people who got acquainted with this industry, but they choose to leave it.

And that's sad!

So the solution would be to find a group of smart, structured, pragmatic people with a high tolerance for bureaucratic pain and give them the resources to make this industry more efficient.

And here's my plug: That's exactly what we're doing at OpenRegulatory [1], albeit for medical device software and not for clinical trials. Among other things, we've published all our document templates for free (also on GitHub), host a free Slack community for companies to help each other out, and offer a compliance SaaS with transparent and fair pricing. It's still a long way, but it's a start :)

[1] https://openregulatory.com

BeefySwain• 4 years ago

Your company is very interesting. I briefly looked over the website, but couldn't tell what country(ies?) you operate in? My assumption is the regulatory requirements of each country are dramatically different, so seeing the pricing in not $ makes me think it's unlikely you offer services in the US, for instance.

jtwaleson• 4 years ago

They operate mostly in Germany, but the Slack community around it is broader, I think 90% is EU based though.

olieidel2• 4 years ago

As jtwaleson already correctly replied (thanks!), we're based in Germany and are mostly focused on EU regulations. The "good" news is that if you gain regulatory approval within the EU you can bring your device to market in any EU country.

There's quite an overlap with US (FDA) regulations but we're less experienced there.

Most startups (at least in my bubble?) focus on one of those two areas.

mandmandam• 4 years ago

This all sounds so familiar.

How many industries could these sentences descrive, stripped of context?

Or more easily, are there any industries where this is not the norm?

an industry which is severely lacking in human capital and fully captured by bureaucratic tendencies
going through this process is an extended exercise in sheer absurdity.
meaningless bloat
a farcical “data-driven” scoring process
there was an intermediary clinical organization we attempted to work with which was so obviously attempting to grift money from our coffers, the entire relationship escalated into the level of back-and-forth legal threats
One cannot help but get the impression that this is an industry which is, at best, severely lacking in anything that could be called “human capital” and, in reality, simply deeply unserious
despite the very large amount of boilerplate content that every single actor in this ecosystem has spent thousands of hours of manpower developing, there seem to have been very few actual gains made in simplifying common tasks or processes!

... Et cetera.

To me, this sounds a lot like processes in education, teaching, energy, environmental protection, agriculture, research, arms, media, law, IT, etc.

I don't believe we can afford to remain this stupid, however much money the inefficiencies are making for those at the top.

knodi1233• 4 years ago

Or more easily, are there any industries where this is not the norm?

porn?

Or in general, any business with highly mobile customers/employees and narrow profit margins.

photochemsyn• 4 years ago

Interesting article, but the lack of mention of 'patent' or 'intellectual property' sort of stands out. There may be many out-of-patent drugs that are effective treatments for various conditions, but typically clinical trials for alternative uses of out-of-patent drugs are not pursued as anyone could then manufacture and sell them.

This is not new - it dates back to the origins of the modern for-profit pharmaceutical industry, such as with Bayer's discovery of the anti-bacterial action of sulfanilimide (patented 1909) in the early 1930s (by which time patent had expired). Bayer made a derivative of sulfanilimide, i.e. Prontosil (patented in 1932) and tried to convince the world that this was the more effective product. In reality, it just broke down to sulfanilimide once ingested, that being the active species.

As far as clinical trials in the USA, there have been numerous scandals regarding lack of consent, failure to halt trials immediately on indication of severe side effects, etc. Major stories about this came out in 2005, 2008, etc. Then there's the world of unregulated third-world clincal trial testing, and actual falsification of clinical trial data by contractors hired to run trials for various Big Pharma outfits, such as this from 2022.

https://endpts.com/clinical-trial-coordinators-sentenced-to-...

"Tellus, which has worked with big-name pharma sponsors in the past like Pfizer, Gilead, Takeda, Boehringer Ingelheim and Eli Lilly, initially came under scrutiny for two trials related to opioid dependency, two for irritable bowel syndrome, and one for diabetes."

That's just organized crime 101: hire a contractor to do the dirty work, keep your own hands clean, claim ignorance, act shocked, promise reforms, etc.

Best solution I've heard of is a move to open-source (patent-free) drug discovery with federally financed clinical trials. Since almost all new drugs are discovered at universities with federal funds (see remdecivir, mRNA vaccines, etc.), before being exclusively licensed to private interests, the quickest way to that is to revoke Bayh-Dole and state that universities must non-exclusively license their discoveries to any parties for a flat fee.

refurb3• 4 years ago

The FDA will still give you regulatory exclusivity if you get an off patent drug approved in a new indication.

https://www.fda.gov/drugs/development-approval-process-drugs...

Orphan drug exclusivity is 7 years. The FDA will not approve a generic version of that drug for the same indication for 7 years after your approval.

Of course the challenge is that doctors are free to prescribe whatever they want, including compounded versions of your drugs.

There are ways around it (formulation), but it's not like there is no option at all if you don't have a patent.

invitrom• 4 years ago

I am eagerly waiting for the clinical trial on BC007 for Long Covid. As a chronically ill patient the slow pace of research is almost unbearable.

jokowueu• 4 years ago

There was a group buy for it a while ago on a discord server . Iirc it was about 10k$ for a single course .

Have you tried stellate gangelion block for your long covid ?

pinky14172• 4 years ago

I worked at a small-to-mid-size clinical research organization for a hot minute. That company had its own set of - let's say "unique" - problems*, but the author touched on many of the issues I saw in my short experience at a CRO. I was in an odd (for me), pseudo-managerial, HR-type role at the company (a role in which I'll be the first to admit I wasn't good at) so perhaps I can offer some additional perspective.

But first, I want to give some background on an important part of the CRO world: the "clinical research associate" or "CRA". CRAs are are the CRO's meat-and-potatoes, boots-on-the-ground, core, etc. after contracts have been signed. A CRA is the person who actually goes to clinical trial "sites" (e.g. a doctor's office, a hospital, or, even a dedicated facility for clinical trial participants). The CRA goes to each trial site, talks to the doc, makes sure participant recruitment is going well, checks/writes documentation, and follows up as the trial goes on. In other words, the performance of the CRAs strongly & directly affects the performance of the clinical trial.

Anyway, my top issues with clinical trials today are:

(1) Poor patient recruitment - this is a major problem. There is, of course, lots of variability between trials and CROs with respect to recruitment, but it should be telling that "trial rescue" is often done (at least where I worked) as a result of the incumbent CRO's inability to recruit enough patients.

(2) (Related) Limiting pay to study participants: although it's up to the institutional review board's interpretation (IRB, an ostensibly independent entity that's supposed to sign off on the trial's ethics), good clinical practice (GCP) guidelines states that paying participants too much would be "undue influence". I have no idea how this plays out in practice, but I think it's a bad guideline. Most of the other parts of GCP are good things - stuff like informed consent and not coercing subjects - but this one is (again, in my opinion) probably raising trial costs through recruitment delays or total non-recruitment, even accounting for the extra cost of paying participants more.

(3) Clinical Research Associate (CRA) training & hiring: this isn't unique to the CRO industry but it seems like a bigger problem than in other industries. I've seen US median salary ranges from $70,000 to $120,00 from publicly available sources but my time in the industry rarely demonstrated numbers that high. Although CRAs don't treat patients directly, it is a role that requires medical knowledge, attention to detailed paperwork, and good communication skills. There seemed to be a lot of cutting corners on training & hiring CRAs - e.g. hiring highly experienced CRAs from India and putting them into entry-level roles in the US on the cheap. I don't mean to say there aren't great experienced CRAs from India who are also great when dropped into an entry-level CRA role but doing so as a matter of unofficial policy is a bad idea: India's drug industry has a different regulatory environment (although there is significant cooperation between FDA and Indian manufacturing facilities) and, perhaps more importantly, there will be a wide variation in communication skills simply because english is a second language (or, in the case of India in particular, "Indian english" has taken on a life of its own).

(4) Over-promising, under-delivering - true of any contractor even outside the CRA industry. Even when working on a fixed price basis, there are perpetual "change orders" and arguments over change orders.

Side-note: if you want to learn more about the process by which drugs come to market, I strongly recommend New Drugs by Lawrence T. Friedhoff. It's short and the HN crowd won't have any problem understanding. Amazon link: https://www.amazon.com/New-Drugs-Insiders-Scientists-Investo...

* "Unique" problems in the sense that they were personality-based. I won't go into specifics nor do I wish to speak negatively of my prior employer who, to their credit, took a chance on me that just happened to not work out. But I will give some general advice: if, during a job interview or in early conversations, senior management insults his/her own family and dumpsters their own employees, you're going to have a bad time.

esel2k• 4 years ago

I would add to your list of problems: Decades of easy-money milk-the-cow style companies (Often CROs) with very outdated systems with folks sittings in cheap countries to do some data change requests and manual tests. Truely horrible and outdated tech, mixed with bad salaries, high turnover and bad management. I used to work for a company were because of timezone issues a wrong date was entered, CRA requested correction (edit manually, sign, scan) and wrong assumptions etc we had to change 7 times the whole thing and send CRA onsite and recorrect the mess. In short: Inefficient mess.

I left that industry quickly and can say I that others are better. Lab tracking is way better with diagnostic samples with private labs or when mixed with good logistics partners. It is not everywhere as bad as this.

TLDR: add outdated tech, greed and bad management

selecsosi• 4 years ago

An interesting company I talked to a while back working on (1) in the spaced is deep6.ai. Interesting approach and there is a lot of work that can be done here sourcing patients more effectively. I think they said something like only 10% of studies are able to be carried out due to lack of patient recruitment.

FrontierPsych2• 4 years ago

what do you mean by ""Indian english" has taken on a life of its own"

protomyth4• 4 years ago

I worked on one in the 90's that was $50,000 a day for three years. The drug company, assuming the trial was successful, had about 4 years to make all its money back before the patent expired. Testing blood, urine, and stool for multiple things is expensive.

Intellectually, I can understand the need for control groups, but I still think it's immoral. When you stare at a spreadsheet and see 70% of the control group is dead because some random number generator sorted them there like hell's own sorting hat, and the 98% of the people getting the drug are alive, you have no business talking about statistics. That graph will haunt me til my dying day.

s1artibartfast8• 4 years ago

You didn't kill 70% of the control group, you saved 98% of the experimental group.

Also, with the appropriate trial design, you can stop the control and transfer patients once you see these big differences. Same thing should happen if you see the opposite, e.g. killing 70% of your experimental group.

protomyth4• 4 years ago

Everyone knew the results of "standard treatment", that's why they were researching the drug. There were years and years of statistics.

We killed 70% of the control group. Doomed by a random number generator.

disgruntledphd2• 4 years ago

Yeah if it's a potentially life saving treatment you really need that design that allows peeking at the results.

Khelavaster• 4 years ago

Peek-and-do-more is compatible with double-blindness, as long as the peeker-and-decider is different from the dispenser and patient.

Slaminerag• 4 years ago

It's possible to stop a trial because the new drug is much more effective than the old drug. It's also possible to stop a trial because the new drug's significantly worse than the old. We've done both. Also, I hope that no one ever tells you that you have to do a drug schema change because "too many kids are dying."

refurb3• 4 years ago

It's not unusual for trials to be stopped earlier for that exact reason. However, you can't just glance at a spreadsheet and say "it's working!".

Statistical analyses need to be pre-defined and powered to measure an effect.

I've been on clinical trials where the innovator drug showed a fantastic effect early on, then two overall survival curves crossed a year later.

manv1• 4 years ago

FYI, the RFP process is basically the same in other industries.

apohn• 4 years ago

I came here to post the same thing.

I used to be in enterprise software sales and consulting and have been involved for RFIs/RFPs across multiple industries. I don't know a single person who thought the process was sensible or efficient. Unless you have an person whose job is dedicated to responding to the questions, it's basically just a game of passing a hot coal around until you have something you can submit.

steve_gh• 4 years ago

I work in an engineering consultancy organisation.

A lot of the big tenders are incredibly wasteful, with lots of money being wasted on them.

We have an aggressive sales pipeline process. We prune our sales pipeline hard, and ensure we are spending money where we can win.

Result, we win about 70% of tenders that go to the end - we save money by recognising where we can't win and walking away early.

On a related note, we are great fans of the 'pre-proposal'. You should be able to get the gist of any proposal down in 2 sides. That should be enough to get buy in from stakeholders on all sides. Only when everyone agrees on the pre-proposal do you then write the full proposal - so you save effort (and make the client happy because they don't have huge amounts of boilerplate to read!)

SubiculumCode• 4 years ago

Nicely written piece highlighting the value of in-house expertise and capabilities, providing a lesson that translates very well imo to why government projects should have more in house expertise and not contract everything to the private sector.

getsloped• 4 years ago

The logistical inefficiencies mentioned in this article are mind boggling when compared to what we've come to expect in different industries. We all know healthcare is always a little behind the curve, but with clinical trials, we're talking decades.

For example: most clinical studies have no way to track samples going from sites to labs beyond a hope, prayer, and constellation of unconnected spreadsheets. The status quo is to write a 50 page lab manual and include a wall of text and bullets outlining a list of multiple stakeholders that need to be emailed a tracking number each time a sample is shipped. Not to mention there are days you really shouldn't ship samples. Labs have no idea when samples are going to show up. Seriously, samples just show up, they open the box expecting a piece of paper to identify the samples.

An incredible amount of perfectly good, usable samples get LOST or DESTROYED due to the fact that there's no way to track any of these samples or coordinate stakeholders. We're talking critical samples, like blood from kids with cancer. It's absolutely ridiculous.

Compare this sob story to the fact that I know in realtime where all the dumb shit I bought on Amazon is. The cognitive dissonance here is beyond comparison!

+ + +

About 8 years ago my brother and I were given a window into this problem. Being Amazon FBA sellers ourselves, we thought, "hey now that can't be too hard to fix" and thus began the wild journey of Slope [0]. What we learned working on this problem is that clinical research sites literally get crapped on left and right with ridiculous contracts & protocols written by people who don't understand the clinical setting or logistics in general. With each research site running 30-50 clinical studies and each of those studies having its own catalog of kits, shippers, and other inventory, it's no wonder why clinical research is so expensive! And this is just supply chain folks.

Slope started by making an inventory and sample management software to address the needs of sites and have worked backwards into selling this platform to Sponsors and CROs. While I have your attention, we're hiring a head of engineering now and will have a JD up for a product manager next week [1]. Thanks for reading!

[0] https://www.slope.io/ [1] https://www.slope.io/jobs/head-of-engineering

trap_goes_hot3• 4 years ago

People complain about industries with bureaucracy but nobody wants to take the hit and actually work in those industries. They want magically for their pay/benefits/culture incentives to align.

ativzzz2• 4 years ago

Exactly - like the OP you replied to - Slope are trying to hire a Head of engineering for a salary that competent mid/senior level engineers should be easily getting. It's no tech industry, and the work is less fun

napier• 4 years ago

Best explanation I've read online. Most attempts at describing why boil down to the circular "clinical trials are very expensive because the process of getting a new drug approved is lengthy, complex and expensive".

AlbertCory• 4 years ago

I have to admit I found this tl;dr. That's my bad, because it's an important article.

All the bureaucratese in the RFPs and heavy-duty lawyering just add, unnecessarily and hugely, to the cost. At bottom, though, you have:

1) A lot of human beings who have to be recruited and paid to be subjects

2) Medical institutions that have to administer the drugs or run the devices

3) A sufficient amount of elapsed time for effects and side effects to manifest

4) A reporting system that's trustworthy

So at best you could cut (puts finger to the wind) half the cost? Surely not 90%.

CharlieDigital• 4 years ago

I worked as Director of Engineering at a company for 10 years that built a SaaS for clinical trial document management.

There is a weird perversion of incentives in the industry driven under the guise of "quality" and "safety" that from a software engineer's perspective, looked pretty bad to me because of how manual and error prone the process is.

The problem is that the people who have the knowledge aren't incentived to make it more efficient or cost effective; the purpose of many of these folks is to simply extract rent from the system and blame regulations and "patient safety" concerns.

At an MES startup, one of the industry stalwarts (starts with V) was more or less using mafia tactics to force the startup to use their software or they would voice no confidence with pharma companies where big V was already entrenched.

One of my passion projects would be to build an open-core clinical trial document management system with a focus on automation and templatization of common processes and workflows. It's insane how much waste this system has.

robocat2• 4 years ago

Call me cynical, but this could apply in most any industry:

  Suppose that one tries to push back on inefficiency in generalーsomeone naive but well-intentioned, like myself, who is tired of the Long March of interminable meetings. In fact, I expressed my concerns about the proliferation of unproductive meetings in the clinical department to the C-suite executives (with whom I remain on good terms!) and, together, we carefully planned a series of reforms to company culture designed to reduce meeting burden. We hoped to implement a soft cap on the number of meeting participants and the length of a meeting, to normalize efficient behavior such as leaving meetings midway if you no longer need to participate in the remainder, starting meetings on time without 15 minutes of small talk, and a mandated review of all recurring meetings with the intention of pruning overall meeting load by >50%.

  Ironically, on the morning that we planned on roll out these reforms, we received urgent and vehement protests from a member of middle management who was notorious for obsessively overscheduling meetings with huge participant lists just for the sake of “inclusion” and “engagement.” Because this particular manager was currently playing a crucial role in managing the rollout of several clinical trials at the same time, executive management judged that the risk of alienating him was too high, and to this day I believe these reforms remain unimplemented (although I hear he has since left the company, so perhaps my slides will eventually see the light of day).

wins32767• 4 years ago

The author doesn't really understand the details of the operations of clinical trials, though this is a good overview of the kinds of outcomes that show up. I worked on operations software for clinical trials for about a decade, and there are (mostly necessary) regulatory drivers a large fraction of the complexity he complains about. The first order effects of the regulations are pretty straightforward, but when you get into second, third, and fourth order impacts, you start seeing some of the outcomes he's describing.

Melissa_bime• 4 years ago

I totally understand the frustrations experienced by the author of this post. I am a nurse and my exposure to the Clinical Trial Industry came from a more bottoms up entry point. Its mind boggling to the the total costs CRO's charge for the services compared to the eventual experience for the staff and patients taking part in these studies. CRO's are in dire need of disruption. I also agree about the incestuous industry hence the very slow progress being observed. It's hard to show any progress when it's the same people moving around the same companies and implementing ideas that are somewhat outdated. Also, from a cost perspective, i think the future is also coming to a place where Clinical trials get outsourced outside the US. Sites and PI's outside the US should be empowered to run this clinical trials as they provide a more cost effective solution with regards to operational costs and also access to an ever growing number of patients, reducing clinical trials timelines that lead to increased costs. I understand the need for CRO's to maintain quality, and they can do this by building out the right relationships and partnerships with overseas contractors, but i believe in the long run, this is the solution. I believe this so much that i launched my own CRO company. Infiuss Health. The goal of this company is to unlock the potential for studies to the run by CROs and smaller life science companies outside the US. We are starting with Africa. As soon as we can unlock this population that has traditionally just been taboo to life science companies, imagine the lengths we could go with with access other geographies.

naijaboiler5• 4 years ago

This is a very naive. As a techie that became a physician and I'm now back in the tech world, it took me a long time to understand a fundamental difference in approach. The medical world moves very very slowly and it seems very very inefficent to a tech person. It is a necessary evil. In medical world, lives are at stake. The ethos is first, do no harm. It is inherently and arguably necessarily slow to change.

The ethos in tech is just fundamentally different. Move quickly, break things. Its an Agile mindset of quick changes, pivots, rapid advances. Inefficiencies get identified and are tackled.

Medical world is different. The inefficiences are a feature not a bug. They are there to exactly do what they are doing, slow the rate of change. Clinical trials is one place where both worlds collide, the fast pace technical innovation world is about to interface to the slowly changing medical world. All of these guys frustrations is exactly about that.

He might think the solutiom is to speed up clinical trials phases. It isn't. Even if you did, sooner or later you will still come to another traffic stop of slow changes and inefficiencies because the medical world moves at a different pace, and necessarily so. When lives are directly involved the burden of proof to enact changes have to necessarily be very very high.

And what techies do not understand and probably keep asking is "doee it have to be so?". Yes it does. We didnt arrive here by accident. Even if we started healthcsre industry anew, it will inevitably end up in this slow-changing equilibrium. When lives are directly involved, better to leave things as is, than change and make worse. The burden of proof for even the slightest of changes is extremely high. what's optimal for health is indeed a slowly changing world. It is not for lack of talent or intelligence, as this author naively suggests. If you want to solve problems with quickly, don't go near that world. Stick to the world of optimizing ads

naijaboiler5• 4 years ago

To add to this comment. An analogy would be trying to drive highway speeds on a crowded downtown roads with heavy foot traffic, street vendors, open stores. Noticing inefficiencies in how cars are passing through is missing the point. The point of highways is to move people quickly. The point of crowded downtowns streets is to move people slowly so they can shop, interact and spend money.

If this efficiency-optimizing author were to take his SV mindset to tackle thise inefficiencies at industry scake, what this author will quickly find out is that it is an impossible task. The industry itself is necessarily slow and its not by accident. It can be fast when it absolutely needs to be e.g Covid vaccines. That alone should let you know those inefficiencies are only inefficiencies to you because you are naive to what the industry is actually optimizing for.

At best, you sound like the impatient driver honking on a crowded street, aggressively driving into every available opening thinking you're optimizing for efficiency. You arent.

i_am_toaster2• 4 years ago

This isn’t the government, it doesn’t need to be painfully slow to ensure it continues working properly. I’m having a hard time responding in a constructive manner because it’s a bit infuriating to hear someone make excuses for such a corrupt industry. Have you ever worked with one of these companies? I just don’t see how it’s possible for someone to see it firsthand and then come to the conclusion that the inefficiencies and high rate of error in data collection/management are not just preferential, but even required outcomes.

I will note that you addressed none of the points made in the article. My assumption is that either you feel the need to attack the authors conclusion, having no defense of your own, or you’ve foregone reading the text altogether.

naijaboiler5• 4 years ago

I read the text. The author sees the trees, misses the forest. At the micro level, the inefficiences are glaringly obvious and pervasive. But they exist and continue to exist for a reason. It isn't because people are stupid or corrupt, it is because that's what the industry optinizes and incentvizes for. That doesn't mean things can't get better or there aren't things to improve on, it's just that this author doesn't seem to understand the big picture yet which makes me very dubious of such a person's ability to execute any meaningful changes

mleonhard• 4 years ago

Can you share specific things that need to happen slowly and why that slowness helps protect study participants?

Some things mentioned in the article clearly increase risk to participants: manually re-assembling translated documents, using custom software and paper forms, and diluting important documentation with boilerplate. Other things seem to purely increase cost: starting meetings with smalltalk, overcharging for server hosting, and cronyism.

Dove• 4 years ago

Why are things this way? cowardice

So many things in this story, and in corporate life in general, arise from fear and selfishness and shortsightedness, which take many forms. The process of bidding, and proposals, and counterbidding described here, the meetings and trade studies and boilerplate text, are CYA efforts. They aren't there to help you make the right decision. They're there to prove you did everything anybody reasonably could be expected to do, to make the right decision. Every coward who passes through the process raises that CYA line of reasonability a little higher.

The inefficiencies that no one fixes are mutually supporting; in a project this size, a little inefficiency doesn't matter, never mind that it's the whole pile of them that makes the project this size. Fixing something is risky and difficult and you may break something and you don't get anything for it. Going along works. Is there an order of magnitude worth of bloat that can be trimmed here? Shoot, maybe two. But is there any gain in doing that? Would a bid an order of magnitude (or two!) too small even be taken seriously? If anything, such a bid would be a negative thing -- if it didn't work out, for whatever reason, it would be easy to accuse the guy who went with the cheapest bidder.

To say "I personally reviewed this process, and I am willing to say, on my own judgement, that it's good" takes an almost unbelievable level of courage, in a world so ready and willing to judge you for any mistake. And yet, people doing exactly that is the only reasonable solution. The only thing that will make good sense prevail is people standing up for good sense. It is the only thing that can possibly prevent... this.

It isn't just medicine. It's every industry. It's every community. It's politics writ large and HOAs writ small. I guarantee, where you work, in your own back yard, you can find examples of this problem. The substitution of the system for the hard work of understanding and the courage to say what you think and see. CYA over common sense.

What should be done?

The first thing to realize is that the fear that motivates this behavior is entirely justified. People who start revolutions get shot at. It's a fact as old as history. Look, in the world of nutrition, at people like Atkins and Taubes, who were ridiculed and ostracized for years before their ideas became mainstream. Gary Fettke got into legal trouble. That can absolutely happen to you, for nothing more than advocating an unconventional view on what you believe are solid, perhaps even scientific, grounds. (Pardon the example if you don't agree with these folks -- the fact remains that for each of them, the personal cost of telling the truth as they saw it, and trying to fix something they thought was broken, was, at least temporarily, high.)

But the second thing to realize is that the fear that motivates this behavior is largely unjustified. If you try to make things better, people usually appreciate it and try to help. Sometimes the resistance you're so worried about doesn't exist. And when it does exist, it's often weaker than you might expect.

Two contradicting realizations? Consider the case of Martin Luther King Jr., whose case was basic, and obviously moral, and inspiring, and who nonetheless had every reasonable thing to fear--whose enemies were absolutely real, and dangerous, and persecuted and ultimately killed him (point 1!), but who was also taking on the absolutely massive and invincible dragon of the generations-entrenched psychological direction of an entire massive country, and who substantially won that fight! (point 2)

Do you want to change medicine?

Look at Dr. Garrison Bliss, a primary care physician who hated the way primary care was done, figured out how to do it another way, and started a movement in which I-don't-know-how-many-thousands-maybe-millions of people get to do it that better way.

Look at Dr. Jason Fung, a doctor who chased a "what if" curiosity about type 2 diabetes right into a revolution that is right now on the edge of saving millions upon millions of people.

These guys could easily have gotten run over by the massive industries they were taking on, but for one reason or another, they just . . . did what they thought was right, and the worst just . . . never happened.

Stepping away from medicine for a moment, do you know what else used to be this way? Taxis. I remember back when there were never enough taxis, and you had to wait for them for a long time, and they were so expensive and unpleasant that it was a running joke in pop culture. For the same basket of reasons -- corruption and cowardice and short-sightedness and selfishness and not my problem. And you know, when Uber started to change it, they were definitely maybe doing something illegal in a lot of places. Were they going to get legally slammed for it? I sometimes wondered. But the laws obviously made the world a worse place, and the alternative was so nice by comparison, and here we are -- in a world which still has its problems, but obscenely, irrationally, how-did-it-ever-get-this-bad taxi service isn't one of them.

It's not your job to fix the world. (And it turns out that even if you did become dictator-for-life, fixing the world would remain surprisingly hard and require surprisingly many of the same character traits -- so there is no point in waiting for official permission.) It's not your job to fix medicine. It's not your job to fix education. It is your job to fix what's right in front of you. Want to be the SpaceEX of CROs? Sure sounds like somebody can. Maybe it's you. Too big? Fix your own clinical trial. Too big? Fix some of the inefficiencies. I'm glad you tried to fix the meeting thing. Keep trying. Fix one meeting.

PG's advice about startups -- "just don't quit" -- is surprisingly apropos to any revolution. It seems black and impossible right up until it doesn't anymore. If you find something important enough to be worth that to you, and if you just keep swimming, if you don't give up and constantly pivot to trying something new when you find the way blocked, constantly figure out what to do about problems, it's astonishing what one person can accomplish.

You may die in the process. In the fifteenth and sixteenth century, the Catholic Church was in this place -- corrupt and ineffective and unbelievably powerful and invincible and had been ruling the world for literally hundreds of years. Enough people laughed at it and hated it for it to be a running cultural joke: the fat monks and corrupt bishops of Robin Hood are exactly this joke, and the story is about how an outlaw can obviously be the better person. The story is about exactly this problem, and it was popular in the 15th century because that was what was on everyone's mind! And yet even such story did not propose a solution. It settled for talking about the problem. There were no solutions. Alternatives were unthinkable. Every now and again, someone would try what I recommend, they'd try to fix their little piece and make things better, and they'd be crushed by the great power and die and fail. Nobody knew Tyndale would be different -- and all he was trying to do was get better knowledge to more people. Nobody knew Luther would be different -- all he was trying to do was start a debate and get some theological reform. Their ambitions were small, and people before them had died both for less and for the exact same things, and the fact that they tried anyway is why they were able to succeed, and why we have the world we have today. (Pardon the example if you don't see this particular history this way -- you can find other examples where many failures preceded success, where fears are justified, and yet where taking that chance results in slaying absurdly powerful dragons.)

I myself am convinced that someone like that, doing something like that, is the only reason we ever have any nice things at all. The CYA process sure isn't producing them, I can tell you that much for sure! And so to answer the question of "Why things are this way", well -- in that particular corner of the world, nobody ever did that. What's to be done? Do it. :)

johnys• 4 years ago

I had the same experience as the author when I first started learning about clinical trials from the perspective of the software world.

The amount of money invested in clinical trials and lack of visibility is staggering. We've companies that spend the entirety of their $100Ms in funding and not realized until month(s) after the fact when their accruals caught up!

A lot of it is messed up and can be improved. Indeed, we've have more demand than we can service since we productized what we're working on at Condor (https://condorsoftware.notion.site/About-Condor-d8a466f7722e...).

However, similar to what others have alluded, Chesterton's Fence is a relevant concept here (https://fs.blog/chestertons-fence/). It's critical to understand the rational reasons why the industry is so apparently messed up before you can fix them affectively. I wouldn't be working on the problem we're tackling today, if it weren't for my cofounder's deep knowledge across several disciplines in the industry.

One thing that continues to make a meaningful impression on me is the mission driven motivation behind so many people working in the industry of all levels. Consistently from the chief scientist to the entry level accountant you hear people talk about how they tie their work to the patients. Many software companies say they are "changing lives," many life science workers can actually point to saving them. It's been a big motivating factor for my cofounder and I to realize the same companies we are helping extend R&D runway are the ones making cancer drugs for some of those close to us.

Messy industry dynamics and backwards software means there are lots of BHAGs (https://www.jimcollins.com/concepts/bhag.html) to tackle.

Always down to talk shop on clinical trial finance, software, or ways to collaborate with good people whose interest this piques! Contact in bio.

collyw• 4 years ago

Gotta pay off all the right people.

Get that covid one on to the childhood vaccination schedule? Emergency use authorization, unknown long term effects, more harm than good for this age bracket, no problem.

Why are clinical trials so expensive? Tales from the beast’s belly

Related Stories