Book Review: From Oversight To Overkill

https://astralcodexten.substack.com/p/book-review-from-oversight-to-overkill

by apsec112 • 3 years ago

149 46 3 years ago

astralcodexten.substack.com

scotty79• 3 years ago

I love how the word oversight has two opposite meanings.

eru6• 3 years ago

You'll like https://en.wikipedia.org/wiki/Auto-antonym then.

epgui• 3 years ago

Side note: the French example “apprendre” listed in the article seems like it might not really belong in this category. On it’s own it always means “to learn”, and it is only when you use it in specific grammatical constructions, for example following indicative personal pronouns, that it takes the opposite meaning: “lui apprendre” or “leur apprendre” means “to teach him/her/them”. The meaning is not merely inferred from context, it is determined by grammar.

civilized• 3 years ago

... ethicist Jerry Menikoff [...] had decided maybe it was unethical to do RCTs on ventilator settings at all. He asked whether they might be able to give every patient the right setting while still doing the study. The study team tried to explain to him that they didn’t know which was the right setting, that was why they had to do the study. He wouldn’t budge.
Finally, after a year, another panel of experts ruled in favor of the investigators and gave them permission to restart the study right away. They did, but the delay was responsible for thousands of deaths, and produced a chain effect on ventilator research that made us less prepared for the surge in ventilator demand around COVID.

People who do not understand or accept the role of risk-taking in the expansion of scientific knowledge and the betterment of the human condition should not be in positions of authority over the scientific process.

I have to ask: what went wrong here? Why would such a person be in this position? Surely a judgment like this is exceptionally bad...? Or has the rot spread much farther, to the point where incompetence this severe is normal?

AbrahamParangi• 3 years ago

That science requires risk taking means that the more institutionalized it becomes, the less scientific it becomes as institutions are naturally allergic to risk.

biomcgary2• 3 years ago

Institutions are allergic to short-term, acute risks that result from their actions. They are insensitive to long-term, chronic risks that results from their inaction.

Meta-risk, the risk that comes from not taking any risk, needs to be drilled into everyone's head: the public, journalists, politicians, IRBs, etc. I think the concept that not taking risks is risky should have its own label, but if someone doesn't like "meta-risk", maybe they will accept "risk tradeoffs".

BurningFrog2• 3 years ago

I've been thinking for a while that it's time to start fresh building institutions that can take over the tasks our current universities have largely drifted away from.

e2021• 3 years ago

Why would such a person be in this position?

Such a person will always be in this position, it is the type of personality that thrives in a bureaucratic institution. No amount of fiddling with the process will fix it, you need an alternative form of decision making

armchairhacker• 3 years ago

The IRB demanded that he give his patients consent forms warning that they could get smallpox. Dr. Knight tried to explain that smallpox had been extinct in the wild since the 1970s, the only remaining samples in US and Russian biosecurity labs...
...the IRB demanded patients sign consent forms in pen (not pencil) but the psychiatric ward would only allow patients to have pencils (not pen)...

hirundo• 3 years ago

In '84 I had a college class in experimental psychology, and the culmination of that was to perform our own experiment, which had to clear the IRB. Mine involved measuring the student-subject's galvanic skin response and showing them photos of art. To calibrate that device the protocol was for the experimenter to slap their hands together a few feet away from the subject. That was too much for the IRB even 40 years ago. They did not allow the slap. Too traumatic I guess. It didn't really make a difference because it was a toy study for a class. But that was a great lesson in IRBs.

HDMI_Cable2• 3 years ago

Ezra Klein calls this “vetocracy”, rule by safety-focused bureaucrats whose mandate is to stop anything that might cause harm, with no consideration of the harm of stopping too many things. It’s worst in medicine, but everywhere else is catching up.

This really scares me. It seems like every important facet of American life that can be touched by increased, predatory, and superfluous litigation and regulation has. If we take Scott's word for it, this 'vetocracy' costs 10 000 lives a year due to IRB over-diligence, makes Bay-area housing completely unaffordable, reduces the quality of education, and so much more.

It really seems like our first priority as a nation should be to dismantle this 'vetocracy' and let researchers perform studies, and teachers teach.

eru6• 3 years ago

It really seems like our first priority as a nation should be to dismantle this 'vetocracy' and let researchers perform studies, and teachers teach.

The housing is also really, really important: the Silicon Valley, San Francisco and New York etc are some of the most productive parts of your country. If more people were allowed to live there [0], your economy could grow a lot. A bigger economy directly translates to lives saved.

To give a simple example: your highway engineering people work with a value of life of something like x million USD. Any improvement to safety that costs less than x million USD per life saved gets implemented. The value of x gets set in vague relation to how rich society is.

[0] No any one individual is directly barred from moving to SF, if they pony up enough money for rent; but the NIMBY climate effectively sets a limit on the total number of people allowed to dwell there.

GauntletWizard• 3 years ago

Location has almost nothing to do with the productivity of workers in Silicon Valley.

stonogo2• 3 years ago

your highway engineering people work with a value of life of something like x million USD.

This is not currently nor should it ever be how civil engineering works.

dmix2• 3 years ago

There's probably nothing in modern history that hurts upward mobility and quality of life more than the disaster that is housing in western countries. Like most things it's a product of good intentions and previous winners using gov to kick the rug out for everyone else. The Bay Area is the classic example but this exists in urban (and increasingly suburban) areas in almost every wealthy western country: UK, Canada, Australia, etc.

The only major-changes that actually help are stuff like work arounds ala working from home or using AirBnB to offset mortgage costs.

I've read that the meme of "middle class is dead" is mostly bullshit, incomes have stayed consistent, the only major difference is cost of living, which housing ranks #1 by far.

KerrAvon2• 3 years ago

It’s not bullshit, it’s just more nuanced than that, as things often are.

https://www.pewresearch.org/fact-tank/2022/04/20/how-the-ame...

Karrot_Kream• 3 years ago

And like the IRB there's historical precedent for the laws the previous winners use to freeze development. Before the creation of laws like NEPA and CEQA in the US, civil engineering projects would frequently lead to environmental issues. Before local control over housing became the norm, minority neighborhoods and shops were razed to accommodate highways. Now those laws are being weaponized to freeze development.

dreamcompiler• 3 years ago

Don't forget college. The [very good] public university I got my degree from in the 80s charged me about $180 per semester for tuition. The price now is $7500. That doesn't include books, housing, food, etc.

amadeuspagel• 3 years ago

The US, the UK, Canada and Australia are all common law countries. Wealthy western countries that have administrative law, like Japan, Germany and France do not have this problem to the same extent.

KerrAvon2• 3 years ago

watch out for Chesterton’s Fence, though

bawolff• 3 years ago

Interesting. The thing that seems missing here is deeper comparison to other countries (there is only a brief mention of britian). Its presented as an american problem, so how does the american system compare to other systems? Do they have the same problems? Did everyone copy america?

niccl• 3 years ago

I think there are two things at work. First, if you pay someone to find problems with a study, they're going to want to find problems with the study to justify their existence.

Associated with that is that many people don't consider the level of risk when they are considering whether something is or is not a problem.

And both things don't just apply to IRBs. A friend told me about when his workplace had a security assessment (for Reasons). the assessor's only finding: that a houseplant by the door had sharp leaves and could be used by a malicious intruder as a weapon. Yes, I suppose so. Job done by security assesso. But, just perhaps, a malicious intruder might not choose to rely on being able to slash someone with the houseplant, in which case worse things might possibly happen.

TedDoesntTalk2• 3 years ago

if you pay someone to find problems with a study, they're going to want to find problems with the study to justify their existence.

This is the same reason a home inspection report during a real estate sale ALWAYS finds problems with the property for sale

groby_b• 3 years ago

If Scott's wondering whom specifically to blame: Accretion of safeguards. It's inevitable, it seems. Something goes wrong, the process is amended to avoid that failure case. Something else goes wrong, add another amendment.

All individual additions are likely net positive - positive in 99% of the cases, negative in the rest. Assuming events are mostly independent, you'll sooner or later are in a world were net positive outcomes are becoming less likely than net negative ones.

You can sometimes sidestep that by combining/rewriting rules to some extent, but in the limit, you can only avoid all additional risk if you prevent any changes.

That's not a fault of IRBs. That's a fault of trying to minimize risks. I'm not sure there is a good answer here outside of "here's the maximum amount of risk we're willing to take, and here are the few clear ethical boundaries we'll never overstep".

There's a reason any place that's dealing with a lot of constraints is concerned with risk budgets, not risk elimination. And I'm not sure there's a lot of thinking on probabilistic ethics. (Or maybe I just can't find it)

TedDoesntTalk2• 3 years ago

Balancing safety against freedom and responsibility. Let’s hope the pendulum swings like free-range parenting (freedom, responsibility) is having a moment over helicopter parenting (safety):

https://www.freerangekids.com/

https://en.m.wikipedia.org/wiki/Free-range_parenting

archon1410• 3 years ago

Lawyers sue institutions every time they harm someone (but not when they fail to benefit someone)
pull the culture towards celebrating harm-avoidance as the greatest good, and cast suspicion on anyone who tries to add benefit-getting to the calculation

Minor nitpick, but it is more accurate to say that they prefer prevention of new harms over the prevention of old harms.

The current wording (plus the commentary over Gnosticism) kinda makes it sound like negative utilitarianism, or harm aversion (preferring the reduction of suffering over the addition of "positive pleasures") which would be a legitimate position. But the post is something more like "new risks aversion", i.e. preferring current harms over possible new harms regardless of the lower intensity of the "possible new harms", which sounds pretty irrational.

hoseja• 3 years ago

This is a big trolley problem, isn't it? Do studies that risk 2 deaths per decade, or do no studies and let tens of thousands die.

That Hans Jonas quote is insane. I hate insane americans, why do they have such outsize influence on culture there (and through cultural imperialism everywhere else)?

dsign• 3 years ago

Let’s engage in some speculative thinking:

Somebody, let’s call them Dr. X, invents a computer algorithm that cures cancer. It works by running simulations of the patient’s cells. Its output is a list of diverse interventions, for example, pairs of RNA fragments and custom delivery mechanisms for each fragment. For a given patient, the algorithm outputs between 100 and 10000 interventions, all of them different and tailored to the patient. The simulation also takes into account possible adverse effects, it’s really a cure.

With the current regulatory framework, does Dr. X has any hope of his invention being used at hospitals in less than a decade, or all he can do is to use it on himself and/or make the algorithm available in the dark web?

walkhour• 3 years ago

The fact that bureaucrats err on side of minimizing deaths caused by experiments is just a consequence of deaths being caused by experiments being unequivocally trivial to prove.

Meanwhile finding out the number of deaths caused by extra caution and delaying the experiment requires a calculation, and then a very lengthy blog post to be communicated.

Bureaucrats don't care about the second kind of deaths because they can hardly be linked to their actions. Now the first kind of deaths are directly their responsibility. And guess what, they can be avoided with a flick of a pen.

x-complexity• 3 years ago

(Internal anger: 150/100)

Add another tally mark to "bureaucracies can only kill, never create".

Try saying "accept what we cannot avoid" to my skewed feet, OHRP. I WISH I had the finances to fix them, but because of people like you, any advancements on cheap treatments for this issue will be met with hundreds of miles of red tape just to form a simple test.

This quote sums up how I feel about the OHRP, and all bureaucracies similar to it:

A survey of how researchers feel about IRBs did include one person who said “I hope all those at OHRP [the bureaucracy in charge of IRBs] and the ethicists die of diseases that we could have made significant progress on if we had [the research materials IRBs are banning us from using]”.

ZeroGravitas• 3 years ago

It seems a common human failing to not be able to cope with rules that affect them.

One you regularly see here is the complaint about Wikipedia: "I tried to edit the page about me/my work/my family and they wouldn't let me. But I'm the expert on me/my work/my family and it's currently wrong".

Which sounds outrageous!. But at the same time "people can freely edit their own pages" has obvious failure modes too.

Similarly, getting mental patients to sign in pen can be presented as pointless nonsense. Or it can be seen as part of a strategy to stop defenceless patients being taken advantage of by malicious doctors and stop vulnerable doctors being sued by malicious patients.

ogennadi• 3 years ago

"Doctors are told to weigh the benefits vs. costs of every treatment. So what are the benefits and costs of IRBs?

"Whitney can find five people who unexpectedly died from research in the past twenty-five years. ...

"What are the costs? ...the monetary costs are around the order of $1.6 billion.

"What about non-monetary costs? ... Low confidence estimate, but somewhere between 10,000 and 100,000 Americans probably die each year from IRB-related research delays.

"So the cost-benefit calculation looks like - save a tiny handful of people per year, while killing 10,000 to 100,000 more, for a price tag of $1.6 billion. If this were a medication, I would not prescribe it."

Book Review: From Oversight To Overkill

Related Stories