All models are wrong, but some are completely wrong

https://rssdss.design.blog/2020/03/31/all-models-are-wrong-but-some-are-completely-wrong/

by magoghm • 6 years ago

305 207 6 years ago

rssdss.design.blog

smitty1e2• 6 years ago

One feels that, irrespective of the models, the data in the covid-19 case may be unusually bad.

It may be time to add a third error category[1]

I. False positive

II. False negative

III. Deliberately skewed off the map for propaganda reasons.

[1] https://en.m.wikipedia.org/wiki/Type_I_and_type_II_errors#Ty...

YZF2• 6 years ago

This is driving me crazy ;) the poor quality of data.

You'd want health organizations around the world to be publishing every possible detail (anonymized) so that the disease can be better understood. Yet three months in, with over a million cases worldwide, we still have experts disagreeing about things like asymptomatic transmission, use of masks, droplets vs. aerosol, how much distance one should stand from another, viability on surfaces, etc. etc. Even for treatment options rather than insisting on randomized double blind trials start by using the natural experiments that are already happening.

We should have the data to answer a lot of these questions (or at least draw out some probability distributions), or at least someone has it. This stuff is going to be critical in informing exit strategies.

magoghm2• 6 years ago

My experience with data is: you never get good data. Always, the closer you look at it the more problems you find in how it was collected.

thaumasiotes2• 6 years ago

You'd want health organizations around the world to be publishing every possible detail (anonymized)

If you're publishing every possible detail, then your data isn't anonymized. Anonymization consists of the removal of almost all of the details.

martingoodson7• 6 years ago

Author here: happy to take comments or criticism

dwohnitmok2• 6 years ago

This is a hard problem, mainly because of the problem of motivated reasoning as mentioned by another comment.

You're requiring cooperation from multiple different parties here (scientists, journalists, policy makers, readers, etc.) and any of these parties can warp the results in any number of ways regardless of the cooperation of other parties.

Climate science still hasn't solved this problem despite trying to implement what you're talking about.

It's an age-old problem, if a priori you're looking for something hard enough you're bound to find it.

martingoodson7• 6 years ago

I think, at the least, if journalists get some quotes from other scientists before publishing a piece on a new model it would be a major win.

YZF2• 6 years ago

I'm finding myself in disagreement with rule #6. Using a model effectively is about a lot more than just the domain knowledge. I'd value analysis from a mathematician/statistician more highly than from an infectious disease physician. There's the stuff that informs models, i.e. the observations, the experimentation etc. and then there's the science of modelling itself which isn't really in the same domain.

martingoodson7• 6 years ago

I agree with this - but I do think it should be clear that the model is from outside the mainstream. Not to dismiss it but to clarify its status. Check out the New Yorker piece I link to in the article - it's quite shocking the misinformation that's out there.

oivey• 6 years ago

Don’t discount domain experts. Models are meant to predict the real world. In order for them to be accurate, the model itself needs to capture how the real world works, and the math underlying the model has to be correct. Domain experts are the most likely to have experience in both areas. Drawing an example, a physicist models the universe and knows the math and model behind electromagnetism. A mathematician probably knows the math but maybe not the model.

noobermin3• 6 years ago

Domain knowledge is very, very important in modeling because it helps you figure out what to put in the models. Math is important but certain odds and ends that are of great interest to mathematicians are not helpful for the usual simulation, and the simulations that are most helpful in informing policy.

guscost5• 6 years ago

If any of the scenarios from the famous Imperial College model turn out to have been based on just-as-bad assumptions, would you be willing to write a follow up about that?

martingoodson7• 6 years ago

I'm not taking a position on the Imperial College model. I'm explicitly advocating that all models should have their assumptions examined. And that policy makers should use a range of model and not depend on just one.

garmaine2• 6 years ago

The final point seems like a bunk attack. A lot of epidemiologists were upset at Silicon Valley data scientists and hedge fund quants who were putting together prediction models that disputed what the government was saying. Well, so far these models have been more accurate than the official ones.

As we should expect, because data scientists and quants are EXACTLY the people with the set of skills necessary to make such predictive models. At best, it's an ancillary skill for epidemiologist, and we've seen many cases in this pandemic where they have wielded these tools incorrectly.

etangent• 6 years ago

Your entire post seems to be about #6. And therefore it's entirely wrong.

Given that "people with background in infectious diseases" have largely failed, as a group, to warn us in January-February about this pandemic (see this Twitter thread for a slew of concrete examples https://twitter.com/RokoMijicUK/status/1246509433145917443), my conclusion is that unless someone has a background in hard quantitative field (regardless of what that field is), that person should not be let anywhere near quantitative models.

svat• 6 years ago

This is a great start on how scientists (and journalists) should communicate to the public! Thanks for writing this, and the world would be a better place if everyone remembered to follow these principles.

I think in fact it would be better to go even further: not speak from a position of superiority (even if one knows more), but try to acknowledge the audience and persuade effectively. Here's a recent article on the topic: https://undark.org/2020/03/19/coronavirus-myths/ and here's one of my favourites (in a different field of science) from three years ago: https://deansforimpact.org/why-mythbusting-fails-a-guide-to-...

tomcatfish• 6 years ago

I very much enjoyed this article you linked with the Times reporter and the law professor

https://www.newyorker.com/news/q-and-a/the-contrarian-corona...

empath75• 6 years ago

I think starting with the global warming example was a poor choice because there are a lot of vested interests who would find something to attack and distort even in a flawless paper.

scribu• 6 years ago

Reading the examples, where people jumped on a single mistake to discredit an entire report points to some sad conclusions:

1. Scientific literacy is super low in the general population.

2. Motivated reasoning is rampant. People will believe anything that enables them to do what they wanted to do anyway.

xg15• 6 years ago

2. Motivated reasoning is rampant.

This is an important point. Scientific scrutiny is extremely important, but there is still a difference between a judge that is stern but fair - and one that actively wants you to fail.

Motivated reasoners have no problems holding opposing parties to impossibly high standards while accepting claims without any evidence as valid arguments for their side.

Today, climate scientists have learned the lessons and improved communication and modeling considerably, even to the point where we now how "attribution science" we we can discuss climate change in the context of particular weather events. We also start seeing changes in weather patterns that are hard to ignore even for laymen.

Nevertheless we are still having the same discussions as before.

lopmotr• 6 years ago

We also start seeing changes in weather patterns that are hard to ignore even for laymen.

Hard to ignore by motivated reasoners! Changes in weather patterns observed by individuals using their own experience are not evidence of global climate change. If there's science showing that, then yes, but personal experience doesn't add any value to the conclusion, it just reinforces whatever the person already wants to believe.

chrisco255• 6 years ago

I think the author oversimplifies the problems with climate models. They've had numerous mistakes and extremely critical ones. Not least of which is that they haven't factored for Multi-decadal changes in cloud cover albedo. The climate system is incredibly complex and our models do not have good models for all the subsystems that make it up, including the oceanic oscillations like Atlantic Multi-decadal Oscillation, PDO, etc. We can't even predict many of these subsystems with any degree of accuracy so we are hopelessly inaccurate at the higher level.

djsumdog• 6 years ago

I agree, and there are the same politician leanings in climate change non-profits and think tanks as with anything else. Sometimes they may justify sightly more alarming views of data to gain more funding; justifying it with, "Well, it's going to be bad anyway"

rathel• 6 years ago

We can't even predict many of these subsystems with any degree of accuracy so we are hopelessly inaccurate at the higher level.

Or do we? This sounds like a Fermi problem

https://en.wikipedia.org/wiki/Fermi_problem#Explanation

I may be wrong, though.

cassiet2• 6 years ago

The headline really cuts off the points nose.

martingoodson7• 6 years ago

Yeah - I was trying to play off the George Box quote but maybe it's too obscure https://en.wikipedia.org/wiki/All_models_are_wrong

magoghm2• 6 years ago

The reference to the George Box quote in the title is what originally got my attention.

KerryJones• 6 years ago

Interesting, I did _not_ know the reference (I'm a SWE, not in scientific community), and I thought it was a clickbait headline. Obviously this differs from the person above me ^ who clicked the link _because_ of the reference.

Just perspective, and I'm very happy to know the reference now.

cassiet2• 6 years ago

It’s less the obscurity, and more that to a layman, you are saying in the title that modelling doesn't work, or it really doesn’t work. Understanding your audience and all that. Someone should write an article about it so we aren’t misrepresenting our point by trying to be clever and assume the layman understands that which has not been explained to them...

grawprog• 6 years ago

What I've noticed about models, or at least when people are talking about them or trying to prove a point about them, is that people forget models are a simplified version of a specific part of reality, much the same as models of airplanes or something.

No matter how many variables you include, you can never capture the utterly massive and unpredictable amount of variables that exist in reality. But they're not supposed to. They're tools that are supposed to be used to help get an idea about how something might happen, that's marginally better than just guessing, due again to models always being incomplete, despite any amount of best efforts.

Like any tools, using models incorrectly, fucking around with them until you just get what you want to see or using the wrong one for the wrong task can be potentially dangerous, especially when those models are used to guide large decisions with serious consequences and impacts.

Reasoning and common sense need to be used alongside models and if the models don't seem to reflect reality, the problem is with the model and not reality and the model needs to be adjusted or thrown out otherwise continuing to use it will just lead to lousier and lousier decisions.

hinkley5• 6 years ago

It’s funny, but when you point out that a bad analogy is actually pretty accurate if you actually know anything about the other concept, people don’t want to talk about it any more.

We all know that hill climbing algorithms are often naive and sometimes hilariously wrong. Nobody will disagree with you about this, until you start talking about prioritizing work, and then everyone vigorously defends their favorite hill climbing algorithm, from task management to performance tuning.

My preferred strategy for optimization more closely resembles how a fruit grower would pick fruit, and the term, “low-hanging fruit” galls me because you would go bankrupt using that strategy. And probably lose most of your trees to disease. It could be a quite good analogy, if the lessons taken from it weren’t just so childishly naive.

edmundsauto2• 6 years ago

Can you elaborate on your preferred strategy?

tartoran• 6 years ago

The low hanging fruit metaphor meaning do the simplest or easiest work first is not really applicable to fruit growing but seems to have been coopted by the busieness world instead, then we started using it too in our contexts. Not crazy about this metaphor either, but it could be useful in some contexts and while also misunderstood in others. The same goes for most jargon.

discreteevent• 6 years ago

I doubt many people think of it from the perspective of a fruit grower. I think of it as picking wild fruit, in which case the analogy mostly works.

BiteCode_dev• 6 years ago

Humans have the tendency to think they understand something if they have a label on it. It helps them take decisions faster, whether it's good or bad.

If you tell people you don't know what causes their respiratory illness, they may think you are incompetent, and feel helpless.

If you tell them they have acute bronchitis, suddenly they feel empowered. It just means they have an inflammation of the bronchi. It says nothing of the direct cause, nor the indirect cause: a virus, tobacco, dust, air pollution or asthma. It says nothing about how bad it is. They don't even know really what it implies for their body or their treatment.

But now they think they know.

Call something a democracy, and nobody will check if it is.

Call something a privilege, and people will start to want it, working hard for it.

Models can be used to help your interact with the world. But they are also used to avoid interacting with the world: if you use them as labels, instead of comparing the model with reality, you get a shortcut to take decisions. It becomes a mere name to justify a decision process instead of a tool for the decision process.

LudwigNagasena• 6 years ago

If you tell them they have acute bronchitis, suddenly they feel empowered.

In the The Imaginary Invalid (17th century) Molliere makes fun of it. Doctor explains how opium works through its "dormitive virtue."

mattkrause• 6 years ago

I'm not sure I agree.

While technically bronchitis could be any inflammation of the brochial tubes, a diagnosis of bronchitis usually means that that it's the patient's major problem. It'd be odd to describe a cancer as "bronchitis" even if it involved brochi. So you do learn something, in a Griceian sense, from the relatively uninformative diagnosis. There's also the implication that there are some tools for managing it, the doctor's seen it before, etc all of which are reassuring.

dchyrdvh• 6 years ago

"Mind is the great destroyer of reality" - a quote from one of those ancient dusty books. Our mind is a machine that creates models and labels for everything, to avoid dealing with the illigible reality.

dennis_jeeves2• 6 years ago

If you tell people you don't know what causes their respiratory illness, they may think you are incompetent, and feel helpless.

Agreed, on the medical front this also leads to a focus on sophisticated diagnostic tools and numbers versus a 'true' cure.

Call something a democracy, and nobody will check if it is. Call something a privilege, and people will start to want it, working hard for it.

Let me add one more to the list. Call something a 'right' and people feel denied if they are not given it. Eg. voting rights. Even where one can vote, consent is generally manufactured but people feel empowered when they vote.

earthboundkid3• 6 years ago

Medicine is super-bad about not being able to tell a symptom from a disease. A lot of "diseases" are just symptoms with Latin names and a set of empirical remedies.

Psychology in particular seems to have come to the conclusion that if you can name an assortment of 4 out of 10 symptoms, that counts as having identified single disease.

I think part of the problem is that we're so good at applying various techniques that no one bothers to think anymore.

schnable• 6 years ago

Which leads to the saying that this headline is a play on: "All models are wrong, but some are useful." (George E.P. Box)

We probably should be repeating that a lot to make sure people hear it.

puranjay4• 6 years ago

I fail to understand how you can reasonably model an unprecedented event in modern history. We have no data on how people will act under a weeks, even months long lockdown. Will they stay indoors and follow guidelines? Maybe. Will they watch their livelihoods get affected, their mental health deteriorate, or get careless over time and break quarantine? Maybe.

We just don't know because we just don't have the data or heck, even anecdotal evidence from similar events in the past

ajuc• 6 years ago

There's very little unprecedented events in modern history, and pandemic certainly isn't one.

puranjay4• 6 years ago

Complete lockdowns across multiple countries and cessation of all economic activity certainly is unprecedented.

martingoodson7• 6 years ago

If you're interested in this specific area, here is a useful summary of what's known: https://www.thelancet.com/journals/lancet/article/PIIS0140-6...

tl;dr not much

veddox• 6 years ago

On the contrary, epidemics are reasonably easy to model (at least at a basic level), and modelling them has a long academic tradition. As with all modelling, it all depends on what kind of detail you want to get out of your analysis/predictions.

If you‘re looking at a simple population infection model, lockdown efficacy is just a factor affecting the contact rate among uninfected actors. You run multiple scenarios for multiple levels of this factor, and see where that gets you.

Or did you mean something else?

(Source: I‘m not an epidemiologist, but as an ecological modeller I work with very similar tools.)

puranjay4• 6 years ago

So many of the models I've seen focus on the efficacy of lockdowns in controlling the spread of the disease. Most treat the length of the lockdown as a mathematical variable - X week lockdown leads to Y infection rate.

My point is that we can't reasonably predict how people will behave in lockdown past a certain point. We've never had similar lockdowns in a world that was as globalized, as hyper connected as ours. You could go from 2 to 8 weeks of lockdowns if everyone was living in isolated villages a la 1918 Spanish Flu, but that's not our present world.

How do you model a situation where after 4 weeks of lockdowns, a social media post about food shortages goes viral, causing mass panic and breaking of quarantine?

We can't because we've never had a situation like this, or the tools for spreading (mis) information as we currently do.

heurifk• 6 years ago

We have recent data from Wuhan.

We have old data from 1918.

This is a white swan, not a black one.

More importantly, the mortality, while much higher than flu, it's still relatively low.

Now imagine a virus as contagious as this one, but with 10% mortality over all age groups. That would be unprecedented and probably cause society meltdown.

puranjay4• 6 years ago

How do you reckon social media will impact your models? Rumors and information - real or fake - has never been easier to spread in the world (Wuhan's data becomes less relevant here). We've already seen calls for defiance of lockdowns in right wing circles. In my country, old videos are being circulated to spread misinformation about the treatment.

Again, our data for all older epidemics is applicable to the epidemic in isolation. But there is no way to accurately model how the epidemic interacts with people simply because the way people live has changed drastically from past epidemics.

a1369209993• 6 years ago

More importantly, the mortality, while much higher than flu, it's still relatively low.

Anecdote: someone was trying to convice me to panic about Coronavirus because "three hundred and [something] people died just today!"

  $ units
  8 billion / 80yr
  /day

"Actually, three hundred thousand people died today. Probably more, even."

yk• 6 years ago

If you look at the models, they have something like an order of magnitude uncertainty. And the reason for that is precisely what you are stating, they rely on future behavior, which we just don't know yet.

However, the utility of the models is to give us a sense of how the different parameters interact. There are parts of the model were we can have a lot of trust, for example that people with severe conditions will need hospitalization, or that people will react quite similar as they reacted yesterday. So for the short term, the models give us quite good guidance, and for the long term, they help to map out scenarios.

So if you actually look at the report in question, you will see that they are actually trying to estimate the impact of various non medical interventions, like encouraging social distancing, by comparing different countries. It is just that newspapers as usual just run with the most immediately digestible number, independent wether that number is important or useful.

The study in question:

https://www.imperial.ac.uk/media/imperial-college/medicine/m...

Some overview video from Dr. Campbell on youtube: (in general, I think his youtube channel is quite good)

https://www.youtube.com/watch?v=c1aoULlMpn0

DoofusOfDeath• 6 years ago

Regarding the "all models are wrong" maxim...

Is that statement 100% true for the low-level models that physicists use and develop? In particular, I'm curious if quantum-physics models are 100% right, just not 100% precise.

tomrod• 6 years ago

I keep this distinction in mind:

- Models are deliberate simplifications of reality, in order to guide thinking and otherwise pull in only important information

- Formulations (formalizations) are encapsulation of principles into a mathematical framework

While there is significant overlap, the two categories do not overlap 100%. I see formalizations of physics as the latter, and we use the former to help keep our understanding of the latter clear.

Jabbles• 6 years ago

Some predictions of Quantum Electrodynamics have been verified to within 1 part in 100,000,000.

https://en.wikipedia.org/wiki/Precision_tests_of_QED

However, we know it cannot be completely accurate, as it has no way of explaining gravity.

carapace2• 6 years ago

The Standard Model has no known contradictions in the real world. https://en.wikipedia.org/wiki/Standard_Model

In "QED" Feynman states that the predictions are as precise as being able to measure the distance between (points in) New York and Los Angeles accurately to within the width of a human hair.

https://en.wikipedia.org/wiki/QED:_The_Strange_Theory_of_Lig...

magicalhippo• 6 years ago

Neutrino oscillation[1] is a contradiction of the Standard model as-is, no?

[1] https://en.wikipedia.org/wiki/Neutrino_oscillation

qchris• 6 years ago

At the level that you're talking about, you'll start running into the unresolved questions of modern physics.

One of the best ways to look at this is through the Standard Model of particle physics, which essentially defines how the fundamental particles of the universe are related. Between the number of observations and large-scale experiments dealing with high-energy collision products, astrophysics, neutrino detectors, etc., some people consider the Standard Model to be the most thoroughly-tested and verified framework in all of science. That's a pretty grand claim, but hey.

But it still falls short in some ways--for one, it starts breaking down past a certain scale. Classical field theory as defined by general relativity (another model that has had enormous success under test both theoretically and experimentally) and particle physics don't get along. Neither one fully explains reality, and the interface between those two models of reality hasn't been found. That's why people research things like string theory--they're attempting to find a mathematical framework that can resolve those two frameworks, among other things.

So while each of them describes the universe extremely accurately in their own domain (check the sigma values and number of observations of experiments run on the LHC), they're not 100% right, since they can't be correctly extended to cover all scales and frames. The models remain just that--models which provide a useful framework to interpret reality, but don't fully describe the physical reality itself.

lidHanteyk• 6 years ago

Yes, it is completely true, and not in a probabilistic way, either. A model is a mathematical construction which relates observations to predictions. However, there is no epistemic basis by which predictions can be turned into observations; we can never ultimately draw conclusions just based on inferences.

Perhaps you have heard of the idea that "the map is not the territory" [0]. Models can never be exactly descriptions of reality, not without some sort of special rationale and argument from the outside of reality.

In particular, QM models aren't 100% right. Gravity is missing almost entirely from the model (!) and there are some glaring experimental discrepancies, particularly around the vacuum catastrophe [1]. We know that the combination of QM and relativity gives a hybridized model that cannot work at all scales but explains things like the color of gold [2], so we know that there ought to be a single unified model which does work at all scales and has the same explanatory power.

[0] https://en.wikipedia.org/wiki/Map%E2%80%93territory_relation

[1] https://en.wikipedia.org/wiki/Cosmological_constant_problem

[2] https://en.wikipedia.org/wiki/Relativistic_quantum_chemistry

paulsutter• 6 years ago

"the best material model of a cat is another, or preferably the same, cat” - Norbert Weiner

LittlePeter• 6 years ago

Wiener

firefoxd• 6 years ago

This is a good time for schools to go back to teaching about Ignorance:

https://www.nytimes.com/2015/08/24/opinion/the-case-for-teac...

kgwgk2• 6 years ago

The FT chose to run with an inflammatory headline, assuming an extreme value of ρ that most researchers consider highly implausible.

The headline "Coronavirus may have infected half of UK population — Oxford study” is not really out of line with the preprint the article talks about. What's questionable is that it was a good idea to write about the preprint at all.

"Importantly, the results we present here suggest the ongoing epidemics in the UK and Italy started at least a month before the first reported death and have already led to the accumulation of significant levels of herd immunity in both countries."

"Our overall approach rests on the assumption that only a very small proportion of the population is at risk of hospitalisable illness. [...] Three different scenarios under which the model closely reproduces the reported death counts in the UK up to 19/03/2020 are presented in Figure 1 . [...] [In two of those scenearios] By 19/03/2020, approximately 36% (R 0 =2.25) and 40% (R 0 =2.75) of the population would have already been exposed to SARS-CoV-2. [...] [The third scenario] suggests that 68% would have been infected by 19/03/2020."

The secondary headline “New epidemiological model shows vast majority of people suffer little or no illness” was much worse as that's the assumption in the model and not the result. It was changed to "New epidemiological model shows urgent need for large-scale testing" in the amended article.

Since its publication, hundreds of scientists have attacked the work, forcing the original authors to state publicly that they were not trying to make a forecast at all.

"Attacked" sounds as if the criticism was unwarranted.

galaxyLogic• 6 years ago

Maybe the best thing about models is not the numbers they produce but the insight into how different variables affect each other, how steeply.

But this information is already contained in the model itself. Therefore people should be reading the models, not their results.

The models should of course be verified by comparing their results to empirical data. But that does not often exist with global things like pandemics and climtae change.

rossdavidh• 6 years ago

So, I basically agree with everything this article says, but it seems to miss a basic point. If journalists do what this paper suggests, they make less money.

Journalists, and the newsmedia corporations and organizations that employ them, don't run with the most inflammatory headline possible as an accidental fluke of a mistake that they were too careless to catch. Even public sector newsmedia organizations use measures of how widely read their articles are, as a figure of merit to how well they are performing. Private sector newsmedia are rewarded financially in more or less direct proportion to how widely read (or at least clicked on) their articles are, not how well informed the reader is after they're done reading it (if they even do read past the headline).

If there is one less to be learned from this whole Covid-19 debacle (and I'm sure there are several), it is that our entire news ecosystem, public and private, is fundamentally structured wrong for doing what is supposed to be its purpose, which is to make people better informed. It's not bad at it by mistake, it's bad at it as an inevitable consequence of its design.

jbreckmckye• 6 years ago

A small (but important) correction: generally, journalists do not write headlines; sub-editors do. Most subs have been journalists but most journalists do not become subs.

marcus_holmes2• 6 years ago

I think the distinction is moot. They all work in the newsroom, and are that side of the "journalistic integrity" divide. Subs are not part of the management team, or commercially motivated.

chris_f• 6 years ago

Exactly. I stated calling systems that work this way "Bad by Design."

Sometimes the design is deliberately setup that way by an individual or group with something to gain, but other times it is just because the system contains different parties with competing interests.

Anytime I see a clearly inefficient or ineffective system the first question I try to figure out is whether it works that way by design. In many cases once I learn more about it, the answer is yes.

alharith3• 6 years ago

I don’t think the deficiencies of major news organizations have ever been as blindingly obvious to me as they have been over the past month. I knew it was bad. I didn’t realize exactly how bad it was. For them, even a national crisis is just an occasion to twist against Trump. Yes Trump is far, far, far from perfect, but there’s a huge difference between honest, factual reporting and these transparent efforts to use a tragedy to take him down. Nothing more obvious than the attempt to pit Dr. Fauci as somehow being "against" Trump or being "silenced" only for him to come on national radio and set the record straight.

parasubvert• 6 years ago

The anti-Fauci stuff is mostly posted by GOP-partisan organizations though as a way of bolstering Trump.

On the other hand, it is pretty standard mainstream news when in the same presser, the doctors are saying the opposite of what the President is saying.

MaxBarraclough3• 6 years ago

If journalists do what this paper suggests, they make less money.

This is clearly true, but I wonder if must always be so.

In academia, lying in a publication can be career-ending, which acts to broadly 'scare them straight'. For academics, it's desirable to be highly read and cited, but to be seen to be dishonest is the end of the road. Could it be possible to create similar incentives for journalists?

mola2• 6 years ago

It's easy to lie without lying. As a society we replaced any sort of value for our value proxy, money.

So truth ethics etc. are all secondary values.

JadeNB2• 6 years ago

In academia, lying in a publication can be career-ending, which acts to broadly 'scare them straight'. For academics, it's desirable to be highly read and cited, but to be seen to be dishonest is the end of the road. Could it be possible to create similar incentives for journalists?

The problem is that these incentives for academics aren't created out of purity of heart, but because of the system to which they belong; you will no longer be highly read and cited once it is clear that your results can't be trusted. Unfortunately, this seems not to be true in journalism—you can purvey intentionally, and explicitly, wrong information, and it will still be consumed actively by those whose biases are confirmed by it. We can discuss how to change that, but changing what kind of news people want to read is surely even harder than changing what kind of news journalists write and publishers publish.

wayoutthere3• 6 years ago

This doesn't really solve the Fox News problem: if an audience agrees with what you're saying, they will begin to trust you regardless of the facts. Many people have no way of knowing what the truth really is when they get their news exclusively from the a small group of propagandists coordinating a message.

Who gets to decide when someone is lying? What happens when that power to determine truth from lies is obtained by a bad actor to create a new truth?

sbierwagen• 6 years ago

In academia, lying in a publication can be career-ending,

Can be. Among others, Matthew Walker still has a job. https://yngve.hoiseth.net/why-we-sleep-institutional-failure...

mikekchar• 6 years ago

At least these days, it's not so easy for a journalist to have a career ending mistake. My example for this is Daniel Lyons, for whom I have practically no respect at all as a journalist. His participation in the SCO debacle should legitimately have been career ending (and IMHO was not at all mitigated by his subsequent apology). His "Fake Steve Jobs" work breathtakingly poor judgement, again IMHO. I mean there is trash journalism and then there is making stuff up and hinting that maybe you know more than others do (all the while saying, "It's just a parody"). Let's not even talk about bit that's mentioned at the end of his Wikipedia article (I really hope that's not true!). But let's face it, he's a talented writer. He writes engagingly on topics that are interesting for a lot a people. The fact that he exercises absolutely no judgement whatsoever in his research is unfortunate, but even this is appealing to a lot of people who want to believe what he writes. I suspect that there are more people on HN who love him than hate him. How would you filter this kind of writing without some draconian censor forbidding him from publishing?

watwut2• 6 years ago

The majority of publications and articles did not got it fundamentally wrong. There is subset on right wing that is popular and got it wrong intentionally. But mainstream had bad article here and there while majority being not fundamentally wrong.

renewiltord• 6 years ago

The problem is on the demand side, not solely on the supply side. People desire reinforcement not challenge. That is natural. I suspect it is true of us all. Those who are capable direct their potential to be challenged in specific ways, allowing themselves to have biases reinforced in the unimportant places. And they have access to the knowledge in their specific areas.

This is the reason for the enduring power of the filter bubble: it is a stable equilibrium because it serves both the purposes of the demand and supply side.

You can test this by making high-reliability websites that state honest priors. You'll get an audience but the audience will be pretty specific to your subject matter and not be popular. No information source has had all of the following characteristics:

* Broad-based popular support

* High information content

* Novel information, i.e. information you can't get elsewhere

* Sustained presence

This may actually be desirable. Novel reliable information is an advantage, but it may not be a present sufficient advantage, and species survival may depend on presently boosting those capable of acquiring and utilizing information advantage. i.e. a time may come when we need to be good at it - if we have more people with this characteristic then, it'll lead to better outcomes.

jlawson• 6 years ago

Definitely true. We could compare it to the food industry (from production to retail including restaurants).

Its ideal purpose is to provide people with nourishing, healthy, enjoyable food.

But its actual incentive is to give people the food they choose to buy, which often isn't nourishing or healthy.

marcus_holmes2• 6 years ago

This is true. You can do good journalism and lose money (or have a wealthy backer who funds your loss-making newspaper) or you can make a profit by getting your "journalists" to publish lots of sensationalist stories. Y'know, Clickbait. (source: I ran a newspaper).

Viliam1234• 6 years ago

The problem is also in the change of how the demand can manifest. For example, I would never buy a clickbait newspaper. But sometimes I click on clickbait articles online. In both cases, there is a part of me that is interested, and a part of me that knows better. If it only requires a mouse click, the interested part sometimes wins. If it requires taking out my purse, the interested part loses this fight.

The horrible thing about online advertising is that it allows people to make profit from making you look at something, even if it immediately makes you disappointed. From that moment, the need to write non-disappointing articles has decreased significantly.

amitdeshwar• 6 years ago

What about 538?

closeparen2• 6 years ago

This is well known in the industry. The attitude among journalists was that they had a sort of professional and civic duty to “feed people their vegetables.” But that’s a bit too high on Maslow’s hierarchy to be thinking about when you’ve laid off 80% of the staff and still barely make payroll. (My dad lived through the peak and much of the decline at our city paper before he got out).

sah2ed3• 6 years ago

If there is one less to be learned from this whole Covid-19 debacle (and I'm sure there are several), it is that our entire news ecosystem, public and private, is fundamentally structured wrong for doing what is supposed to be its purpose, which is to make people better informed. It's not bad at it by mistake, it's bad at it as an inevitable consequence of its design.

You speak of a failure in design as the root cause of the failure to achieve the purpose of “to inform”.

From a teleological perspective, viewing the media as a tool designed “to inform” would be a mistake, it would amount to nothing more than a supposition.

Having “better informed” people was never the stated purpose of mass media so it would be wrong to say it is not fulfilling its design objectives. What would be applicable here to help us better understand the actual purpose of the media is POSIWID: ”the purpose of a system is what it does”[0]

[0] https://en.wikipedia.org/wiki/The_purpose_of_a_system_is_wha...

rytill• 6 years ago

POSIWID

Can we either make a reasonable-sounding word or phrase that means the thing or just say the thing? Who is going around really saying “POSIWID”?

perl4ever3• 6 years ago

"From a teleological perspective, viewing the media as a tool designed “to inform” would be a mistake, it would amount to nothing more than a supposition."

I and perhaps many other people think the media should inform people; that is a statement of its purpose, and it appears that the collective actions of people in society have redesigned it to eliminate most of the informing.

It doesn't have to have ever been designed to inform by specific and aware human intention; if it informed as a side effect of being unoptimized for disinformation, then in that sense it was designed to inform.

By analogy, natural foods are not designed, in one sense, to keep you healthy, as evolution can make some things good for you and others bad.

...but the purpose of eating them is healthy nutrition, and if we started systematically only eating the toxic ones, then we would say something about our society is structured wrong for the purpose that we have of getting nutrition.

monocasa2• 6 years ago

And particularly, as the concept applies to mass media, quite a few have noted that the purpose seems to be to create the consent of the populace for what the leadership wants to do.

Sometime's that's informing from a real benevolent perspective of let's get everyone on the same page for what's best to the best of our knowledge. Other times it's an obfuscation scheme to allow some to profit off of the rest of us.

https://en.wikipedia.org/wiki/Manufacturing_Consent

kerkeslager4• 6 years ago

If you're smart enough to come up with a point like this, why not use that intelligence to make a less inane point? You're not wrong, you're just correct in a way that responds to a very narrow reading of what your parent comment was saying, and doesn't make any attempt to figure out why they'd be saying it.

Clearly the person you're responding to wants a system that makes people better informed, and I think there's a lot of other people (myself included) who want the same thing. So the question is obviously, how do we change the news ecosystem from what it is, to a system that makes people better informed?

akiselev• 6 years ago

From the BBC's mission statement: to act in the public interest, serving all audiences through the provision of impartial, high-quality and distinctive output and services which inform, educate* and entertain.*

From the CBC's mandate: the Canadian Broadcasting Corporation, as the national public broadcaster, should provide radio and television services incorporating a wide range of programming that informs, enlightens* and entertains*

From PBS's mission statement: PBS is a membership organization that, in partnership with its member stations, serves the American public with programming and services of the highest quality, using media to educate, inspire, entertain and express a diversity of perspectives. PBS empowers individuals to achieve their potential and strengthen the social, democratic, and cultural health of the U.S.

You'll find that kind of language repeated for most public broadcasters. The corruption of those ideals by private enterprise in the name of profit should not come as a surprise.

sgustard2• 6 years ago

I get why corrupt politicans hate the media, which is because it holds them accountable. Why do you choose to help them tear it down? Are we better off with no media and just the raw tweets of our leaders?

AgentME• 6 years ago

The article here is about how the media can often help spread misinformation, which can be really bad in a pandemic, so going to guess that's what they're concerned about.

lonelappde• 6 years ago

Thia isn't specific to news. It's everything in the marketplace. Anything for sale is optimized for short term profit -- lowest cost of production, highest impulse purchase pressure -- junk food, fake healthy food, furniture made of shoddy materials, shiny appliances that will break, etc.

dntbnmpls2• 6 years ago

If there is one less to be learned from this whole Covid-19 debacle (and I'm sure there are several), it is that our entire news ecosystem, public and private, is fundamentally structured wrong for doing what is supposed to be its purpose, which is to make people better informed.

No. What is wrong is that people believe the news industry exists to make people better informed. That is an industry PR lie such as "Do no evil", "Fair and Balanced", "Serve and Protect", etc. The truth is that newspapers have always lied because they were created to lie.

"Nothing can now be believed which is seen in a newspaper. Truth itself becomes suspicious by being put into that polluted vehicle."

"I will add, that the man who never looks into a newspaper is better informed than he who reads them; inasmuch as he who knows nothing is nearer to truth than he whose mind is filled with falsehoods and errors." -- Thomas Jefferson

The NY Post was created by Alexander Hamilton solely to attack his political opponents. The lies hamilton printed in the ny post about aaron burr is one of the reasons why he got killed in the duel.

Your frustration stems from a false premise : News industry exists to keep you informed.

Instead of banging your head trying to make your conclusions fit a false premise, why not do away with a false premise?

All men are good. Harvey Weinstein is a man. Therefore, Harvey Weinstein is good.

Would you waste your life trying to find ways to show Weinstein is good? Of course not. You'd work your way through your logic and conclude your premise was wrong and that all men are not good.

wavegeek• 6 years ago

Incentives matter

mxcrossb• 6 years ago

Let me start with the following caveat: I’m not convinced journalism’s quality now is worse than any time in the past.

That being said, I think content aggregators are partly to blame. I mean if you’re just reading the daily paper, maybe the front page has exaggerated headlines, but once you’re in the issue it can mellow out. But content aggregators are always trying to find the latest splash.

IBCNU• 6 years ago

All models are wrong, some are deadly.

cousin_it2• 6 years ago

Journalists must get quotes from other experts before publishing

No, this isn't enough. This whole way of thinking isn't enough. It's a big part of the reason for the current situation.

Journalists should report what's true, not what Tom, Dick or Harry said. If a journalist isn't qualified to make object-level claims on a given topic, don't write on that topic. For example, if Bob says there's a forest fire, then instead of publishing "Bob says there's a forest fire", you must do enough legwork to tell your readers "There's a forest fire" or "There isn't".

I allow myself to ignore all journalism that don't follow that guideline, and it makes me happier.

acqq2• 6 years ago

Journalists should report what is true, not what Tom, Dick or Harry said.

Especially they should not give 33% plausibility to Tom, 33% to Dick and 33% to Harry. Which is what they typically do, and call that "professional journalism."

If Tom represents 95% of scientists and Dick and Harry the fringe 5% also financed by (let say) tobacco industry or oil corporations, or Boeing, or those paid by the CIA, they should not even mention Dick and Harry in the same article (or TV a show), especially not in anything worth a major headline. They should appear with the smallest possible note in some smallest possible corner and with the title like "oil corporations paid these persons to support them again."

Sadly, but that sounds like a dream. The world would be very different then.

thu21114• 6 years ago

What you describe here is a deeply problematic fallacy and in fact what journalists already do. They are very, very far from your hypothetical "33% to Tom, 33% to Dick, 33% to Harry" scenario and it's one of the factors that undermines trust in journalism.

What actually happens is this:

1. Journalists have a story they want to write. They probably already decided what the message is going to be, but let's be generous and assume they didn't.

2. They consult a rolodex of, almost exclusively, government funded academics. This is true even if the story they're writing is about activity or science that takes place only in the private sector and for which the academics in question have no actual experience. They may consult two or three grant funded academics if they want their story to seem especially robust. If they give the private sector a chance to reply at all (often not) they will cite one or two sentences of a brief phone call in which the person being talked to doesn't know what they're trying to defend themselves against and is probably just confused. It's not a real interview with an actual in-house expert.

3. This story is then published as "Experts say, ..." even if what the chosen experts say flatly contradicts common sense or things that can be checked with 10 minutes and a search engine.

4. Readers comment below the line, pointing out the flaws in the story. If a specific company is involved, they may do a blog post explaining their side of the story which the journalists will either completely ignore, or if they think they can get away with it, selectively quote one or two sentences in a misleading way.

Industrial scientists/engineers are sometimes assumed to be inherently evil and untrustworthy. But the whole idea that journalists are untrusted because they very rarely quote people in industry is a saw of the left; it's not true, which is why the examples given always seem so curiously weird and out of date. Tobacco industry? when was the last time you even read a newspaper article about them? Oil corporations? Those firms that have spent the last 15 years rebranding themselves as energy companies because they now make solar panels too?

There's a lot of really good analysis of the trouble journalism has got itself in for, and a significant part of the blame is laid at the feet of journalists uncritically reporting anything academics say as the Whole Truth and Nothing But The Truth, when in fact they routinely contradict each other, make up statistics, report obvious common sense as "findings", are hugely over-confident in their own predictive abilities, mis-use statistics and so on.

A good book to read on the topic is "Wrong" by David Freedman:

https://www.goodreads.com/book/show/8134625-wrong

He's a former journalist and so has direct experience of this problem. He cites many examples where expert testimony caused misleading or wrong stories to be published, but IIRC none of them involved corporate scientists. Mostly academics like nutritionists, psychologists and so on.

sgustard2• 6 years ago

Um, so if you want to publish an article on how many people will die of the virus, how do you ensure it's true? Obviously you either don't write the article, or you quote the most reliable projections you can find. But those are still projections.

gpm• 6 years ago

You can't honestly write "an article on how many people will die of the virus" which gives a number. No one knows.

What you can do is write an article about projections of how many people will die. In it you talk about the data sources being used in the model(s), the assumptions being made in the model(s), and the result they give including likely major sources of error..

You can put more or less detail depending on your audience. But it should always include enough detail that your audience understands that it is an estimate based on assumptions and likely flawed data, and you should always understand the model you are writing about even if you don't explain it.

qppo• 6 years ago

That's a bad example. Take the claim:

- 3 million people lost their jobs <citation link>.

- 3 million people lost their jobs, according to <expert>

- 3 million people lost their jobs, according to <expert>, while <another expert> estimates as many as 5 million during the same time frame.

Which one of these is the best framing of "the truth?" Because rarely is something worth reporting on some axiomatic statement of fact. Not only because boolean states don't normally exist - they're not compelling.

A journalist's job isn't just to tell you something happened. But to give you understanding and context, and make it compelling. What you're asking for is Wikipedia, not the news.

Quanttek2• 6 years ago

Uhmm, no? Reality is way too complex for anyone to make a statement about what is "true" - even for domain experts and especially for journalists, whose, at best, entire specialization is "public health". We may be able to state what is "true" for very obvious matters where there are clear dividing lines for a matter to qualify (e.g. forest fire, election results). But when it comes to the sciences, it is incredibly difficult to make any such statement, especially when it comes to emerging research and predictive modeling.

Instead of journalists demonstrating the effect of Dunning-Kruger in a manner similar to what many computer scientists and engineers love to do about unrelated fields, they should rather listen to the experts and try to gather multiple opinions in order to triangulate what is probably correct.

cousin_it2• 6 years ago

Given a choice between an engineer who did some calculations on an unfamiliar topic, and a journalist who triangulated expert opinions but didn't do any calculations, I'll listen to the engineer.

nitrogen• 6 years ago

Would we encourage an epidemiologist to apply ‘fresh thinking’ to the design of an electrical substation?

Yes, absolutely. If an epidemiologist identifies and models a trend in human disease around substations, or a trend in failures of substations, or a new way of modeling the ways electrical demand can change over time and influence demand in other times and places, etc., then their input should absolutely be considered.

Just as it's annoying when a total outsider claims to know everything about a field, it is equally problematic when insiders refuse to acknowledge anyone on the outside.

salty_biscuits• 6 years ago

That rubbed me up the wrong way too. Applied mathematics is really quite transferable across disciplines, you just usually don't have the domain knowledge to stop you chasing dead ends or reinventing the wheel.

Avamander• 6 years ago

Not to mention that sometimes experts in their fields miss common knowledge in other areas. Like the biologist that reinvented integration, Riemann sum - http://care.diabetesjournals.org/cgi/content/abstract/17/2/1... (https://fliptomato.wordpress.com/2007/03/19/medical-research...).

noobermin3• 6 years ago

The easy medium is talking to other scientists, may be getting them on as a co-author.

m0zg• 6 years ago

I'd like to point out that we _still_ don't know if that "high R0" model was right or not. And we won't know that until we randomly test a sufficiently large random sample of the UK population for antibodies. They imposed containment _very_ late, and they go to pubs _all the time_. It is not implausible that the majority of their population already had COVID19 without even knowing what it was.

What we do know is that "doomsday style" models from IHME that just last week were predicting 50K beds needed in NY are off by a factor of 3-4, and hospitalization are starting to flatten out already. You can guess the direction they were wrong in. And before you start an uninformed argument, yes, these models assume the current isolation measures.

In the meanwhile NY hoarded the ventilators and medical supplies because it anticipated this prediction to be true. To be clear, I don't blame NY - they used the best information they had, which turned out to be bullshit. Better be safe than sorry.

These are not harmless errors. When this is over, someone should study these fiascos and estimate the death toll just from bad models alone.

https://twitter.com/AlexBerenson/status/1246465515704463360

dham• 6 years ago

All I can think about with these models in general is the book: Jurassic Park. A lot of talk about chaos theory in the book(kind of skimmed over in the movie). I don't know how accurate all of it is, but it seems trying to model how a dinosaur park would work in the modern day could be a lot like modeling what a virus will do(although probably easier to model a Dinosaur park)

strenholme2• 6 years ago

I am working on trying to make an accurate model for predicting COVID-19 growth based on the per-county figures we have with COVID-19 (courtesy The New York Times). To say the data is noisy would be an understatement.

What I have found, so far, is that if we look at current daily growth (averaged over seven days) and use exponentiation to predict future growth based on the previous week’s figures, the numbers are too high (usually by a factor of two, but the error amount is all over the place).

Point being, we’re seeing a more complicated growth model than simple exponential growth; the actual growth is lower.

My work so far is on GitHub: https://github.com/samboy/covid-19-html

This is a work in progress and I’m nowhere near being able to make a simple easy to read graph showing a reasonable projection of COVID-19 growth in the United States.

justanotherc2• 6 years ago

It could only be exponential if the population was infinite. It has to be more of a bell curve, because as infection grows, there are fewer hosts to infect, and therefore growth would start to decline.

strenholme2• 6 years ago

Even in places where 0.05% of the population has confirmed cases of the virus, I’m seeing the curve flatten. Whether that’s from quarantine of from the virus hitting its limit, I can not say. But the curve is flattening.

wazoox• 6 years ago

A pretty good model is the one I've found in this random Google Sheet:

https://docs.google.com/spreadsheets/d/1PqMVPU0VYcDWrUnDh1Cs...

Basically, it uses the fact that mean time from contagion to death is 17,3 days. For France, it gives reasonably accurate predictions. Best than almost any other model I've seen, in fact.

lazyjones• 6 years ago

Regarding rule 2: I stumbled across https://www.sciencemediacentre.org/working-with-us/for-journ... which does a decent job of aggregating expert reactions to questions that pop up in the media.

I have many more issues with current journalism than the author of the blog post, rooted in their "fire & forget" nature of publications (no visible revisions, no corrections, almost all currently accessible articles are too old to be useful or even correct).

johndoe42377• 6 years ago

It is so strange - when I write exactly the same assertions I am getting downvoted into oblivion lol.

65103• 6 years ago

I think making medical claims is quite punishable if one doesn't have a license. This could easily be extended to include journalists and politicians as well as other areas of science (if the stakes are high enough)

It would work like this:

For each field there has to be a committee similar to the ones medical practitioners already have. If the behavior is not up to standards they do an inquiry and if need be the credentials are stripped.

Someone with credentials may sign-off on articles written by journalists and must do so for laws made by law makers.

Such articles and laws stay active for as long as the credentials are valid. If the person dies a new one has to sign off on it. If that doesn't happen the law is abolished automatically and the archived news article will have to clearly state at the top that such validation is missing as-of [say] march 2043. It may also solicit such review.

We can make a convenient api that allows people with credentials to stick our their neck to approve a publication. The list of professionals endorsing the perspective must be made available from the article or law.

cryptonector2• 6 years ago

I think making medical claims is quite punishable if one doesn't have a license.

Well, it's tricky. If you say "I don't believe covid-19 is a serious problem", that can't be punishable as there is a large spectrum of legitimate thinking as to its severity and what trade-offs we should want to make. Such a statement is not remotely like practicing medicine without a license, but some will argue that it is if it could help them shut up the speaker. While saying that "chloroquine phosphate is a good prophylactic for covid-19" to someone who would believe it certainly should be punishable (as attempted murder perhaps! and regardless of whether the speaker is a licensed physician!).

This could easily be extended to include journalists and politicians as well as other areas of science (if the stakes are high enough)

Journalists? Eh, maybe, but government officers generally have privileges and immunities -- good luck getting them to let those go.

Anyways, the rest of your comment reads like 1984. Quis custodiet ipsos custodes and all that. Regulatory capture and all that. There's no Objectivity. There's no way to set up a mechanism that yields objectively-correct results. All systems will be susceptible to collective delusions and other failures. There's no silver bullet here, and free speech should be part of the mix. Reactionary thinking is fun for the angry, but not good for society.

antpls• 6 years ago

While saying that "chloroquine phosphate is a good prophylactic for covid-19" to someone who would believe it certainly should be punishable (as attempted murder perhaps! and regardless of whether the speaker is a licensed physician!).

I would disagree. As an example in the history, saying "The Earth is a sphere" was once punishable by death. Fake news and pseudo science has always existed, but censorship is not a solution. Only research, education and communication can help, and even then, each generation will always have his own set of questions without answers.

I don't agree with the parent' solution either, for the same reasons.

65103• 6 years ago

I didn't put much thought into the post, its a pretty raw idea that could be refined (and of course abused) but it doesn't have to be very Orwellian.

We have professional communities who get things wrong regularly but the rest of the time they do a very good job. Professionals have a tendency to get things right sometimes.

In the media we see journalists interview experts and twist their words into click bait. Journalists loves to attribute the work to professional sources. There is an abundance of professionals who could operate a thumbs up/down interface. Doing this comes with a certain risk to their career. When hiring someone you can pull the articles endorsed from the database. The choice to voice their opinion depends on how much they care about a topic.

The article can be published regardless. The banner will just say something like: "Zero security experts endorsed this article."

Then we will see how many dieticians are willing to put their name under the "Eating chocolate every day can help you lose weight" publication. The endorsement doesn't have to be permanent but the log will be.

tallgiraffe• 6 years ago

Absolutely true. The amount of bogus science being shared is astonishing. Yet, almost no one seems to notice, care or demand better. All that’s said in the article should be, but personally I’ve lost all hope. We might be living through a generation that values TikTok And click bait more than truth, and it won’t end well, but that’s where we are heading.

carlmr• 6 years ago

Would we encourage an epidemiologist to apply ‘fresh thinking’ to the design of an electrical substation? Perhaps we should treat with caution the predictions of electrical engineers about pandemic disease outbreaks.

I kind of disagree with this point. The models we see are mostly statistical models. Anybody with a statistics background (mathematician, physicist, chemist, biologist, engineer, computer scientist, epidemiologist) can have enough of an understanding to make valid points.

Discarding opinions based on somebody's background is an argumentative fallacy in and of itself. You should of course check if somebody is trustworthy, but epidemiologists should be scrutinized under this aspect like everybody else.

jeegsy2• 6 years ago

wouldn't you need to be an SME to interpret the results properly and put them in their proper context? Not to mention what inputs, variable would actually make sense in the real world?

timkam• 6 years ago

An observation from inside academia: there are some (potentially many) researchers who use the opportunity to push obscure models and simulations of their research sub-subdomains, even if they have no knowledge about public health, epidemiology, or even strong past real-world success stories of the methods they are advertising. Sometimes, this happens with a "talk to the media first" approach (before subjection of their work to the scrutiny of actual domain experts or even any form of peer review), which is certainly dangerous.

jeegsy2• 6 years ago

Perhaps we should treat with caution the predictions of electrical engineers about pandemic disease outbreaks

Or former software company CEOs for that matter

All models are wrong, but some are completely wrong

Related Stories