Intermediate sci knowledge associated with overconfidence and negative attitudes

https://www.nature.com/articles/s41562-023-01677-8

by taylorbuley • 3 years ago

250 208 3 years ago

nature.com

izzydata• 3 years ago

Is this similar to the dunning-kruger effect?

xeonmc• 3 years ago

more like the "midwit distribution" memes.

stanford_labrat• 3 years ago

https://knowyourmeme.com/memes/iq-bell-curve-midwit

hard to find a good example that is funny while also not crass/political, but they generally go something like this: https://i.imgflip.com/5dh26p.jpg

lo_zamoyski• 3 years ago

Or "Ackchyually guy"[0].

[0] https://knowyourmeme.com/memes/ackchyually-actually-guy

oldandtired• 3 years ago

It's obvious from the Abstract that they are not referring to the Dunning-Kruger Effect as it is NOT mentioned in the Abstract, which is the highlight of the paper.

Actually who the hell knows when the paper is essentially behind a paywall. The Abstract gives nothing away about what they have found.

lusus_naturae2• 3 years ago

The study has an interesting approach to avoiding self-reporting level of confidence.

We propose and use in the paper, an indirect measure of confidence in knowledge, defined as the ratio of incorrect to ‘don’t know’ answers in any knowledge questionnaire, as long as it had the format true/false/don’t know (or similar). The rationale is that an incorrect answer corresponds to an overestimation of one’s knowledge (more details in the main text).

On face value it seems like a good way to avoid the pitfalls of self-report surveys, perhaps also useful in affective modeling.

svnt3• 3 years ago

How do they/you remove the confound of becoming more conservative vs less confident, or do they assume that overconfidence and being conservative are opposites?

I might be motivated to be more conservative because I have a reputation to protect, but not actually be less confident. This may have an effect just as much as because I am more experienced.

I may also have just learned not to express my overconfidence interpersonally, while still exhibiting it in practice. Expressing appropriate levels of confidence is a skill, not just the absence of overconfidence.

I can’t access the paper currently.

cutemonster2• 3 years ago

Isn't participating in such research usually fairly anonymous? And I'm guessing the researchers didn't tell the participants how they were going to interpret "I don't know" vs wrong answers?

Expressing appropriate levels of confidence is a skill,

That's interesting, confidence can be gamed, just like other traits in personality tests

subw00f• 3 years ago

learned not to express my overconfidence interpersonally, while still exhibiting it in practice

What does that mean? If you're overconfident in the forest and nobody hears it, it doesn't really matter, right?

lusus_naturae2• 3 years ago

Per my understanding of their method, the participants were just asked to do the survey without mentioning what was being measured. It's like you said, someone can manage their expression of confidence to a degree (or not, maybe), but privately they may not do that, i.e. when just doing some surveys for a study. One thing I remember from running my own studies like this: usually the participant is subjected to a series of such tests (called a battery, sometimes) in succession so they're likely to get fatigued. I think that's a tougher effect that can possibly confound studies. Though I am not sure mental fatigue matters so much for this study.

Edit: I'd share the article but have no idea where to host it.

mistermann5• 3 years ago

One issue though is that in realtime cognition under non-laboratory conditions, "I don't know" often isn't available, unlike when it is explicitly given as an answer.

The number of science fans I've met who sincerely proclaim they possess knowledge of the unknowable is scary.

colordrops• 3 years ago

Very meta study.

archarios2• 3 years ago

What does "intermediate knowledge" mean in this context?

archarios2• 3 years ago

Like I have a bachelors degree in engineering. Is that intermediate or advanced? I feel like saying my science knowledge is anything beyond intermediate is a bit of an overstatement personally.

wongarsu• 3 years ago

I feel like saying my science knowledge is anything beyond intermediate is a bit of an overstatement personally

That's not overconfident, I'm lead to conclude that you have advanced science knowledge.

burnished• 3 years ago

I didnt find out what standard they use, but personally I'd say intermediate sounds about right. I also have an engineering degree and while I know more than the average bear in many scientific disciplines I couldn't say that any of it is advanced.

I'm thinking of times where I went to the library to dig deeper on a topic and discovered a huge and complex topic just laying in wait.

at_a_remove• 3 years ago

I have one in physics, but I have also taken a rather a lot (given the degree) in biology, chemistry, civil engineering, and electrical engineering. Went some places also not usual in math. And I've worked in IT since forever. There's a lot of stuff where I take a glance at it and realize I'm just looking at a single hull plate on a battleship.

What I am getting at is that if you have just intermediate knowledge in a bunch of places, I think it lends itself to sensing that you're just a paramecium stuck to the side of some N-dimensional construct. There's so much. I had a professor who was the expert in the second excited state of Helium-3. That was his thing. Just a single needle in the whale-sized blowfish of physics.

threeseed2• 3 years ago

Like I have a bachelors degree in engineering

I do to and unquestionably I am beginner level.

Each scientific field has specialised and become so dense with knowledge that even new graduates in that field would barely be classed as intermediate.

xormapmap• 3 years ago

In my view, bachelor's degree is beginner, especially if you haven't got any further knowledge than that from work experience or anything.

smcin2• 3 years ago

Like high-school level, a very basic familiarity with terms, experiment design, inference. The authors cite and link to the five surveys used in the 'Data availability' section: https://www.nature.com/articles/s41562-023-01677-8#data-avai...

* Surveys EB, Pew and GSS are publicly available and data and details can be found in refs. [27],[28],[29], respectively.

  \* take the interactive Pew survey (11 questions) here: https://www.pewresearch.org/science/quiz/science-knowledge-quiz/

* The Fernbach study was published in ref. [9] and the authors made the data available.

* Lackner survey data are available at: https://doi.org/10.5281/zenodo.7920776

[27]: Bauer, M. W., Shukla, R. & Kakkar, P. Public Understanding of Science in Europe 1989–2005—A Eurobarometer Trend File (GESIS, 2012); https://www.gesis.org/en/eurobarometer-data-service/search-d...

[28]: Smith, T. W., Davern, M., Freese, J. & Morgan, S. L. General Social Surveys, 1972–2018 (NORC, 2019); https://gss.norc.org/get-the-data

[29]: Funk, C., Kennedy, B., Johnson, C., Hefferon, M. & Thigpen, C. L. American Trends Panel Wave 42 (Pew Research Center, 2019); https://www.pewresearch.org/science/dataset/american-trends-...

[9]: Fernbach, P. M., Light, N., Scott, S. E., Inbar, Y. & Rozin, P. Extreme opponents of genetically modified foods know the least but think they know the most. Nat. Hum. Behav. 3, 251–256 (2019). https://www.nature.com/articles/s41562-018-0520-3

smcin2• 3 years ago

(useful to note all those surveys are 2019 pre-pandemic, before there was severe partisanization of the phrase "trust in science". I wonder how hard it would be to construct a neutral methodology post-pandemic, now that even the basic vocabulary itself is loaded with associations.)

Even then, the 2019 Pew article is depressing reading: https://www.pewresearch.org/science/2019/08/02/partisanship-...

calf• 3 years ago

If the abstract actually defined intermediate we wouldn't have comments talking so much about advanced education like PhDs and STEM majors...

Jensson3• 3 years ago

Intermediate knowledge of what's being tested, everyone is the most overconfident on subjects they have intermediate knowledge about.

If you test high school concepts then if you are shaky about high school concepts that applies to you. People who have no clue about high school and people who understand high school well will be less overconfident on that test.

Or if you talk about college algorithms, then that is the basis for intermediate knowledge. An average comp sci grad will be the most overconfident, a person who never studied algorithms and a person who teaches algorithms for years will be less overconfident on that test.

emodendroket• 3 years ago

I feel like we've all encountered this guy.

malfist• 3 years ago

Some of us have been that guy.

not_enoch_wise4• 3 years ago

Hacker News is this

neilv• 3 years ago

One of my favorite parts of n-gate.com's 'summaries' of HN posts was the phrase "incorrecting each other".

mathisfun123• 3 years ago

to wit: <gestures broadly around>

not_enoch_wise4• 3 years ago

Fucking fantastic

maxwell• 3 years ago

Alexander Pope was right.

biomcgary5• 3 years ago

As a scientist, I've seen lots of people who are overconfident in both themselves and overconfident in science. IMO, a good PhD science program helps students master a discipline, but also to recognize the limits of the discipline.

For example, 23andMe thought they would revolutionize drug development with a huge genetics dataset. In practice, genetic information alone is not sufficient to treat the majority of diseases that affect individuals and society. There is too much environmental variation affecting human biology for purely genetic approaches.

Understanding the real limits of knowledge is vital to pushing knowledge forward where we can. As a biologist, one of the things I most appreciate about Sabine Hossenfelder (a physicist) is that she highlights the limits of knowledge in her (and adjacent) fields. She gets a lot of push back (and is sometimes wrong), but having the discussion is vital to science.

Acknowledging the limits of science is not a negative attitude about science, but a positive one. A clear idea of the current limits of science (both theoretically and practically) is instrumental to pushing through them. For example, the scholarly papers highlighting the replication crisis (https://en.wikipedia.org/wiki/Replication_crisis) are actually very useful to maintaining the health of science as a human endeavor, not a critique of science. Scientists need a clear understanding of the scientific foundations that they are building on.

NikolaNovak5• 3 years ago

I find that coloquially we use the term "science" in several distinct meanings, two of which are:

1. Science as the body of knowledge

2. Science as a method / approach

I also find that mixing the meanings/perspectives/intent up in a single conversation is common, sometimes accidentally sometimes intentionally.

In my ignorance (ComSci major, so not a real science :), I would describe myself as extremely positive / confident to "Science, the approach/method" and, through that approach, pragmatic about the "Science the current body of knowledge".

In other words:

Yes, there are very much limits to what we currently know, and some of what we think we know will turn out to be wrong, subtly or catastrophically. There are definitely huge limits and uncertainties to Science the body of knowledge!

But, acknowledging it is kinda the point, and the best way to figure it out that we are currently aware of is through scientific approach. (I've just realized I might even have become a zealot that you describe, because I can't even figure out what a plausible & feasible alternative method is, if your goals are to actually figure things out. To that point, I find humility and skepticism about your current science body of knowledge a crucial part of science the method, something which most other methods lack).

I find distinction is crucial especially in political and religious discussion frameworks. Otherwise, I never know if I agree or disagree with statements regarding "limits of science" etc.

(This is all further mixed up by zillion of daily popular articles where "Science says that [...]!!!" or "[...], scientists find", which... ugh, oversimplify at best and deceive more likely)

What are your thoughts?

theGnuMe• 3 years ago

The science of computing is a real science.

Sharlin• 3 years ago

I think there is a third definition of "science":

3. What is actually happening in academia

I think most people exposed to science#3 from the inside can agree that science#2 works – and indeed works surprisingly well – despite science#3, not because of it.

austin-cheney• 3 years ago

What’s important is not what you think. More important is what other people think, even though their observations are at fault no differently than your own. What’s most important is how you measure compared to other people, as everything else is a biased faulty guess.

Younger people are more at risk of getting this wrong because their scope of knowledge and experience are shallower. Introspection grows with age, but when introspection is not deliberate older people are more catastrophically at risk of getting this wrong.

biomcgary5• 3 years ago

I think your distinction between 1 and 2 is very important (but, as a computational biologist, I disagree about CompSci not being science :-).

Along these lines, I think the role of consensus in science has been overly dramatized by those with various policies to push. Max Planck's principle is famous in the short form "Science progresses one funeral at a time". One of the professors I worked with as a graduate student had a sign on his desk, "First They Ignore You, Then They Laugh at You, Then They Attack You, Then You Win".

These two quotes captures an important tension in the practice of science: consensus both retards the progress of science and captures it for others to build on.

nonrandomstring2• 3 years ago

There is also:

3. Science the institution.

As in "What Science says about X" as if there was a single authoritative, coherent entity we could call Science.

ThinkBeat• 3 years ago

"" > I would describe myself as extremely positive / confident to "Science, the >approach/method ""

But what does that actually mean? How do you manifest this in your daily life and in your decisions?

My grandmother told me God told me I heard on the news

Well, that does not seem scientific nor following the tenants of science. But how can you evaluate information you receive that is called science and in so far as you know from a source of a scientist. Esp. these days in the US everything is incredibly politicized.

There is no way to dig deep enough into every tidbit of knowledge we are exposed to. A lot of scientific fields these days are so complicated that you need a degree to start understanding what is going on or to evaluate data.

If we are lucky we know a few people in different fields whom we trust. We trust the people to we trust what the say, since they are scientists. We all walk around and -believe- in various things we hear and elect not to believe in others. Then we claim that we believe in this or that "because of science". and because of the scientific method. But we dont know that for a fact because we dont know and probably could not understand all the steps from beginning to end needed to ensure that the scientific model had been applied appropriately at all stages.

I dont think real science should state "THIS IS TRUTH BECAUSE THIS IS SCIENCE" it should be "This is our best understanding right now, and there are some other theories out there that may also be valid."

mistermann5• 3 years ago

I would add:

3. Science, the mass psychological phenomenenon.

Like many ideologies it behaves a lot like a religion, and online discussions are chock full of artifacts.

In my case, I have a negative view of "science" because of this phenomenological aspect of it, which I consider dangerous because it results in irrational, tribal thinking (see: covid, climate change, etc)...and it ain't only the "deniers" who are guilty.

Yes, there are very much limits to what we currently know, and some of what we think we know will turn out to be wrong, subtly or catastrophically. There are definitely huge limits and uncertainties to Science the body of knowledge!
But, acknowledging it is kinda the point, and the best way to figure it out that we are currently aware of is through scientific approach.

Pro-science people absolutely love this meme, I encounter it several times a day in the online spaces I frequent.

eh_why_not• 3 years ago

As a scientist, I've seen lots of people who are overconfident in both themselves and overconfident in science.

I feel that I've never had the first problem, but have definitely had the second.

On one hand, there is the natural limitation of Science itself in terms of the type of questions (that are amenable to the scientific method) it can answer. On the other hand, it is still the best way of generating knowledge that we have.

My overconfidence was that scientists, as individuals and as a community, would always do the right thing, driven by, and honestly following, the scientific method. But in the past few years I've had to revisit this assumption several times and be reminded to always retain some healthy skepticism.

Most recent example is this climate scientist who just published in Nature, and then went ahead afterwards and penned an op-ed [0] saying he actually misrepresented the actual factors in order to get published.

[0] https://www.thefp.com/p/i-overhyped-climate-change-to-get-pu...

perfect-blue• 3 years ago

My overconfidence was that scientists, as individuals and as a community, would always do the right thing...

This is a great point. I'm shocked by how often I end up working on a project with a colleague who is taking the path of least resistance. In my field, this usually results in using decades old statistical methods than have been proven time and time again to be unsatisfactory. They just don't want to learn new methods or their technical expertise aren't good enough to learn how to implement the new approaches. So they just coast. I'm not sure what to do in these situations other than just try and set a positive example.

dekhn2• 3 years ago

23&Me failed in drug development because they started from a mistaken premise- that the data they collected (very specifically, genotype arrays) would produce data that was correlated with human health closely enough to identify targets (proteins or pathways to disrupt/modify). They have a huge genetics dataset, they don't have a huge genomics dataset, and the underlying relationship between the genome and phenotypes (especially complex disease phenotypes) is a highly nonlinear function.

Environment is important but we could still have huge improvements in medical care using genomics. It's easy to obtain and still has a very strong relationship to disease, and is a problem best solved by deep learning. I've watched people tilt against this windmill for 25+ years and it's kind of funny just how bad our labelling of diseases is.

biomcgary5• 3 years ago

Excellent point. I work at a biotech startup where one major focus is "how do we fix the disease labels?". I call it the pyrite problem (your gold standard data contains fools gold), but it is known more prosaically as the mislabeling problem.

dahinds2• 3 years ago

This is an inaccurate view of what 23andMe is doing.

First we did not "fail" in drug development, our collaboration with GSK yielded substantially more programs than we had anticipated when we started. The verdict on whether the strategy is successful or not will not be clear for another 5-10 years because being better at picking targets doesn't shorten the timeline for bringing a drug to market, it mostly means that you should have a modestly higher proportion of programs that end up being successful.

Second we did not start with a premise that we could or needed to do a great job of predicting disease from genetics alone, that is not required to identify good targets. You're correct we don't have huge genomics datasets, we largely use the same genomics datasets others use. Identifying targets with genetics in the simplest sense requires using genetics to identify an association at a genomic location, and then using genomics for functional interpretation of that association. And having the largest database of genetic associations enables the first step of that.

I'm not sure what you mean by "the underlying relationship between the genome and phenotypes... is a highly nonlinear function". Simple additive models account for most of the heritability of complex phenotypes.

f1shy2• 3 years ago

The most over confident in theirselves and science people I know are PhD. With egos and arrogance greater than the solar system. People that go early to the industry and see the “real world” tend to have a more tamed expectative of what they could achieve and science can offer.

brigadier132• 3 years ago

Ironically, you are taking this study at face value. This study reminds me of the "Republicans tend to be sociopaths more often" study. That ended up being completely refuted.

crabmusket• 3 years ago

Acknowledging the limits of science is not a negative attitude about science, but a positive one. A clear idea of the current limits of science (both theoretically and practically) is instrumental to pushing through them.

"The gods did not reveal, from the beginning, all things to us, but in the course of time through seeking we may learn and know things better. But as for certain truth, no man has known it, nor shall he know it, neither of the gods Nor yet of all the things of which I speak. For even if by chance he were to utter the final truth, he would himself not know it: for all is but a woven web of guesses"

Xenophanes via Karl Popper

XTHK• 3 years ago

one of the things I most appreciate about Sabine Hossenfelder (a physicist) is that she highlights the limits of knowledge in her (and adjacent) fields.

I suppose you missed her recent video about economics? Her content is getting more and more clickbaitey by the day. Some really half-baked data creeping into her videos.

ramraj072• 3 years ago

It really is sad indeed. You die a hero (stop posting videos once you’ve exhausted your expertise) or live long enough to become a villain indeed.

biomcgary5• 3 years ago

I don't follow Hossenfelder that closely, so I haven't seen any of her videos on economics. I have seen her walk back some of her misunderstanding around global warming, however.

Celebrating someone's achievements in a specific domain is not giving them carte blanche across the board. Personally, I think it more productive to celebrate someone's successes while lamenting their failures. If one thinks that mediocrity is the norm (which it is, by definition), then transcending it, even briefly, it something to celebrate and use as inspiration, even while acknowledging that all individuals have flaws (to some degree).

GoblinSlayer• 3 years ago

As a biologist, one of the things I most appreciate about Sabine Hossenfelder (a physicist) is that she highlights the limits of knowledge in her (and adjacent) fields.

I only heard her arguing others are wrong because she's right. That's not a demonstration of the limits of knowledge.

photochemsyn• 3 years ago

The article is paywalled but the research group has a 2021 arxiv publication on the same topic: "A little knowledge is a dangerous thing: excess confidence explains negative attitudes towards science"

https://arxiv.org/pdf/1903.11193.pdf

It's a survey-based study but the obvious thing to consider is, what do they mean by 'negative attitudes towards science'? Are they talking about the scientific process itself, i.e. experiment and observation coupled to theoretical modeling as the basis of discovering how our universe functions? Or are they talking about negative attitudes towards the 'science-based' pronouncements of governmental and academic institutions?

Certainly the modern scientific process takes place at such a level of specialization that even working scientists in one field are usually unable to judge the quality of results obtained in another field without doing a lot of time-consuming research, but if a long record of failure of peer review exists (and it does) then it shouldn't be surprising when people lose faith in academic and governmental institutions - but I'd guess they still believe that the scientific process itself is valid, it's just that the received wisdom of the white-robed annoited priesthood is no longer taken at face value.

There are many reasons for this - e.g. while science may have been viewed by all as wonderful in the 1950s, many discoveries (environmental carcinogens, fossil fueled global warming, etc.) have upset major economic and institutional powers leading to coordinated attacks on the reliability of science in major media outlets. Then there's the long record of pharmaceutical skullduggery (the push to prescribe opiates for just about any condition, leading to an addiction epidemic, the failure of a wide variety of science-approved medications to live up to claims and/or the production of negative side effects (Vioxx etc.)) - and as far as the vaccine controversy, yes it was a terrible idea to put organometallic preservatives in multi-use bottles of vaccines, and yes it was done to cut costs, but the claim it led to an epidemic of autism isn't well-supported, there are many more plausible industrial sources of heavy metals to blame for high childhood exposures, but that's not as convenient for class-action lawsuits due to vaccination records, etc.

This doesn't mean that most science isn't fairly reliable, but the glaring failures are what make the headlines, and there have been quite a few of them.

If we really want to regain public trust in academic institutions, divorce proceedings aimed at kicking the corporate interests out of the academic sphere will have to be initiated, meaning for example no more exclusive private rights to NIH-financed inventions and no more revolving doors between academic institutions and pharmaceutical executive boards. Don't hold your breath, we live in an era of systematic insitutional corruption that Trofim Lysenko would have fit right into.

bedobi3• 3 years ago

This study basically describes 99% of people here, lol. Because you know how to write code, you're suddenly an expert not only at that but also at economics, medicine, international relations etc etc. Every field is ready for you to disrupt it. The hubris is astonishing.

paulddraper5• 3 years ago

Why doesn't _____ just ______? I've thought about this for only 2 minutes and it's already obvious.

nyc_data_geek1• 3 years ago

I would simply _____.

meepmorp• 3 years ago

And the closely related I could write _______ in _______ days.

svnt3• 3 years ago

Clearly you’re not in the field or you’d know that ______ is actually better understood as _______, which means your question doesn’t even make sense.

cutemonster2• 3 years ago

At the same time, with "just" removed, those questions are often pretty good I think.

It seems I would just remove just, and make the question great again

nologic01• 3 years ago

Indeed. Intelligence is just I/O and gradient descent. The Universe is code. etc.

Coupled with an asocial, greed-driven and essentially anti-humanist agenda its just the worst possible moment (severe environmental stresses across the planet) to hijack whatever potential digital tech offers...

Sigh.

nyrikki• 3 years ago

Unfortunately the lack of a definition of intelligence makes all claims unfalsifiable too.

But Western reductionism/Laplacian determinism is still taught even at the PHD level as cannon and not as a target for practical models.

Perhaps the rise of research into indecomposable continua and the discovery of Strange non-chaotic attractors in nature may help.

Obviously math and computer science as taught today is insufficient.

I particularly blame the way we teach things as absolute truth and then pull the rug out from under those previous supposed hard facts.

But I am showing my own ignorance here because we do learn about the computable set etc...

It doesn't seem to help with people making claims about universal quantifiers being a few quarters away.

passion__desire4• 3 years ago

Is economics really a science?

Economics is generally regarded as a social science, although some critics of the field argue that economics falls short of the definition of a science for a number of reasons, including a lack of testable hypotheses, lack of consensus, and inherent political overtones. Despite these arguments, economics shares the combination of qualitative and quantitative elements common to all social sciences [0]

[0] https://www.investopedia.com/ask/answers/030315/economics-sc...

DiggyJohnson2• 3 years ago

Whether it's a science or social science or research domain or discipline, how is that relevant to this discussion? It doesn't change GPs point.

simpleuser27• 3 years ago

Seriously. Folks in tech like to complain about MBAs, but it's a toss up in my book for "most egregiously overconfident".

Doctors get a good shout as well.

tivert3• 3 years ago

This study basically describes 99% of people here, lol. Because you know how to write code, you're suddenly an expert not only at that but also at economics, medicine, international relations etc etc. Every field is ready for you to disrupt it. The hubris is astonishing.

Aka "Engineer's disease." There's an overestimation of one's personal competence and the effectiveness of the tools and mental models you're most familiar with. Basically think asshole software engineer who tells everyone they're dumb and should just solve the problem like they're writing software.

cududa3• 3 years ago

One thing I've realized about this site are the insane amount of software engineers that have a disdain for the medical field. It comes up all the time deeper into comment threads where people say medical school entrance exams being so hard is a form of "gatekeeping" - which it is, on purpose, but they're using it as a pejorative to tell themselves "I could've been a doctor too if they didn't make it so purposefully hard."

The other very very disturbing trend are the "makers" who denigrate medical devices as overly complex. One thread I'll never get over was on old pacemakers. The prevailing sentiment was they're designed for failure because "evil medical industry profit", not that, you know, they wear out.

The other part felt like watching the theory of memetics demonstrate itself in real-time. One person commented that the single small mechanical component is rated to actuate 5,000,000 times or something like that. Someone dismissively said "Well yeah but Adafruit keyboard switches are rated at 2-3 million presses, they mass-produce them, thus it can't be that hard" (adafruit's data sheet says 1,000,000 btw). Very quickly, people picked up that line and repeated that the actuator in a pacemaker isn't "actually that complicated", citing the 2-3 million keycap example. I still see that keycap "argument" pop up all the time.

BTW: Pacemakers are actually rated for about 100,000,000 stimuli cycles.

mr_mitm• 3 years ago

I wanted to write a very similar comment, except about cosmology instead of medicine. The amount of people with high school level physics knowledge who think they know better than almost all luminaries of the field combined just because their gut feeling is telling them something about dark matter is astonishing.

Jensson3• 3 years ago

the insane amount of software engineers that have a disdain for the medical field

What field doesn't they have insane disdain for? Software engineers even have insane disdain for software engineers!

CatWChainsaw• 3 years ago

The disdain extends to all other fields because they were told software was going to eat the world - they felt like special snowflakes entitled to call everyone else special snowflakes, and it went about as well as you'd expect.

adasdasdas• 3 years ago

Cross field contamination a fantastic product of our interconnected world, but it's more fun to shit on people when they fail. I'm sure some folks here would've loved to tell davinci to stay in his lane.

bratgpttamer3• 3 years ago

Well, at least we're not physicists, amirite?

https://xkcd.com/793/

ignorethefacts2• 3 years ago

Not everyone who posts here is a coder but I do agree with you the narcissism and astro-turfing on HN is high.

I suspect the high-confidence/narcissist combination is especially high here due to this being an industry/entrepreneurial forum where being confidently wrong is a positive trait.

The astro-turfers appealing to the confidently wrong go by unnoticed to the majority.

The abstract of the study states the metric for over-confidence they developed is "the tendency to give incorrect answers rather than ‘don’t know’ responses to questions on scientific facts". I think that's true. I maintain that being confidently skeptical of confident claims and research is not the same thing.

davidktr• 3 years ago

full paper here: // deleted. Looks like my choice of an anonymous file hoster was inappropriate.

malf• 3 years ago

This page tries to send expensive SMS messages.

bagels2• 3 years ago

Site doesn't look legitimate. Risky click.

bagels2• 3 years ago

researchgate has it: https://www.researchgate.net/publication/362183387_Knowledge...

You can click on "Read Full Text"

Also here: https://www.science.org/doi/10.1126/sciadv.abo0038

GMoromisato2• 3 years ago

I blame Carl Sagan[1]. Carl Sagan was the Martin Luther of science communication. He showed us how you could figure out the size of the Earth just by measuring shadows at noon. He showed a vessel full of gases creating the precursors of life with just a little electricity. He reduced the history of the universe to a 1 hour PBS show. In short, he made science understandable to anyone.

But just as the Reformation led to faith-healing and megachurch preachers, Sagan's teachings made it seem as if anyone could be a scientist. As long as a chain of reasoning made sense to you, then it was true!

Take something like whether the earth goes around the sun. Sagan showed how Mars sometimes moves backwards in the sky, and he implies that this can only happen in a heliocentric model. In reality, of course, you need much more evidence to come up with the correct model. It wasn't until Newton could predict orbits from simple equations that there was no longer any doubt.

But watching Sagan you get the idea that if you can just come up with ONE piece of evidence for whatever you believe, then that's enough to prove it! This is why flat-earthers rely on meme-like "evidence" that takes effort to refute.

In the end, it's a trade-off. Do you want people to just trust authority or do you want people to think for themselves? If we want the latter (which I think we do) then we have to put up with those who are wrong.

-----

[1] I love Carl Sagan. He was a huge influence on me, and I think he benefited society, even if I think his teachings also (ironically) stoked some pseudoscience.

santiagobasulto• 3 years ago

Lol, this is a great comment.

nonethewiser3• 3 years ago

It makes me think of the Its Always Sunny in Philadelphia episode where Dennis hypocritically appeals to authority in an effort to disprove creationism. The valuable point the skit makes is that there is a sort of “cult of science” where people mistake faith for reason and you have a bunch of science “believers” who dont realize it. Of course this is not what science really is.

Personally, I have always been irked by phrases like “Science say” as if science itself is a centralized institution rather than a collection of people applying the scientific method.

In a similar vein, I sometimes see posts of some natural phenomena (like ants changing colors after drinking dyed water) with captions like “Isnt science cool?” Im still trying to figure out why this irks me but its weird to imply science somehow caused this.

GMoromisato2• 3 years ago

Lol--that's a great reference.

nerpderp822• 3 years ago

I was just mentioning to a friend of mine, a lady in her late 60s early 70s about spiders using electrostatics to travel.

https://journals.aps.org/pre/abstract/10.1103/PhysRevE.102.0...

And she remarked, "isn't evolution amazing! It figured out so many things"

When someone says "Isn't science cool?", they are saying lots of things

* I appreciate facts/truth

* Knowing things is fun

* Figuring things out is fun

* There is so much we don't know about the universe

I think you are putting to much semantic meaning into the grammar they are using to express their sentiments. Same goes for, "science says". It isn't enjoyable to discuss things and at the same time qualify every single statement.

mistermann5• 3 years ago

The valuable point the skit makes is that there is a sort of “cult of science” where people mistake faith for reason and you have a bunch of science “believers” who dont realize it. Of course this is not what science really is.

Whereas with religions, the shortcomings of it's delusional followers are attributed to it.

Science has got to have the shrewdest marketing department in the history of religion.

ramblenode3• 3 years ago

Everything you just described is what I call "Scientism."

aschearer• 3 years ago

But just as the Reformation led to faith-healing and megachurch preachers

Quite a leap. Break that one down...

Sagan's teachings made it seem as if anyone could be a scientist. As long as a chain of reasoning made sense to you, then it was true!

How does Sagan give this impression? Namely that "as long as it makes sense to you, then it must be true."

And why is this analogous to the Reformation? Seems like a non-sequiter.

paulddraper5• 3 years ago

> But just as the Reformation led to faith-healing and megachurch preachers
Quite a leap. Break that one down...

Prior to the Reformation, you needed not just Truth but Authority as well. There was an actual hierarchy, and you respected that hierarchy.

Luther said that any man could connect to God, be a spiritual leader, assemble a congregation. And so the number of religious sects increased from 1 to much more than 1, including for example faith healers and megachurches.

passion__desire4• 3 years ago

People trust "authority" because they don't have time, resources, training to build the body of knowledge themselves. It is not feasable neither desirable for everyone to know about everything. Societies build chain of trust and that's why institutions are important. Think of authority as a function call in a big framework which provides an answer with reasoning for that answer. In my previous function calls, the authority provided answers properly. If I get a wrong answer, there is a "bug" somewhere in the system.

itronitron3• 3 years ago

Why call it pseudoscience? It's just science done poorly by someone doing science. We don't call a bad haircut a pseudo-haircut.

paulddraper5• 3 years ago

Same as psuedostatistics. There are fundamental, objective principals. [1]

Violate those, and it's not what it claims to be.

A bad haircut doesn't claim to be something that it's not, and its goodness is subjective.

[1] https://en.wiktionary.org/wiki/pseudostatistical

nonrandomstring2• 3 years ago

I think lifespan plays an important part in this. I know a few aging scientists, some with extraordinary achievements, who having accumulated a lifetime of wisdom are finally entering the "We don't really know" phase about every question.

next_xibalba• 3 years ago

I am 100% guilty of this. I try to consistently remind myself that having read mostly pop-sci and the occasional abstract of a scientific paper does not in any way qualify me as either a scientist or even a knowledgeable non-scientist.

koromak• 3 years ago

"They don't know I watched a 20 minute video on relativity"

quickthrower25• 3 years ago

Meanwhile… everyone else aged 10000 years.

itronitron3• 3 years ago

If they want the research to "help inform science communication strategies" then make the article freely available.

ObservingFuture2• 3 years ago

I only glanced at the paper, but I wonder how much of this is explained by just random chance?

It looks like they used multiple choice quizzes to determine both knowledge in science and a propensity to respond "don't know" indicating confidence. Any "don't know" response was counted as an incorrect response, while a correct guess increased the participants "science knowledge".

Thus, a willingness to guess something at random in the multiple choice test would both increase "science knowledge" as well as make the participant appear overconfident.

moffkalast2• 3 years ago

Sounds like your intermediate knowledge of the paper is resulting in an overconfident negative attitude :P

read_if_gay_• 3 years ago

oh irony

cududa3• 3 years ago

Well yeah but that's where data modelling comes in.

ObservingFuture2• 3 years ago

I mean, the data modeling assumed people guessing were doing so completely at random without eliminating any options (In the section "Simulation").

If I'm looking at the right document, one question was about which city out of Chicago, New York, and LA have the greatest annual temperature range (accompanied with a plot). Almost all respondents said New York or Chicago, rather than LA or "All equal".

cycomanic2• 3 years ago

I am not sure why we have all these comments about confidence/overconfidence of scientists and the limits of science.

The article has some very different findings. The crucial paragraph from the abstract is:

We find a nonlinear relationship between knowledge and confidence, with overconfidence (the confidence gap) peaking at intermediate levels of actual scientific knowledge. These high-confidence/intermediate-knowledge groups also display the least positive attitudes towards science.

It seems to that it is essentially a refinement of the Dunning-Kruger effect. It also matches some previous study I remember which found that science sceptisim and many conspiracy theories around science are most prevalent with certain engineering disciplines (I can't find the study right now, but will update with a link once I do).

It is interesting that someone mentioned Sabine Hossenfelder as a positive example, because I find much of her recent content peddles to exactly the crowd who has intermediate knowledge of and very negative attitude toward science. In particular she often comments on topics where she herself has very little understanding (I know because I have seen it for topics where I am an expert) and just pushes a scepticism opinion without much understanding, but acting like she is an authority.

mxmlnkn2• 3 years ago

The article has some very different findings. The crucial paragraph from the abstract is:

It's kinda funny how only the abstract is referred to when trying to paraphrase the findings of the paper. I would say that the abstract by itself is more like an opinion. The real hard data backing up the abstract, should be in the article. But, it can't even be faulted because this "science" is behind a pretty hefty paywall with 40$ for the full PDF and 10$ for a 48h rent. It's basically the same price as a full movie.

cycomanic2• 3 years ago

The abstract is not an opinion, it is a summary of the key methods and findings written by the authors. It is the perfect place to get a short quote to outline the main results of the paper.

Now if we wanted to investigate the methods and results in detail, we certainly would have to read the full paper, however the main findings will not differ from what is in the abstract. You might find that the methods (and hence results) are not valid, and can come up with a detailed rebuttal, for that you certainly would need to read the full paper. However, I assumed here that the findings are valid, because even though I have access to the paper, this is well outside my area of expertise so I am not well placed to investigate the claims in detail and rather refrain from that.

hnthrowaway03282• 3 years ago

From my personal experience (thus does not have statistical significance), it's not the amount of knowledge, but the position. Teachers/professors are a group of people that I interact frequently with (both my parents and parents of my friends are university professors/lecturers), and it just happens that the ones that I know the most are all overconfidence.

Basically they think they know everything, to the point of educating the doctors about medicines when they are in the hospital (as patients). I don't know what got them into this but I found the arrogance distasteful.

Maybe it's in the culture though. Back in the day teachers were respected and the relationship between teachers and students are somewhat closer to father-children than customer-merchant. Now time changes but old habits stay hard.

FactualOrion• 3 years ago

What do you mean by educating the doctors about medicines? I've seen people without degrees advocate for themselves as patients and be more knowledgeable about certain medications and treatments than doctors or nurses were.

hnthrowaway03282• 3 years ago

Like "you don't know as much as I do, I Googled a page and it says..." type. No they definitely know very little about the stuffs. And I personally heard and saw they tried to persuade OTHER patients, of different symptoms to listen to their "theories".

Workaccount22• 3 years ago

This is something that always kind of gets me, because when I have something wrong with me I will do a deep dive on it and read whatever studies, publications, and fellow patient accounts I can find.

This often puts me in a position where I feel pretty confident that I have more fresh knowledge about it than the doctor before me. Someone who in all likely hood has a mild familiarity with it, and is filling in the blanks with rote medical intuition.

What I really wish is that I could get doctors to drop the veil, and just openly admit what there depth of knowledge on the illness is. I would love it if they googled stuff right in front of me. I absolutely do not expect them to have a full medical encyclopedia in their head, and I am smart enough to be able to not be put off by "I don't know/I'm not sure".

SkyMarshal2• 3 years ago

Indeed, given the PE-driven profit motive of most (US) hospitals these days, people are increasingly less trusting of their diagnoses and prognoses, and more likely to take an active role in their own care and treatment.

HPsquared• 3 years ago

I think with teachers and professors it can be a case of "habit".

They're used to being the smartest person in the room and telling the people around them what's what. This can sometimes rub off on interactions outside work. Same kind of thing can happen with other professions, of course.

wolverine8765• 3 years ago

Do you disagree with the OP, that it's the people with intermediate knowledge that are most overconfident?

xamuel• 3 years ago

I feel like this has been part of the latent background memetic knowledgebase for many years now: https://knowyourmeme.com/memes/iq-bell-curve-midwit/photos/

threeseed2• 3 years ago

I can imagine it's not just restricted to scientific knowledge.

And having met many of these types of people they all seem to share one trait: definitiveness.

Everything is black/white and there is always a "right" answer or approach where as people with little knowledge and those who are experts tend to be nuanced and flexible.

karaterobot• 3 years ago

I'm 100% guilty of this. I try not to be, but hey.

There's different levels:

1. The guy who reads the title of the journal article.

2. The guy who reads the abstract of the journal article.

3. The guy who reads the text of the article.

4. The guy who's read all the other significant research in the field and can put it in context alongside their own personal experience as a practitioner.

I'm #2, and I strive to be #3. It's really hard to be #4, especially for more than one domain.

not_enoch_wise4• 3 years ago

That’s why I skip even reading the title, and get right to mocking other commenters.

jrflowers3• 3 years ago

A true Posting Elemental needs no context to ply their trade

quickthrower25• 3 years ago

#2.5 someone who lets Karpathy explain it all :-)

antisthenes• 3 years ago

#3 is pretty orthogonal.

There's a reason abstracts exist, and they usually provide enough information as to whether the paper discovered any significant findings, so whether it's worth reading at all.

If you are #1, #2 and in #2 you include reading not just the paper but the meta-analysis papers of the field, that already puts you ahead of 95% of the general public.

Not even scientists themselves are full #3 people. It's just impossible, considering the amount of work that exists in the field, and considering that most studies just confirm existing findings from 10-20-50 years ago.

riccardomc• 3 years ago

You claim you're a #2.5 which is intermediate. So according to the article you might very well be an overconfident #1, instead...

You should probably add a "5. don't know" option and check that one...

jrflowers3• 3 years ago

#3.5: The guy who skims the text of the article with just enough effort and time to be proven correct about their opinion of the article’s title

This is an incredibly common guy

naniwaduni• 3 years ago

0.5. The guy who reads the press release for the journal article!

(0.2. The guy who reads the title of the news article for the journal article on HN and proceeds straight to the comment section...)

moffkalast2• 3 years ago

0.1. Guy who reads the edited HN title and makes an offhand joke in the comments without even reading any other comments.

gus_massa• 3 years ago

I read the title, skim the abstract and jump directly to the tables/graphics. There is a lot of bullshit in the introduction and conclusions.

shostack• 3 years ago

I'm not a scientist, I'm in digital media and ads, but I've been doing this for almost 20 years and feel confident in my level of technical and business knowledge of the space.

I never realized how wrong papers could be until I read some from my industry and dug into their methodology of which I'm sufficiently knowledgeable to spot issues. I don't recall specifics as it was a while ago but I do recall the district impression I was left with which amounted to "holy crap, I wouldn't let this person pull reports on my ad account let alone run campaigns" due to the buy side 101 mistakes they made.

Since then I read all scientific papers with a huge heaping of salt.

ramblenode3• 3 years ago

2.5. The guy who reads the abstract then skips to the figures.

gumby• 3 years ago

I think this is also a root of science denialism (e.g. anti-vaxxers). I have some antivax family, and the root of their opinion is that the scientists changed their advice as the pandemic developed!

My theory is that in primary and high schools, science is taught as a series of immutable "facts". You get tested on remembering them, and when you do an "experiment" you get marked down if your reading of the litmus paper isn't 4. No sense of doubt, and no exposure to the epistemic boundaries to controleld trials, measurements, etc.

If your takeaway is that scientists uncover and explain immutable facts, hearing them say one thing in January and another in June can in fact shake your beliefs in these scientists.

enjeyw• 3 years ago

In highschool I was lucky enough to join a program run by the CSIRO (Australia's Government funded scientific research body), where students would assist scientists on experiments they were conducting.

I was tasked with measuring the water resistance of a proposed environmentally friendly paper coating.

I asked my supervising scientist what numbers I should expect to get back from the experiment. I distinctly remember my surprise and thrill when he looked at me and said "I have no idea, that's why you're running the experiment".

It was such a new concept to me! A scientific experiment where the correct answer wasn't written in a teacher's lesson guide, or even known to anyone!

dboreham• 3 years ago

Doubtful it was anything so logical. Humans like to believe magic, and are very easy to manipulate.

bratgpttamer3• 3 years ago

This has been my experience, too, as well as people who understood The Science perfectly and that the authorities were just taking a best guess early on, which set them further against any kind of mandate. (Mostly older, conservative physicians with an "it'll be fine" attitude)

The Science didn't change that much (viz: masks in healthcare since forever), but the flip-flopping with messages ("don't horde masks; they won't save you!", "wait, no - everyone needs a mask!"), which was really less about The Science than about different concerns (mask effectiveness, availability) were just the "I told you so" they were looking for.

CrazyPyroLinux3• 3 years ago

scientists changed their advice as the pandemic developed

"Changing advice" is to be expected, but it was the outright lying, as proven by FOIA'd emails, which understandably shook faith.

ignorethefacts2• 3 years ago

That's a bad take.

The science mouthpieces (aka media and government speaking on behalf of scientists) were insisting they knew the immutable facts and any questions or doubt in those immutable facts were heretical. Anyone who has an appreciation of the evolving nature of scientific research should immediately see the red flags in that.

"Antivax" is not a position of anti-science, it's a slanderous label for an alternative theory with its own compelling evidence.

CrazyPyroLinux3• 3 years ago

Especially when dishonestly applied to so many people who had all their other shots and maybe some extras, but just weren't on board with this one brand new one.

cassac2• 3 years ago

Intermediate [insert topic here] knowledge associated with overconfidence and negative attitudes.

Topics: Science, Sports, Cooking, Driving, Programming… literally anything.

I didn’t need a study to know this.

quickthrower25• 3 years ago

It this the B swimming lane thing again?

cassac2• 3 years ago

I do not know the reference. Is it that people claim the middle lanes are faster?

But I do know that a lot of people across a lot of topics over estimate their ability because they don’t know what they don’t know. Science as a topic isn’t special in this regard.

Jr devs are great examples. Your code is garbage and I am brilliant, just let me change this thing here and OH MY THE SYSTEM IS BROKE PLEASE HELP ME! Intermediate knowledge and negative attitudes.

not_enoch_wise4• 3 years ago

Exactly why I developed advanced sci knowledge, so my narcissistic confidence and anti-social attitude can be defended!

bratgpttamer3• 3 years ago

We find a nonlinear relationship between knowledge and confidence, with overconfidence (the confidence gap) peaking at intermediate levels of actual scientific knowledge.

Dunning-Kruger isn't real.

mcpackieh3• 3 years ago

Is that a mantra?

Jensson3• 3 years ago

It is the same result, just that they added the "unskilled" category. People who never drove a car will rank themselves as the worst drivers, and people who are experts rank themselves highly, so the most overconfident are those who can drive, but drives poorly, ie "intermediate skilled" relative the whole population.

p0w3n3d• 3 years ago

I've met both overconfident PhD holders and intermediate sci knowledge "flat-earthers" (conspiracy theory propagationists). Both are really hard to talk to, and both one couldn't convince they are wrong. I observed both being wrong. And meanwhile some of conspiracy theories became true...

It's really hard to by sure of anything these days

gobdovan2• 3 years ago

Could you provide examples of conspiracies you knew were wrong and became true?

scotty79• 3 years ago

Interesting. I think I've seen a research that when it came to covid, both uneducated and highly educated were making the wrong decisions more often. Sweet spot was around master's degree. Those people knew enough to know that they don't know enough to wing it themselves and deferred to expert opinion. Both under and overeducated thought they know better.

pickingdinner• 3 years ago

Or maybe those overconfident with negative attitudes resort to intermediate scientific knowledge?

And aren't there humble optimists with intermediate sci knowledge?

Maybe there just aren't enough humble optimists?

dav_Oz• 3 years ago

Once the activation energy can be maintained (basic skill and consistency) the learning curve is the steepest at the beginning of any given field; the setbacks are still rare and surmountable (and just mostly add to the confidence). But the human mind is only able to suffer a finite amount of beating and at one point your perceived competence level will collapse dramatically, overcorrecting in the opposite direction.

However, the reward structure of overconfidence can be used to maintain the momentum and once you hit the hard wall of insurmountable incompetence congratulate yourself that you actually walked the path of knowing practically nothing.

It's the hardest thing to fully admit one's own ignorance and easy to see all around you, it's only when the giant rationalization machine buzzing inside is finally fully exhausted; for a brief moment the crushing vastness of the unknown pours in.

So in a way overconfidence is just unused fuel. Use it wisely.

gobdovan2• 3 years ago

Maybe that's why all the major materials on climate change I find are sociological, focusing on people's perception on it and on agreement between experts rather than the actual subject -

It's so I don't get intermediate knowledge and become skeptical to my own dismay.

1letterunixname2• 3 years ago

This shall be henceforth known as "The Dunning-Kruger paper". ;)

Certainly, overconfidence and negative attitudes cloud public debate and popularization of conclusions. I've unfortunately come across narcissistic CEO/VC types who believe they have "all" of the answers, when instead they may have a corner of the Rosetta stone or a piece of lint.

STEM knowledge must be tempered with data and supported conclusions over non-experimental biases. The larger issues are behavioral pathologies when people appear to believe they have a superior special monopoly on knowledge, experience, or being. Another pathology is anti-intellectualism, which runs deep in America.

slackfan• 3 years ago

May we all remember that the we can only know that we know nothing.

frankreyes• 3 years ago

Dunning Kruger Effect, one more time.

mimd• 3 years ago

Well, nature apparently won't publish complex articles on topics for reasons, thus causing these issues of "intermediate" science knowledge. I wonder what was cut out to get this paper through.

Intermediate sci knowledge associated with overconfidence and negative attitudes

Related Stories