Your Scientific Reasoning Is More Flawed Than You Think

http://www.scientificamerican.com/article.cfm?id=your-scientific-reasoning-more-flawed-than-you-think

by technology • 14 years ago

48 25 14 years ago

taeric• 14 years ago

This seems to me to be as much "you have a hard time replacing what you know works in some (indeed many) cases with something that might be harder to apply in simple cases but is always correct" as it is "you have a hard time learning correct things."

That is, the ideas (that this piece mentioned) that people had a hard time leaving behind seemed to be those that work as decent guidelines, even if they are not accurate. Speed of processing is a value in our minds as much as it is in a computer. If you have a rule of thumb that works rather well, why abandon it?

jerf• 14 years ago

Sounds like a System 1 vs. System 2 conflict: http://www.amazon.com/Thinking-Fast-Slow-Daniel-Kahneman/dp/... And not even a particularly interesting or surprising one, really.

SoftwareMaven• 14 years ago

Can you go into more detail? Specifically, how does this differ from the second page if the article, where it discusses (somewhat refutes) learned concepts that override naive concepts taking longer just because the pathways aren't as strongly connected?

I have no idea what System 1 vs System 2 thinking is, much less how it applies in this case.

andyjohnson0• 14 years ago

http://en.wikipedia.org/wiki/Dual_process_theory#Systems

mbesto• 14 years ago

Absolutely phenomenal book! Highly recommend reading.

pygy_• 14 years ago

AFAIK, system two is not immune to cruft...

shasta• 14 years ago

Surely the "inconsistent" questions are the ones that initially seem to be possibly tricky or subtle questions. This is not the same as contradicting the intuitive answer. For example, if you ask whether two masses fall at the same speed in air, that would probably trigger a careful response in the same way that asking whether two objects fall at the same speed in a vacuum would, even though in the first case the correct answer ultimately matches natural intuition. That's my guess as to what's going on.

yread• 14 years ago

the authors found that participants who had best mastered scientific concepts (determined by their overall accuracy) were especially slow to verify inconsistent statements

I would expect that it's the other way around ie the people who take longer time to think stuff through will have a higher accuracy. Which doesn't make their results all that surprising

EzGraphs• 14 years ago

Worse than simple ignorance, naïve ideas about science lead people to make bad decisions with confidence.

This is the reason these articles capture my attention.

A related problem not discussed in the article is the tendency to use so-called "scientific" explanations to a non-scientific area of life. A better understanding of the scientific method itself in education is essential. Simply prefacing a statement with phrases like "it has been scientifically proven that..." or "statistics show..." is used to lend a vague scientific credibility various claims. It shows up in a slightly more sophisticated form in political diatribe and in educational settings as well.

Although of course there might be a flaw in my scientific reasoning about the article ;).

beagle3• 14 years ago

Quite a bit of the accepted "scientific" knowledge is wrong, and a lot of people will fight for it vigorously without noticing, or otherwise thinking. Even here on hacker news, and other scientifically minded forums, I've been called a quack for pointing out that:

- the "calories in - calories out" (and especially the carb=4kc/g, protein=4kc/g, alcohol=7kc/g, fat=9kc/g and nothing else matters) is fragile, based on a chain of unproven assumptions, and has countless counterexamples (in other words, scientifically wrong, even if it is somewhat useful for ~80% of the population).

- dietary cholesterol was NEVER shown to be correlated with anything bad. serum cholesterol was shown to be correlated with heart attacks and higher probability of other events, but dietary cholesterol and serum cholesterol are almost uncorrelated.

- sodium intake is correlated with high blood pressure only for 20% of the population. For 80% of the population, sodium intake does not raise blood pressure.

- you need water, but if you are healthy, you can skip food for 40 days with many benefits and no irreversible damage.

- you do not need B12 in your food if you are healthy (and your intestinal cultures are healthy), and animals can't make it any better than humans can, so this argument for requiring meat consumption is utterly wrong.

- red, yellow and blue do not constitute a "primary color" basis with respect to addition - you can't even make green! (red green and blue almost do; no 3 colors can cover the whole human eye visible gamut, but specific red. green and blue maximize this coverage).

I've had a chance to talk to Danny Shechtman (nobel laureate in chemistry, awarded for discovering and documenting the quasi-crystal structure). It turns out that this "make bad decisions with confidence" is not limited to "naive ideas and simple ignorance" - the greatest minds of the century, first among them Linus Pauling, refused to consider his discoveries. (In fact, the definition of what a crystal is was revised to account for quasicrystals, but this was done only after Pauling passed away). There are horror stories behind the nobels given to McClintock ("jumping genes"), Warren (helicobacter pilori is the immediate cause for stomach ulcers), Shechtman, and I would assume many other great discoveries.

nirvana• 14 years ago

There's a great deal of examples of this in economics, and this is not just a theoretical problem-- it causes bad electoral outcomes.

For instance, the idea that "government spending stimulates the economy" is a naive concept. It sounds good- right, because they are spending money, that means it is creating demand and the goods and services they are buying increases economic activity.

The scientific reality is, this naive concept is ignoring the cost of that government spending. All of the money government has comes from two sources- inflation and taxes. Whichever way they raise the money, they do economic damage.

Thus government spending, like Obama's so-called "stimulus" plan, actually hurts the economy.

This is why, for instance, the unemployment rate ended up being higher than Obama claimed it would be if his plan wasn't passed.. even though his plan was passed.

But it is not very hard to find people who believe that some other thing caused unemployment to be higher.

In fact, both the Republican and Demcorat parties, and their partisan's ideologies, reject the science of economics and embrace pseudo-science.

In my lifetime, I've seen a great increase in embracing pseudo-scientific concepts or even anti-science positions, most recently and alarmingly, by people who insist that they are right because "science" agrees with them.

Another example: Glaciers are getting smaller because of global warming. This belief is completely unscientific-- there is no way to know how many glaciers there are on earth, let alone whether they are getting smaller, and nobody has even tried to guess whether more of them are shrinking than growing.

Another example: The idea that the TSA protects us against airplane hijackings. Or that somehow the government is protecting us against terrorism. Or that mass shootings would be worse if guns were less regulated (the stats: 9.2 is the average number killed in areas where guns are banned, but only 2.2 people die in areas where the intended victims are allowed to be armed.)

Politics' natural enemy is science.

zalew• 14 years ago

> Politics' natural enemy is science.

I'd rather say science's natural enemy is politics.

josephlord• 14 years ago

Hmmm, a very political comment...

Firstly confusing Economics with a science rather than pseudo-science (or at least very bad science in that in most cases the assumptions required for the mathematical models are unrealistic and instantly forgotten), then stating as a fact some simple controversial (among economists too) statements which if true at all have a great deal more subtlety behind them and a number of which I believe to be flat out wrong.

I think trying to tackle the points would go way off topic but in my view the parent post gives a good example of some naive beliefs (although economics is insufficiently scientific to be able to prove them inconsistent with reality).

Edit: Changed 'post' to 'comment' and 'unrealistically' to 'unrealistic'.

ppod• 14 years ago

>reject the science of economics and embrace pseudo-science.

You seem to be ignoring the fact that many professional practitioners of the "science" of economics disagree with your analysis. You can't just say that the guys who agree with you are good economists, and those who don't are bad economists.

seanmcdirmid• 14 years ago

Stimulus is more about wealth redistribution in times of duress, when income equality is seriously out of whack. The only problem today is that the money is coming from low interest loans from China vs. heavy taxes on the rich (as during the Great Depression).

The flipside of stimulus spending is supply-side economics (or Voodoo economics according to the elder Bush), where cutting taxes magically increases revenue since there is more economy to tax. In reality, they basically drive up inequality, though either position can be useful in moderation and both are incredibly damaging when used too much.

Neither technique is based on scientific theory, rather they are more like strategies or heuristics to be used in managing the economy that are based more on experience and perhaps ideology. Economics is not a hard science, its a social science. So is forecasting their effects, or forecasting future economic behavior; without stimulus unemployment might have been worse, meaning Obama could have been more optimistic about the future outlook of the economy at that time.

Steuard• 14 years ago

As far as I can tell, nothing that you've written here is at all relevant to the content of the linked article. (It's not a generic "people don't pay enough attention to science" article, though I suppose the headline could be misread that way.)

Also, economics is far, far, far from being a science. Does it have important things to teach us? Absolutely. But calling it "science" just muddies the waters about what science actually is.

yummyfajitas• 14 years ago

Your post would be much better if you avoided hot-button topics and focused on political examples that don't inspire strong emotional reactions. There are plenty of these - farm subsidies, a lot of occupational licensing, patent/copyright regimes, for example.

http://lesswrong.com/lw/gw/politics_is_the_mindkiller/

hazov• 14 years ago

The "market" that economists say about is their model of what they believe a market is, it's a mathematical model with a good amount of gaps, this is why you will always find economists disagreeing among them, it's not like physics because they have to decide to rely on more assumptions and less experimentation.

I used to work in earth sciences although not in glaciology, and their models are very sound to say about the now, not so much about the future, that glaciers retreats and expands is common knowledge (the Missoula flooding event is a good example), but I am sure with a warming planet, if that's the case, more glaciers would be retreating than expanding, this is easier to search for, but I know the World Glacier Monitoring Service have good data about that.

igouy• 14 years ago

>>... there is no way to know how many glaciers there are on earth ...<<

Because...?

brudgers• 14 years ago

The reason people persist with unscientific conceptions of the world even after learning the scientifically correct explanation is because scientifically correct explanations often have little benefit in our day to day lives.

The heliocentric model of the solar system, for example, has no effect on our normal lives, and thus by Pierce's Pragmatic Maxim is of little value.

http://en.wikipedia.org/wiki/Pragmatic_maxim

K2h• 14 years ago

I'm going to bet they used something like harvards implicit to run this test. If you have never seen it, the results are surprising given how simple the tests are (measure your time of response)

https://implicit.harvard.edu/implicit/

the warning statement alone should make you interested:

  I am aware of the possibility of encountering interpretations 
  of my IAT test performance with which I may not 
  agree. Knowing this, I wish to proceed

jodoherty• 14 years ago

Given a multiple choice True/False test with the statement “people turn food into energy”, I would have had to spend some time deciding on what to answer, not because I have to question whether or not the statement is true, but because I'd have to wonder in whether or not the test writer understands the nuanced implications of that statement.

The way I see it, people don't turn food into energy -- they simply move energy around and transform it by breaking the chemical bonds in the food and forming other chemical bonds using enzymes to keep the activation energies low enough to make a net gain despite losses to heat. So based on the conservation of energy, people don't turn anything into energy. The energy was already there -- they simply transformed it.

But then with the conservation of mass-energy, we know that some mass is converted to energy when chemical bonds are broken. So technically the statement is true in a sense. But the amount of energy created from mass is so small it can't even be measured or detected, so for all practical purposes, the statement is false. You have to be breaking nuclear bonds before you see that.

The article's author says the statement is true, using it as an example of a statement that's consistent with our preconceived ideas, but instead of explaining it, he simply does some pseudo-scientific hand-waving -- digestion, respiration, metabolism, and all that. This suggests the author somehow considers energy as something that is created and used up on a regular basis.

Which makes you wonder, what if the writer of the statement was an expert biologist whose major focus isn't understanding the concept of energy or the chemical underpinnings of microbiology? What if they also saw energy in the same way as the author of the article? What if the answer on the answer key is true, despite being arguably false? In that case, the answer is true in the sense that that's what the writer of the test wanted you to answer.

I think a better test would give the option of adding a statement or two to qualify or explain your choice on the test. I know that would allow me to finish such a test sooner, especially if it was computerized and I could type my explanations.

ThomPete2• 14 years ago

Anyone interested traveling even further down the rabbit hole I recommend Kuhn, Lakatos & Feyrabend.

technology• 14 years ago

You mean this ?

http://philosophyfaculty.ucsd.edu/faculty/wuthrich/teaching/...

ThomPete2• 14 years ago

Yes

Your Scientific Reasoning Is More Flawed Than You Think

Related Stories