Epistemic learned helplessness (2013)

http://squid314.livejournal.com/350090.html

by ikeboy • 11 years ago

130 56 11 years ago

squid314.livejournal.com

aharrison• 11 years ago

That is an excellent essay and it scares the crap out of me, because I see it play out every day in both directions and it often does feel exactly like epistemic learned helplessness.

There are so many biases that affect individuals (hindsight, hyperbolic discounting, confirmation, etc) that to get correct answers, you have to look at and trust other accounts. You have to do studies, you have to trust those studies, and you have to track actual hard evidence.

But large enough systems become increasingly opaque, and it can be hard to test your web of trust empirically. This reduces huge organizations to cargo cult behavior, or worse, fundamentally unsound behavior because "it worked for me."

I don't know what to do about this besides what CFAR is trying, which is to get people better at weighing evidence and changing their damned minds. You have to constantly be asking yourself: "what would the world look like if this were true? What would the world like like if it wasn't?" It is hard work, but I think it is critical to our continued improvements as a civilization and a species.

CarVac• 11 years ago

I don't know if the right answer is to be generally more open to changing your mind.

There are far too many things to be worried about: the danger of AI, religion, the environment, other political views, the results of scientific papers in dozens of fields... epistemic learned helplessness is a defence against wasting literally all of your time taking every argument seriously.

At the end, the author notes that we should be glad for the specialists who are well-versed in certain subjects enough to actually be able to evaluate and experiment on outlandish theories and unusual study results.

Maybe we need to emphasize more specialization? Or is it working alright already?

tbrownaw• 11 years ago

The other solution is to not really care all that much about most things.

There's some new physics result about how black holes interact with the quantum foam? I can't build anything practical with it, so beyond "that's interesting" it doesn't really matter to me.

Some food is suddenly found to cause cancer in rats? Unless it's fairly new, the effect on humans can't be all that strong or we'd have noticed by now. Something else will probably kill me first (like whatever the medical term for "old age" is these days).

Someone's claiming that some particular aspect of modern diet causes obesity? It would have to either make you feel lethargic (lower calories out) or make you hungrier (raise calories in). Both of which are directly observable and easily correlated to the contents of recent meals, without having to argue over mechanisms and confounders and personalized gut bacteria and such.

There's a new higher estimate for the percent of scientific results that are fake or poorly done or just statistical noise? This does help for knowing how not-seriously to take everything, but beyond "that's too damn high" the exact numbers aren't all that relevant (unless you're testing a fix).

Someone wants money to research AI risk? Are they looking at how to build safe AI, or how to prevent anyone either carelessly or deliberately building unsafe AI? Only the second is worth looking into further.

n0us• 11 years ago

Some of the longest papers I've ever written were in support of conclusions that I don't even believe in. In my opinion they ended up being some of my best papers as well. The point wasn't to write in support of what I believe in or what I want to write about but what I think I can present the strongest argument for.

danteembermage• 11 years ago

If I find myself being convinced by the argument, does that mean I should adopt Epistemic learned helplessness in response or not adopt Epistemic learned helplessness in response?

I'm partly being facetious, but it would be interesting to try to use a non-argumentative approach to persuade to use one, I'm just not exactly sure what that would look like.

As a kid I remember being told "brush your teeth in circles, it's better" and thinking "I'm sure something else will be recommended in ten years so I'm just going to go back and forth horizontally like I want to" and sure enough circles clean more plaque but push your gums up so downward flicks were recommended. Maybe a dentist can weigh in on current tooth brushing practice... That said, was I better off with my inferior method? That's kind of the crux of it. If we're blown about by every plausible theory, is that better than being blown about by nothing? It seems like this is a nested Bayesian decision problem that needs to incorporate switching costs, which I'd guess for some things are trivial and for other things are quite large.

bpchaps• 11 years ago

I've been thinking lately that "do the best that your intuition can" is the optimal way to solve these sorts of mini tasks. That way, when your intuition is right about 10,000 things in your life, you have a bit of leeway with the other 10,000 it's wrong about.

I'd imagine that this really only works as long as your intuition is in constant check, though.

shanusmagnus• 11 years ago

I like this heuristic, since for things for which it is effective, intuition basically _is_ rationality, on a species-level timescale. The problem is the things for which it is not effective. The other problem is knowing which kind of thing you're dealing with, since determining that requires intuition.

sageikosa• 11 years ago

I brush with circles using more force on the downstroke for the top teeth or upstroke for the bottom teeth.

bgilroy26• 11 years ago

My teeth are lucky I brush them.

If they are holding out hope for more skillful brushing, tant pis pour eux!

greendestiny2• 11 years ago

I've thought down these paths a lot, and I came to similar but different conclusion. It's not simply that I don't trust argument - I weight arguments with evidence.

If you take the simulation argument for instance - if you're contemplating the probability that we are living in a lifelike simulation you must factor in the number of lifelike simulations you have seen.

mtdewcmu3• 11 years ago

> you must factor in the number of lifelike simulations you have seen

Right. I take the position that there are problems that are unsolvable by technology; simulating the universe seems likely to be in that category. That position is fundamentally unprovable, but call it a hunch. Also, Occam's Razor argues against it, FWIW.

chipsy• 11 years ago

It depends on what you want out of life, I think.

If you try to rapidly see and comprehend an ultimate truth head-on, you pay the price of only being able to think about and express such truths in forms which look insane to outsiders(terrorism, fascism, conspiracy theory, etc.) and with correspondingly drastic consequences to the well-being of you and others.

But if you want to live a healthy, ordinary life, you shade yourself from some of those truths, knowingly or not. You express them cryptically and meditate on them in a deliberately obfuscated way, or dismiss them for the moment. You do not allow that knowledge to throw you around. That doesn't mean that you don't act on the knowledge at all(which is how epistemic learned helplessness is presented here), so much as it does that you can proceed to find paths to acting on it that are indirect and do not trip your insanity alarms.

Culture itself demonstrates this property. It changes rapidly enough, but it does so gradually, most of the time. Reality in the year 1985 meant something recognizable yet quite a bit different - in food, fashion, and music, in cultural attitudes, politics, and should-you-says, and in everyday usage of time and activities. Some of the things that were taken seriously then are dismissed now, and some of the things that were laughable then are taken very seriously now. And in the 30 years that transpired, many of the same people are still around, but nearly all have changed their status and outlook in some way. Many have led reasonably content lives while doing so. Isn't that amazing?

noonespecial2• 11 years ago

Interesting that you still accepted without question that brushing at all was important. You didn't simply say "well then, who can know?" and quit brushing altogether.

No comment on that really, just hmmmmm.

mml• 11 years ago

This guy sounds absolutely insufferable.

jdjdirn2• 11 years ago

He sounds like a guy Id love to have a beer with. We might just be able to understand each other :)

dropit_sphere• 11 years ago

Honestly, he's the best.

thuuuomas• 11 years ago

"Rationality" is the new "atheism".

All I got from this article:

"Sometimes, it's best not to commit to a position when you don't have all the facts (or if 'the facts' are essentially unknowable). But it depends!"

OP has a lot of ten-dollar words, tho.

So, what'd I miss?

bsder2• 11 years ago

The problem is that you shouldn't intrinsically believe arguments, at all.

You should believe facts. You should believe logical inference from facts.

You should believe arguments insofar as they corroborate or refute facts.

And, if an argument can't be tested, it's not a very good argument.

analog31• 11 years ago

Indeed, a good argument, and a logical inference from facts, should be one and the same thing.

jdjdirn2• 11 years ago

Lol. And what precisely is a fact, but an argument about the state of the world being a specific way?

bsder2• 11 years ago

And what precisely is a fact, but an argument about the state of the world being a specific way?

Thankfully not. A "fact" is something that can be measured in such a way that even if you are totally antagonistic to my arguments, we will get the same result.

Now your "argument" may attempt to prove the irrelevancy or inadequacy of my facts, but the facts themselves should be unfudgeable.

illivah2• 11 years ago

Here is the difference:

If you try to replicate a fact, you'll succeed If you try to disprove a fact, you'll fail If you find rebuttles to a fact, they'll be hollow

The only thing in the obvious list that doesn't separate a fact from a non-fact is the arguments in support. Hence why scientists are taught not to care for arguments in support, only research, and then specifically arguments that disprove more than those that prove.

john_butts• 11 years ago

Not buying it.

JesperRavn2• 11 years ago

I think the author makes some good points, but I think the term "learned helplessness" together with the arguments made, imply that if reasoned arguments can't be trusted then nothing can. But I think that a better characterization is that people use other ways of thinking, such as intuition, emotion, etc. in addition to reason.

Any expert in a field other than pure math, will tell you that in order to evaluate an argument or evidence, you need experience and intuition, not just argumentation. Arguments are not a formal process that can definitively arrive at conclusions. The same applies to mathematical models. Economists often complain in private about the need to dress up their ideas as mathematical models, even when the model adds nothing to the discussion. This is especially annoying in empirical work, when no one is actually interested in the model, just the empirical results.

paulsutter4• 11 years ago

given that at least some of those arguments are wrong and all seemed practically proven, I am obviously just gullible

Not gullible, he just needs to read Kahneman's "Thinking Fast and Slow".

The intuitive mind is unable to distinguish between consistency and truth. That is, a consistent story will always seem "proven" to the intuitive mind.

"Seeming practically proven" means little. You need to use the more logical part of your mind to crosscheck it.

ThrustVectoring• 11 years ago

I'd be very surprised if he hasn't read Kahneman. And even if he has, knowing about cognitive biases is a completely different skill than avoiding them. It's basically why the Center for Applied Rationality exists.

paulsutter4• 11 years ago

Well if he read it, he certainly didn't understand it.

When I was young I used to read pseudohistory books; Immanuel Velikovsky's Ages in Chaos is a good example of the best this genre has to offer. I read it and it seemed so obviously correct, so perfect,

This guy believes in PROVING PSEUDOHISTORY. It's hard to understand what that even means, but let's look at specifics:

"Noah's Flood couldn't as a cultural memory...

- of the fall of Atlantis

- of a change in the Earth's orbit

- of a lost Ice Age civilization or

- of megatsunamis from a meteor strike."

A generous person could give these a 1% chance of being right. Maybe a 5% chance if you had a very convincing argument.

A gullible person might give one a 30% chance of being true.

But it is utter nonsense to assign 100% probability to any of these (that's what proof means, it means 100% likely). These just aren't provable matters.

Circling back to Kahneman, it really seems that he's getting persuaded by intuitive arguments, and the book "Thinking Fast and Slow" really dives into how this happens.

afarrell2• 11 years ago

He is now a practicing outpatient psychiatrist, so I suspect he is somewhat familiar with Kahneman.

Confusion• 11 years ago

You underestimate the author significantly. Read it again, but this time asssume the author is extremely smart and wise and is truthfully relating experiences. Then draw conclusions about what this tells you about the general trustworthiness of 'arguments'.

illivah2• 11 years ago

Poor guy. He reminds me of Leshrac from Dota 2. Lots of deep philosophy, leads to nihilism.

Here's one of the multiple big problems. Learning that arguments are often not valid is an amazing insight! However, it says nothing about what what you SHOULD believe, relative to the information you have.

The wise choice is NOT to believe based on your ore-conceptions (ie ignoring the stronger arguments). Better arguments are better than worse arguments. mediocre evidence is better than better arguments. And better evidence is better than worse evidence. And since better evidence is not a guarantee, we measure it by the efforts to disprove them.

So, lets take an example: the conspiracies for the world trade center attack. First we have the official story which we more or less know. Well, some people know at least. Then we have the video Short Change, which goes through a couple hours describing all the points where the official story doesn't make sense. It talks about physics, and background deals, and molten this and that. A lot of interesting stuff. It raised a lot of questions. It didn't provide detailed or reliable evidence, so more research was needed. So i watched the rebuttle video, which was twice as long. They provided evidence, with citations, from experts in their respective fields.

The part that was convincing wasn't "oh, this argument is right, this one is wrong". The part that is convincing is that I can now say "well, according to so and so, this is explained, and so is this, and so is this, and so is this, and so is this" down the line. Now, in this case there was basically nothing left at the end of the argument, but even if there was then I consider the sources. One source says xyz is still unexplained, but that source is now known for making things up and/or not knowing what they're talking about. The other source cites dozens of experts with detailed explanations, photo evidence, clear walkthroughs.

It's not "oh, this argument is better" anymore. Now it's "well, this one literally has no reason to believe it, but this one has a great deal of support."

quietplatypus• 11 years ago

Guy makes a shitload of basic philosophical errors, like most rationality training people. They are all centered around conflating logic with reality and ignring computational costs. If you cant connect a theory with reality that is rooted in specific, observable, actionable things, without requiring strong AI with a O(1) time complexity of synthesizing respinses, the theory is a nonstarter. If he was more aware of this, he would never get close to buying into claptrap like pascal's wager or mugger. So what if you believe in god or not? What does doing that or not actually get you?

It's amazing how many of these "rationalists" are using the concept as an emotional band aid against their resentment and other uncomfortable feelings.

And its not about compartmentalization. Ideas vary in their usefulness or actionableness. Sure, it takes instinct to distinguish them. But that is what "rationalists" are scared to death of, there being things like instinct and unconscious processing that, honed right, is often much superior to their laborious, droning but logical attempts to count every outcome, but requires faith in oneself.

It's really just a huge inferiority complex that they have. They are so insecure that they need universe-circumscribing systems of Bayesian logic to do anything. The truly rstional thingg to do is to put down the fucking rationality handbook, stop running the fucking expected value computations, go outside, live life, dare for the world to correct you, and get stronger or die trying. Thats it.

noonespecial2• 11 years ago

Feels a bit like "anecdotes are not data"; "arguments are not evidence" perhaps?

thom• 11 years ago

"Well, you can prove anything with facts, can't you?"

https://www.youtube.com/watch?v=4n-UGQcG3Jw

JesperRavn2• 11 years ago

Well that's a good example of what the original argument was talking about since the comedian's argument was terrible (e.g. by the same logic we should not say that slavery is immoral, because the ancient Greeks practiced slavery).

nabla9• 11 years ago

This is related to things that people like Robert Axelrod have been studying. It's innovation vs imitation dilemma in social learning and evolutionary game theory.

Science. 2010 Apr 9;328(5975):208-13. doi: 10.1126/science.1184719. Why copy others? Insights from the social learning strategies tournament. http://www.ncbi.nlm.nih.gov/pubmed/20378813

Related xkcd http://xkcd.com/1170/

fa• 11 years ago

Like all good philosophy, this idea has a long history. I encountered it in Taleb's Black Swan as Pyrrhonian skepticism, which flourished in Greece as the Aristotelian Academy. Very down to earth and actionable advice on living and doing. Many Pyrrhonians were doctors and that colored their philosophical views—at a time when philosophy was taken very seriously and meant more than it does today (see it as a precursor of today's science).

Kenji• 11 years ago

This is the correct Bayesian action, by the way. If I know that a false argument sounds just as convincing as a true argument, argument convincingness provides no evidence either way, and I should ignore it and stick with my prior.

Wrong. The correct scientific methodology would be to store the new conflicting theories in your head while being unsure whether or not they are true (withholding judgement); later on, if new evidence sheds light on the issue, you can discard or keep it. The silliest thing to do is what this guy advises: Persistence on arbitrary facts because they were first. That's how religion works. Despicable.

I don't like how this guy treats "high school dropout" as an example for someone particularly stupid and naive. It sounds elitist and smug and detracts from the main point of the text. It reminds me of lesswrong.com

I've heard a few good arguments in this direction before, things like how engineering trains you to have a very black-and-white right-or-wrong view of the world based on a few simple formulae

Oh no, nothing is further from the truth. In engineering, you have to do a lot of subtle design decisions that aren't black-and-white at all. Difficult tradeoffs like cache invalidation, where there's no right answer in general.

All in all, a subpar article that goes on way too long and could be shortened to a couple of lines.

tgb• 11 years ago

The author is now blogging at: http://slatestarcodex.com/ It's really one of the most consistently interesting blogs I read. Check out the top-posts section to see if anything catches your eye: http://slatestarcodex.com/top-posts/

TeMPOraL4• 11 years ago

It's one of my favourites too.

There've been a few lists of his best posts created by readers, e.g. [0] and [1].

Personally, my absolute favourite is Meditations on Moloch [2]. It's long, but has high concept density and highlights the very important causes of problems in the world that many people miss entirely when discussing important issues.

[0] http://nothingismere.com/2015/09/12/library-of-scott-alexand...

[1] http://lesswrong.com/lw/mmg/yvains_most_important_articles/

[2] http://slatestarcodex.com/2014/07/30/meditations-on-moloch/

mbubb2• 11 years ago

Great read and interesting discussion. Reminder of how good HN has been over the years.

The idea of "institutional learned helplessness" is a useful one. There is a French term which I will mangle instead of google: 'Deformation professionelle' - built in blind spots which come from having T shaped expertise...

I've been part of a hiring team for a sysops person and found this of interest:

http://www.heavybit.com/library/video/2015-02-24-charity-maj...

Overall the talk is ok - the point which stuck with me was the idea that you should look for a sense of institutional helplessness and filter for it when you hire Ops folks.

The speaker also said to look for folks with 'strong opinions which are lightly held' - a double edged sword.

In a devops role there is an internal debate. Should I press point "x" ahead on principle? Or should i hold off and wait on the business/product requirements. Thinking slow and waiting on a full discussion of requirements before charging forth into battle can be seen as good process or passivity...

At work we wre having a discussion on where and how to involve QA processes in the build/deploy process. Devops took the stance - "this is a business decision - we will implement it as product sees fit" and we got push back for being too passive.

But this is a trivial example. I guess the bigger issues which inspire a "What can i do about it?" response. Recent readings on surveillance state and corporate interests (ie Bruce Schneier, etc) make me vacillate between deep, helpless feelings we are FUBAR to learning about constructive things like gpg, tor, etc

ArkyBeagle2• 11 years ago

On "Should I press point 'x'", do a divide and conquer on it. Try to reduce the number of degrees of freedom from that decision point to a minimum.

IMO, you have to live with the ambiguity, but try to craft strategies that cover both with minimal ... perturbation. Waiting helplessly is not an option :) If all else fails, strawman the least cost solution first.

I also think there's value in having some measure of dissent from within the team. Done right, you'll synthesize solutions you would not otherwise have come up with. But you want this to be without rancor and without sinking large amounts of time.

On "... where and how to involve QA processes in the build/deploy process" - make a complete list of steps, then balance which gets included as the risk profile emerges. Good risk analysis includes a comprehensive and complete plan for what happens when a step fails.

Finally, you simply have to manage perception. Quantify, quantify, quantify. As if you don't have enough to do....

mbubb2• 11 years ago

It is interesting that Pascal gets cited in this. With Pascal's Wager there is no leap really. It is very internal idea - an individual's relationship with God. "I might as well believe, there is no risk if I do so. If I am right there is a great reward. If I am wrong the result is what it would be anyway."

This is what I believe is a distinction between atheism and agnosticism. Belief vs a pragmatic ethic.

Pascal is identifying a throughly modern disconnect between what we privately might think and believe and how we act in public. His idea is radical because that split would probably not been conceived in the same way before. Over time this idea would become more socially grounded. Weber in "The Protestant Ethich and Spirit of Capitalism" would put forth a similar idea in terms of 'visible forms of God's grace'. Thus I am more blessed because there is an Audi A8 in my driveway and you are driving a Jetta...

Another perhaps more relevant case is Kierkegaard in "Fear and Trembling". I am probably misremembering political threory classes from 20+ years ago but the idea I retained is:

Think of Abraham and Isaac. In a nutshell Isaac is the much loved (and only) son and God tells Abraham to sacrifice Isaac. He takes his son up to the mountaintop and builds an altar. At the last second an animal is substituted. (Hmmm just occured to me that Miyazaki could do an interesting treatment of this)

The key idea is Abraham's resignation - which is different from passivity. He is willing to follow through. He hears God's voice telling him to do something batshit crazy. Because it is God he is willing to follow it through to its logical end. He has belief and resignation to God's will is the first step of that belief.

Now to the outside world Abraham is the worst kind of criminal. You would imagine him thrown in a dreadful prison and the guards looking the other way while that staple of tv dramas - prison justice is meted out. That doesn't change.

And Abraham doesn't even know if he is hearing God or if he is just crazy. And no one else knows either.

Back a decade or so ago when Intel's 'Trusted Computing Platform' was cause for debate, I remember someone saying "Just because Richard Stallman is paranoid doesn't mean that Microsoft is not after you" which I thought was hilarious and true and adopted it as my email footer for a while...

When I think of Stallman - I have enormous respect for the man, for his beliefs and his courage in following out his convictions. In my everyday life I use a Mac and install nonfree packages on my Linux servers so my actions are not consistent with that respect. It is too hard for me to browse the web in emacs and I enjoy twitter and the various inane 'Distractions from distraction by distraction' that are the internet.

I think Stallman is more right than wrong. I also believe in a Kantian imperative sense that we would all be better if we aligned our ideas of freedom and use of technology with his ideas. Without his ideas and his actions which broght thse ideas to life, we would be in a very different situation.

But I am much more 'pascalian' in my everyday life choices and actions.

ArkyBeagle2• 11 years ago

But if it is indeed Stallman v. Gates ( which I doubt ) then Gates won the big war, while Stallman continues to win small skirmishes.

This is just all too convenient and narrative for my tastes. IOW, "It's complicated." That's my epistemic learned helplessness.

Epistemic learned helplessness (2013)

Related Stories