Berkson's Paradox

http://en.wikipedia.org/wiki/Berkson%27s_paradox

by xtacy • 12 years ago

39 20 12 years ago

en.wikipedia.org

sp332• 12 years ago

In this notation: P(A|B,C) what does the comma mean?

tredontho• 12 years ago

I think it's the intersection, so, "and".

P(A|B,C) is "Probability of A, given both B and C".

spacehome2• 12 years ago

That's true, may be a slightly confusing way to interpret it, given that C = A∪B. In context I think it's better to read it as P(A|B), in the case that C occurs.

stepstep• 12 years ago

Yeah it's tricky because in most programming languages, '|' has precedence over ',', but in math sometimes it's the opposite. That tripped me up too.

Chinjut• 12 years ago

In this context, the comma is as good as &.

btilly5• 12 years ago

Implicit population selection biases can lead to all sorts of fun.

One that I saw a paper get wrong a while ago was one which claimed to find an inverse correlation between two different traits that individually improved intelligence. The catch? The sample population was chosen from university students at no name school X. But people who did well on both traits would have done well enough to go to a better school and so were underrepresented in the population sample!

(I forget the paper, but pay attention and you'll find lots of other examples...)

gwern• 12 years ago

Speaking of intelligence and Berkson's paradox, it looks like Berkson's paradox may be responsible in a very similar way for why the personality trait Conscientiousness seems to correlate negatively in some studies with intelligence - the studies were using selected samples while more representative population samples show the expected independence: "How are conscientiousness and cognitive ability related to one another? A re-examination of the intelligence compensation hypothesis", Murray et al 2014 (https://pdf.yt/d/Dfl1N6pbR-4vYaKk ; excerpts: https://plus.google.com/103530621949492999968/posts/aQ51UnLC... )

Tarrosion• 12 years ago

This reminds me of a thought experiment I read a while ago, I believe on the Atlantic. I'll paraphrase with a bit more math:

Suppose that acting ability and attractiveness are independently normally distributed with mean 0 and standard deviation 1. Further suppose that to be a successful movie star, an individual must have an acting ability plus attractiveness of 6.

Then among people with the necessary attributes to be movie stars, attractiveness and acting ability will be negatively correlated. In this example, we might expect to see movie stars with (intelligence, attractiveness) around (3,3) or (2,4) or (4,2), but it's much more unlikely that we see many people around (4,4).

btilly5• 12 years ago

For a more thought-provoking variation, suppose that intelligence and test taking ability are independent, and IQ is the sum of the two. What, then, does your IQ score say about your intelligence?

See http://bentilly.blogspot.com/2010/02/what-is-intelligence.ht... for some of my thoughts on that from a few years back.

Retric3• 12 years ago

That's an unlikely assumption.

A better case might be for students at big name university's where GPA (proxy work etc) and intelligence (proxy SAT score etc) need to be over some threshold for admittance.

If that's the case bankers and others that want both a big name school and high GPA are actually negatively selecting for intellect. Which may account for a lot of fairly dumb behavior at banking institutions etc. As they might have a lot of people that can talk about statistics without actually understanding it.

gohrt2• 12 years ago

This is exactly the phenomenon described in the OP.

Berkson's Paradox

Related Stories