Grit Has a Negligible Effect on Success Compared to Intelligence

https://journals.sagepub.com/doi/abs/10.1177/1948550620920531

by cofree • 6 years ago

64 104 6 years ago

journals.sagepub.com

That's hilarious. Extremely counter to the prevailing narrative of how hard work and grit get one to the top.

gridlockd4• 6 years ago

I'd say that's still better than the other fake narrative: The game is rigged, the cards are stacked against you, they're holding you down, don't even try.

fwip6• 6 years ago

Is it? For the question "what predicts your pay", your "grit" beta coefficient is only 0.09, compared to a sum of .55 determined by your gender, background, and intelligence.

That is, raising your "grit level" by 1 standard deviation will only increase your pay by 0.09 standard deviations, when compared to your age-matched peers. Somebody only one standard-deviation down on each of the other factors would have to be 6 standard-deviations higher on grittiness in order to get the same pay. That's the difference between the 0.1th percentile of grittiness and the 99.9th percentile.

The data shows that increasing your grit enough to compensate for even small levels of adversity is impractical.

lambdatronics• 6 years ago

Here's a non-paywall copy: https://www.gwern.net/docs/iq/2020-zissman.pdf

D-Coder• 6 years ago

"...a major concern about the studies which argued that grit has a strong impact on success is that they were based on selected samples such as West Point cadets (Duckworth et al., 2007), employees in technology companies (Jachimowiczet al., 2018) or students in inner city schools (Eskreis-Winkleret al., 2014). These samples were drawn from institutions which are characterized by a homogeneous socioeconomic population (and perhaps even by population that are homogenous in their intelligence)—a characteristic which is likely to impose a range restriction on important predictors of success."

Ow!

bsanr26• 6 years ago

You mean, they... controlled variables?

Between those 3 cohorts, I see a phenomenon that cuts across a wide slice of the population.

sambroner• 6 years ago

Abstract says: "intelligence contributes 48–90 times more than grit to educational success and 13 times more to job-market success. Conscientiousness also contributes to success more than grit but only twice as much..."

Very challenging to evaluate without paying $40 to read the paper. They mention a difference in how they group the subjects (by intelligence). My gut take (often wrong!) is that they're grouping the subjects in some way that pushes part of my understanding of grit into intelligence.

perl4ever4• 6 years ago

Seems to me like intelligence and drive ("grit") are multiplicative factors. Saying either is more important sounds wrong because if either is zero, you have nothing. And if intelligence is fixed, then working harder is the only way to improve your chances. From that perspective, "grit" is the only thing that matters.

nsl732• 6 years ago

There’s inclusive evidence that intelligence is fixed, and there’s evidence that people that believe intelligence is fixed have worse outcomes.

I think it’s very likely environmental factors and personal mindset have considerable impacts both positive and negative on intelligence. I also suspect in the next decade or two research will come out on pharmaceuticals that show improved mental performance. There’s already the beginnings of this.

zepto4• 6 years ago

Firstly, neither is ever zero. That would only ever be true in a dead person.

It makes perfect sense to say that one factor is more important than another if it has more effect on the outcome.

Intelligence is not fixed. It can certainly be decreased by long term stress and environmental toxins etc.

It can also be enabled or inhibited by situational factors such as access to information, available time, etc.

Working ‘harder’ is not well defined. Do you mean going faster at doing the same things, or spending more time working? If not, can you define it?

Working harder or faster on something that is systematically unproductive will only make ones situation worse.

Intelligence is the capacity to select valuable activities in which to invest one’s energy.

Therefore grit doesn’t matter at all in the absence of intelligence, or can even be a negative.

marcinzm6• 6 years ago

I linked to the paper is another comment. They used the Armed Forces Qualifying Test (AFQT) for intelligence.

The AFQT score in the NLSY is the sum of the standardized scores of four tests: arithmetic reasoning, paragraph comprehension, word knowledge, and mathematics knowledge, and is expressed as a percentile score out of the general population

To my non-expert perception this does seem like a test that benefits from studying and general schoolwork (ie: reading books).

They used the The Short Grit Scale to determine grit (http://www.sjdm.org/dmidi/files/Grit-8-item.pdf) which seems like it biases toward un-creativity and "OCD completionism."

newen• 6 years ago

There’s a trick you can do on the internet that involves the words sci and hub and doi.

kortilla• 6 years ago

TLDR; grit helps, but only when in the same range of intelligence as your peers.

This isn’t particularly shocking. There aren’t a lot of high paying jobs for dumb people and nepotism doesn’t scale enough to make up for that.

anongraddebt• 6 years ago

This. There isn't much more to take away from the paper.

As an aside, there seems to be some sort of snowball effect that might confound a naive interpretation of the role of intelligence. For example, getting a high SAT score (and AP scores) is likelier to lead to Ivy admission. Ivy admission is likely to lead to above average training, as well as more influential social networks. This can lead to interviews with above average firms... etc. You get the picture.

Now, intelligence played a role in the high test scores and admission to an elite undergrad, and still plays a role throughout the journey of a successful (whatever the definition) individual. However, there are many feeback loops and amplifying/compounding first and second order effects as well. Quantifying the effect of grit, intelligence, etc seems very difficult.

bjornsing• 6 years ago

Nepotism doesn’t, but I’m starting to think that other forms of corruption can, under the right political circumstances. For example credentialism in combination with friendship corruption is potentially powerful. I think we’re starting to see significant effects of that in Sweden for example.

bsanr26• 6 years ago

Comparing two ill-defined high-level cognitive traits to determine which will more greatly effect success in a unmeritocratic society and extrapolating the results to general humanity: what could possibly go wrong?

The measure of intelligence was taken from the 1999 wave, the measure of grit was taken from the 2013 wave, and the measures of suc- cess from the 2015 wave. Thus, intelligence was measured when the participants were aged 15–19, grit was measured when the participants were aged 29–34, and success was mea- sured when they were aged 31–36.

Jesus.

rowanG0774• 6 years ago

American society is many things but it is not a low merit society. If you are significantly better then someone else at something you will, in general, reap the rewards.

bsanr26• 6 years ago

I'm appalled that this passes muster for you as a rational statement, knowing all we do about how parental wealth and education, professional network size and health, and even residential location can effect career outcomes in even technical fields. I thought we put this canard to rest.

dgb232• 6 years ago

That depends on what „something“ is and how it is rewarded in comparison to something else. And how you value types of rewards.

In my personal, political opinion it seems that being good at convincing others and being greedy gets rewarded way too much over creating direct value.

marcinzm6• 6 years ago

This seems to be the full paper for those noting the paywall:

https://www.gwern.net/docs/iq/2020-zissman.pdf

enkid4• 6 years ago

I think the most interesting thing in this paper is the fact that grit seems to be extremely important in certain circumstances, and much less important in others. It makes sense that getting through a challenge like going through West Point is going to require a sense of the importance of your goals. Most of the time, though, we're not in those circumstances and other factors are more important. Grit, as described, could even lead to a lack of flexibility in some circumstances, which could inhibit success.

marcinzm6• 6 years ago

I agree, looking at the questions they seem to penalize changing your goal which I view as a vital part of success. Work hard towards your goal but stop if you realize your goal is wrong. Then work hard at whatever you consider the better goal.

tsumnia2• 6 years ago

I'm still reading through the paper myself, but that was my first impression reading the Introduction. In heterogeneous groups, intelligence beats grit as an indicator for success. However, if you then consider Duckworth used homogeneous groups, you can argue that grit is what separates you from similar candidates.

fwip6• 6 years ago

The paper is behind a paywall and not open-access, so here are some pieces of information I thought were interesting:

"Intelligence" was measured by an individual's performance on the "Armed Forces Qualifying Test" (AFQT).

"Grit" is self-reported on the "Short Grit Scale" which asks participants to rate themselves from 1 to 5 on statements like "I am diligent."

"Success" was measured by level of education attained and hourly wage rate in 2015.

Control variables include: Gender, household income as a child and level of education by parents. Study participants were born in the early 1980's.

Top-level findings:

Although in our representative sample of the American population grit does explain educational and job-market success over and beyond other predictor variables such as socioeconomic status, intelligence, and the Big Five personality characteristics, its absolute effects are rather minimal. In particular, its effects are negligible in comparison to the effect of intelligence. Finally, the effects of grit were also considerably weaker than the effects of conscientiousness.

puranjay• 6 years ago

Grit" is self-reported on the "Short Grit Scale" which asks participants to rate themselves from 1 to 5 on statements like "I am diligent."

This is a pretty awful way to measure grit and makes me doubt the results

tsumnia2• 6 years ago

This is an issue with most self-report surveys.

gwern• 6 years ago

But the measurement of Conscientiousness was much worse, and it still outpredicted Grit considerably:

"Given the NLSY 2-item measure of conscientiousness and the 8-item measure of grit, this comparison is likely to overestimate the effect of grit relative to conscientiousness (Crede et al., 2012 for the consequences of using short measures of the Big Five personality traits)."

Plus, going from an 8-item test to a longer one isn't going to close a gap with IQ that large, and as they point out, the reliability is decent enough and pro-Grit authors have no trouble making claims based on, so what's sauce for the goose is sauce for the gander:

"The reliability of the scale in our data was .72, which is comparable to the reliability reported by Duckworth and Quinn (2009) and within the ranges reported in the meta-analysis of Crede et al. (2017)."

thyrsus• 6 years ago

Thank you for this. What is the difference between their measure of grit and their measure of conscientiousness?

fwip6• 6 years ago

Conscientiousness was measured by the "The Ten-Item Personality Inventory (TIPI)"

a 10-item measure of the Big Five (or five-factor model) dimensions of personality: openness, conscientiousness, extroversion, agreeableness, and emotional stability

iandanforth• 6 years ago

Standard warnings: Social Psychology is not a science and should probably be ignored. "Grit" and "Intelligence" are so poorly and inconsistently defined as to be meaningless. This is controversy for controversy's sake

programmarchy5• 6 years ago

Not true but typical belief of STEM types. If you go by reproducibility rates you could say the same about medicine and economics. Hell, gravity is poorly and inconsistently defined, but we don’t throw away physics.

enkid4• 6 years ago

I'm pretty sure gravity experiments are reproduced very frequently.

mtgp1000• 6 years ago

Perhaps it is because "STEM types" understand the difference in rigor that mathematics affords hard sciences. That's not an opinion.

boxed2• 6 years ago

Gravity is extremely clearly defined. And yes, we do bash on economics quite a bit. We bemoan medicine, but we don't make fun of it for having reproduction rates in the low single digits. Because it doesn't.

WA2• 6 years ago

about medicine

Partially true.

and economics

Yes.

So what?

marcinzm6• 6 years ago

Economics is not STEM and many people who complain about psychology would also say it lacks rigor (especially the social rather than theoretical/mathematical side).

raincom• 6 years ago

Well, 'gravity' is a technical term defined within a theory--be it Newtonian or Einsteinian. This definition is not coming from the common sense, but the other way around: technical definitions percolate down to the commonsense.

The way the phenomenon of gravity is tested: by deriving (predicting) consequences with auxiliary hypotheses(or theories), then do empirical tests and compare.

That's not the case in social sciences: they pick an explanation(explanans) that only explains what is already picked(explananda).

Just pick up some facts, say, X, Y and Z. And postulate A to explain X, Y and Z. Then do all mumbo jumbo: statistics, surveys, questionnaire. That's not what theories in physics do: they explain/predict something(not from the set X,Y,Z) that is NOT already there.

Larry Laudan in his philosophy of science makes a distinction between confirming instances and positive instances. Social scientists just pick up positive instances(X,Y,Z), Natural sciences are on the look out for confirming instances(instances that are not of the type X, Y and Z).

Take a crude example from our ordinary experience: all swans are white. Go and look out for a white swan. That's a positive instance, not a confirming instance.

lioeters• 6 years ago

Many of the social sciences, including economics and psychology, do not meet the criteria to be considered a science (at least not a rigorous one).

What Separates Science from Non-Science? - https://www.realclearscience.com/articles/2012/05/30/what_se...

osazuwa• 6 years ago

A few points for context:

1. Those R squared numbers in table 2 are quite low. The statistical analysis itself is painfully basic. I'm a statistician, and I've a bookshelf full of excellent books written by statisticians who publish in psychology, so they should have been able to find someone to beef up the analysis a bit. That said, psych publications is also notorious for bad data analysis. I'm an outsider to this field, but this feels like an omission.

2. The paper doesn't use any causal inference techniques despite making a clear causal claim in the title. They use simple linear regression, which is not a causal model unless causal assumptions are made explicit. In my book this is a grave sin, one avoids making causal assumptions that might be hard to defend, fits a model with linear correlation, then makes causal conclusions. To understand what I mean by causal model, read this (these techniques are well-known and if the authors' defense is that they don't know them, it reflects badly on them): https://en.wikipedia.org/wiki/Rubin_causal_model

3. It is worth thinking about what the intelligence assessment, the AFQT test, actually quantifies. One critique of AFQT is that it is more of a measure of literacy than intelligence. Read more here: https://en.wikipedia.org/wiki/Armed_Services_Vocational_Apti.... Maybe the title should be "Having parents that make sure you can read by age five has a greater effect on success than grit."

4. The AFQT and other intelligence tests have a pesky problem of having averages that change in time. This suggests that much of what they measure is not innate, but learned.

5. Social psychology is plagued by a reproducibility crisis. I'm sorry to say, their papers, no matter how provocative the findings, have to be viewed with great skepticism. https://www.psychologytoday.com/us/basics/replication-crisis

6. The paper says social economic background has a significant effect, but about the same effect on pay as intelligence. That seems to conflict with a pile of research that says that social economic background is the strongest predictor of success. To say that IQ is just as important as your parents' wealth is pretty eyebrow-raising.

aidenn03• 6 years ago

I have a friend who got a PhD in psychology. She had me look over her analysis and it was bad. The two "highlights" were:

1. Her advisor told her to use a student's t-test for multivariate data.

2. When the P value insufficiently low to reject the null hypothesis, he walked her through p-hacking the data.

BrandoElFollito• 6 years ago

I had the same thing with a friend who was getting her PhD in nutrition.

She was aware that the statistics were not strong, but they were horrendous.

After helping her to get at least a bit back on track I developped a strong skepticism to anything published without a rewiew of the stat methods. Which is almost everything one can read in the press.

spzb• 6 years ago

I'll take your word for it on the stats - it never was my strong suit. But having read Duckworth's "seminal" book on Grit it's also incredibly unscientific, frequently mistaking survivorship bias for causation.

boxed2• 6 years ago

The hope is that grit can be developed and nurtured in children, while we're pretty sure intelligence is genetic to a super high degree.

sdinsn• 6 years ago

while we're pretty sure intelligence is genetic to a super high degree.

No? There are tons of studies about childhood development that focus on intelligence being affected by their environment / upbringing.

Konohamaru7• 6 years ago

That 10% that comes from grit matters more than the 90% that comes from intelligence, because it's the only aspect that reflects the sovereignty one has over oneself and the ability to define oneself in spite of physical reality (supernatural). The 90% is just an inheritance.

betenoire4• 6 years ago

Matters more to whom? I'll take your inheritance if you are offering.

Konohamaru7• 6 years ago

Why would you choose what's natural (inheritance, physical development, parent's DNA) over what's supernatural (your willpower)?

beisner2• 6 years ago

Any evidence that grit isn’t also a heritable trait?

Konohamaru7• 6 years ago

The above argument doesn't hold if free will doesn't exist. But then I couldn't choose to believe that I have free will if free will doesn't exist, so why bother arguing?

aSplash0fDerp2• 6 years ago

Using the term "tools" loosely, fork and spoon comparisons digest better than apples and oranges.

Eating soup with a fork is still a success with enough intelligence.

Pfffft. Spoon-fed geniuses raised with only forks was a cruel joke.

Let me guess... For an encore, they'll do a study that says AI is more successful.

A suckers iterated ever minute on the Internet with a path/information like that.

aSplash0fDerp2• 6 years ago

I hope the downvote was for being incomplete.

A fork, spoon and a knife are required tools for most meaningful job/professional successes.

It may even mirror the mind, body and soul tools needed to achieve personal successes. 2 outta 3 sucks. 1 outta 3 is worse..

m3kw9• 6 years ago

There is grit before and after a certain point on a “trajectory” that matters. And intelligence may get you far, maybe grit will help you complete the last mile. So no I don’t think you can definitely say what the study says

gridlockd4• 6 years ago

https://www.youtube.com/watch?v=n3Mn6V1IzHw

RickJWagner• 6 years ago

Henry Ford would disagree. Does today's economy make some difference?

fmajid• 6 years ago

Neither intelligence nor grit hold a candle to luck and having chosen the right parents.

candybar• 6 years ago

From the study:

The Short Grit Scale (Duckworth & Quinn, 2009) was used as a measure of grit and was administered to most of the NLSY participants in 2013 (6,476 of the original sample of 8,984 in 1997). It consists of 8 items in total, 4 items for each grit facet—perseverance and consistency of interest. Sample items are “I am diligent” (perseverance) and “new ideas and projects sometimes distract me from previous ones” (consistency of interest). Answers are given on a scale from 1 (very much like me) to 5 (not like me at all).

This is like measuring intelligence by asking people whether they identify with statements like "I am a fast learner." Most people aren't very self-aware. I often find that people that work less hard (in general across a wide range of things) often self-identify as having more grit because people work hardest when they work on something they are motivated to do, but being able to work superficially in the absence of motivation for long periods of time makes people feel they have more grit.

My suspicion is that grit (in the sense that matters, not whatever the grit researchers have been pushing) is mostly indistinguishable from intelligence as it's typically measured. I'd be shocked if most intelligence tests don't already do a better job of determining grit (in terms of the level of perseverance you could expect from someone) than asking someone if they are diligent.

naveen99• 6 years ago

As Sam Altman says, think more about what you want to do. Moving fast in a wrong direction isn’t going to help.

mcphage• 6 years ago

My suspicion is that grit (in the sense that matters, not whatever the grit researchers have been pushing) is mostly indistinguishable from intelligence as it's typically measured.

I don’t know about that. I’m pretty intelligent, but don’t have much grit at all.

jaylittle• 6 years ago

TLDR: Stupid is as stupid does.

Grit Has a Negligible Effect on Success Compared to Intelligence

Related Stories