Report on the role of standardized test scores in undergraduate admissions [pdf]

https://home.dartmouth.edu/sites/home/files/2024-02/sat-undergrad-admissions.pdf

by msravi • 2 years ago

252 404 2 years ago

home.dartmouth.edu

sokoloff4• 2 years ago

Shocking that a “Scholastic Aptitude Test” or “American College Test” would predict future scholastic achievement at the collegiate level.

NegativeK• 2 years ago

Those are marketing terms, though.

It's not trivial to go from performance on a single test to performance throughout the year, which will include long-form writing, lab work, and other forms of learning.

tomtheelder• 2 years ago

SAT/ACT for high achieving students means months of prep/studying and practice test taking. It ends up being a ~somewhat~ similar experience to a college class. Not really similar to humanities classes that depend on discussion section and papers, but if you squint enough it kind of looks like some STEM classes.

grdgyredb2• 2 years ago

You are right it’s not trivial, but on the other hand if you simply try to make such a test you’d do better than completely uncorrelated pretty easily I’d suppose.

bigbillheck• 2 years ago

You can't wish nominative determinism into being.

ryukoposting4• 2 years ago

I know. Being good at taking tests in high school is a positive indicator for being good at taking tests in college? A staggering discovery.

zeroCalories6• 2 years ago

I don't know if you're trying to make a point about test taking, but you're aware that college GPA is composed of more than just being a good test taker? Better way of putting it might be: someone that scores highly on an IQ proxy is good at everything.

doctorwho423• 2 years ago

I would argue that it is even more so in college, where tests make up the bulk of your grade.

magicalist2• 2 years ago

Yes, as a USA patriot, this is why I've always supported the USA PATRIOT Act. I don't get why anyone would bother looking into it.

Scubabear684• 2 years ago

I know. Sarcasm aside though, there is a growing movement of HS and elementary school teachers looking to remove all KPIs from schools. It is very worrying that this protected class of workers are arguing they are special compared to every other problem in the world and their performance and those who they are teaching should never be measured.

asdff7• 2 years ago

Except that the test is pretty clearly perverted. You take two kids, same education, give one an after school testing program their parents pay money for, and they score better. Is the wealthier kid more worthy of receiving a college education? What does that say about how we educate our society then, and what we are actually valuing and giving privilege?

closeparen2• 2 years ago

They score very slightly better, not really meaningfully.

https://slate.com/technology/2019/04/sat-prep-courses-do-the...

hirvi746• 2 years ago

Is the wealthier kid more worthy of receiving a college education?

Not to mention, what about us test-disabled people?

I didn't know I had ADHD and its "friends" when I took the ACT, and I could have potentially benefited greatly from the reasonable accommodations the test allows for. However, it's not like every child is required to have some kind of psychological evaluation while in school.

I find all of this standardization in testing to be nothing more than Goodhart's Law in full swing.

Just what we need, more "Teaching to the Test" and route memorization in our education system...

resters3• 2 years ago

this says more (good stuff) about Dartmouth than it does about the standardized tests. If scoring higher on something that is close to an IQ test doesn't predict higher academic performance, what does that say about the coursework?

abeppu2• 2 years ago

... is it close to an IQ test? Like, people study very specifically for these standardized tests, and success is also related to how much prep work you put in. I think that may also align with how people study for exams once they're enrolled students, how much work they put into projects/papers/etc.

resters3• 2 years ago

True. Having the will to prepare for one test (such as the SAT) might indicate a will to prepare for other tests, such as those considered for academic performance ratings.

zeroCalories6• 2 years ago

Then you just normalize the scores based on socioeconomic indicators like is done in the article. Of course, this highly advantages well groomed students from bad demographics, and horribly disadvantages neglected students from good demographics. That's why China worked hard to ban test prep.

But either way, it's a pretty good proxy for IQ.

paxys3• 2 years ago

The fact that SAT scores are directly proportional to how much time, effort and money you have spent in preparation should tell you that it is very different from a general IQ test.

resters3• 2 years ago

Do we know that general IQ tests are not also something that a person can improve on with practice?

doctorpangloss2• 2 years ago

They are proportional, yes, just 4x less so than having academically exploitable talents.

PaulHoule• 2 years ago

From time to time you see a press release about a Hollywood star who just got an "IQ Test" quoting a score which is the maximum for

https://en.wikipedia.org/wiki/Raven%27s_Progressive_Matrices

Given that that test has a limited number of questions and the nature of the test it seems that anyone who is good at remembering lines could benefit from coaching. In fact they've come out with versions of that test which are different from the standard test because the standard test is too well known.

My perception is that there is a psychologist who administers the test and another psychologist who administers test prep, but a lot of people hang onto the idea that Alyssa Milano must have really gotten a perfect score on Raven since everyone successful must be really smart, the system is fair, etc.

bigstrat2003• 2 years ago

I didn't spend a dime on ACT prep. I barely spent any time on it. I scored 33/36. It's perfectly possible to score well on these tests even if you don't spend money/effort on preparing.

bobobob420• 2 years ago

Smart people are smart.

SnooSux• 2 years ago

To measure academic achievement they use First Year GPA. It makes sense they're correlated, both require studying known material for a test. But is GPA the best measure, especially first year? I would be interested in other metrics like 3rd-4th year GPA or placements into jobs and such.

abcc8• 2 years ago

I didn't read the entire document in depth, but perhaps it has been shown elsewhere that first year GPA is a good quality predictor of GPA in subsequent years.

old_bayes2• 2 years ago

I believe they do look at other things (EDIT: or at least the study this paper was based on did):

"Using detailed admissions data from IvyPlus institutions, Chetty, Deming, and Friedman (2023) show that SAT and ACT scores also predict career success, including high levels of earnings and attendance at elite graduate schools, holding family income constant."

paxys3• 2 years ago

That is referring to another study. This particular one doesn't add anything in that regard, just looks at freshman year GPAs.

535188B17C93743• 2 years ago

I agree. Every other comment here seems to be like "well duh" and I'm... skeptical. My experience is that the ACT/SAT seem to be good indicators of getting good grades in well-defined spaces. But things like creativity, curiosity, work ethic are much better predictors of other kinds of success that frankly matter much more in the real world.

I know some really, really unintelligent people who got good grades in college. They just ate books.

wegfawefgawefg2• 2 years ago

its an IQ test. G proxy.

jvanderbot2• 2 years ago

Undergraduate GPA predicts lifetime earnings[1], incoming test scores and GPA are highly predictive of both advanced degrees (which increase earnings), and increased earnings within degrees [2], [3].

I suggest these effects are because being a good student aka "eating books" is correlated with conscientiousness. They show up to lectures, prepare, and test well.

And conscientiousness is very highly correlated with lifetime achievement, AND fufillment [4]. So measuring conscientiousness, and signalling high conscientiousness is a really good idea.

IQ is great, but conscientiousness is how you get things done [5]

1. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9004755/

1.b (edited) https://www.washingtonpost.com/news/wonk/wp/2014/05/20/heres... might be better. I thought this was "common knowledge"!

2. https://mpreiner.medium.com/what-is-the-impact-of-your-high-...

3. https://www.ssa.gov/policy/docs/research-summaries/education...

4. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3498890/

5. https://docs.iza.org/dp8235.pdf

zild3d• 2 years ago

things like creativity, curiosity, work ethic are much better predictors

of course but how would these be measured

derbOac• 2 years ago

Their Figure 1 is why we keep having these discussions in society. It's grossly misleading and not what a scatterplot for a 0.46 correlation looks like. I know what the figure is, it's just done in a way to overstate a case and ignore variability within bin.

If that figure were about anything else, people would be screaming bloody murder about misleading figures and overly generalized interpretation.

I'm in favor of allowing for the use of test scores but they get abused and the language in this report is a good example of how this happens. Scores have these real but modest correlations with real world situations, but then get used as rulers of atomic precision without any context or recognition of their massive limitations.

It makes the authors of this report look either deceiving or ignorant of statistics or both.

spamizbad2• 2 years ago

I believe there's some research that suggests your college GPA is not correlated strongly with job performance[1]. I suspect if SAT/ACT scores are strongly correlated with GPA, there's reason to suggest it may also not be strongly correlated with job performance (but I can't find anywhere that tests that).

I suppose this shouldn't be surprising. School does not train you to be a good office worker.

[1] https://www.linkedin.com/pulse/does-college-gpa-predict-job-...

boringg• 2 years ago

The goal of school IS NOT to make you a good office worker. The goal of school is to create the ability to think creatively, rationally and critically and make you a better citizen of society through those processes.

Job performance is subjectively measured by your bosses which has political implications. It is inherently a terrible metric.

thaumasiotes• 2 years ago

The reason people measure first year GPA is that, in the past, all freshmen at a college took the same set of classes. If you measure 3rd year GPA, you get confounded by the difference between physics students and French students.

Obviously, the worth of the metric goes down over time as first-year curricula differentiate from each other.

But is GPA the best measure, especially first year?

No. For example, SAT score is a better single measure than GPA is. But you can't use that to check the validity of SAT scores.

Jcampuzano2• 2 years ago

Contrary to what some may think: I would strongly suspect that particularly high scores on SAT/ACT would also correlate fairly strongly with well-paying job placements and long term GPA.

Note I am not saying that people who don't do particularly well on SAT/ACT can't also succeed on college and beyond, I had a pretty average score myself and consider myself fairly successful. But all the people I know who had the highest scores on SAT's were also the people most motivated to study and work extremely hard whether it be based on family pressure or just an inherent drive to be better than everyone else, and even to this day these people I knew who had scores in like the top 95+% are for the most part the most successful people I know primarily just due to an inherent drive to succeed at all costs.

Of course this will also depend on what metric you use for success. Also of course there will be people who do extremely poorly in traditional education but become wildly successful.

ryukoposting4• 2 years ago

There's some nuance to the "work ethic" factor. Work ethic makes a big difference in academic outcomes, but it depends on where that work is targeted.

Anecdotal evidence, part 1:

I grew up in the suburbs and went to public schools that, at the time, were good-not-great within the state. In that setting, I was a high achiever by any sensible metric. I was friends with a lot of the other "smart kids," and I definitely worked harder than some of them, but outside of school, I was also working on different things. My hobbies and extracurriculars weren't strictly academic, but they certainly set me up for academic success better than sports or video games would have.

Anecdotal evidence, part 2:

One of my classmates was "freak of nature" levels of gifted. I shared math classes with him for three years of high school, and to my knowledge he got one math test question wrong in that entire time period. He was also one of the school's best tennis players, and he made it into so many state concert bands/jazz bands/choir groups that their schedules overlapped and he couldn't do all of them. Last I checked, he was finishing up a PhD in neuroscience. But get this: he was our salutatorian, solely because someone else who did no sports and few clubs took more summer classes, and thus had the same GPA with more total credit hours.

waswaswas• 2 years ago

Later GPA is subject to distortion via less prepared or capable students switching into easier majors. Here's a paper indirectly showing the effect at Duke with a racial framing: https://izajole.springeropen.com/articles/10.1186/2193-8997-...

foobiekr3• 2 years ago

Why is this being downvoted?

bradley13• 2 years ago

The thing is: completely unsuitable students need to fail out in the first year. That's what "weeder" courses are for. They prevent students wasting time and money, only to fail out years later.

Which means: you don't have the same stats for 3rd and 4th year students.

nostrademons3• 2 years ago

SAT is well-correlated with 1st year GPA, but not well-correlated with eventual job placement (although it does correlate pretty well with eventual income). However, 1st year GPA is well-correlated with 2nd year GPA, which is well-correlated with 3rd-year GPA, which is well-correlated with 4th year GPA, which is well-correlated with eventual job placement, which is well-correlated with entry-level salary, which is well-correlated with mid-career salary.

A pretty useful model for life is that it's a series of contests, and doing well at the previous contest gives you an advantage for the next couple contests, but only the next couple contests. By the time you get to mid-career, nobody really cares what your high school GPA was. However, because each contest determines which set of subsequent contests you'll face, performance early on can have outsize effects on eventual life outcomes. You typically won't be applying for CEO jobs if you worked retail your whole life, unless you lie your ass off and bullshit convincingly to executive recruiters.

SoftTalker3• 2 years ago

In my own experience, first-year GPA in college was a cakewalk. I had straight As until Junior year, then things got a bit more demanding and I was caught off-guard because college had been pretty easy up to that point. (Large state university).

mp052• 2 years ago

If I were judged by my first year GPA I'd be homeless. Instead I'm really quite successful in spite of my first try (and miserable failure) at university.

When I went back and was paying for tuition out of my own pocket as opposed to magic sky money falling into my lap from the US Dept of Education, things were a LOT different.

projektfu2• 2 years ago

I think some of us (like me) were too immature at 18-22 to make good use of college. I'm not sure the amount of skin in the game would have made much difference, but I wouldn't have been able to pay for it myself, so that's a benefit. When I went back and got high grades, I also had manna-money from Direct loans. I just had a lot more life experience and maturity.

paxys3• 2 years ago

So, students entering Dartmouth with a higher SAT score also have higher first year GPAs. Which shouldn't be a surprise to anyone considering they are both basically testing the same thing.

The fact that over the last couple decades employers at large have stopped asking for college GPAs (and by extension SAT scores) on their job applications should instead tell you that all these numbers have very little relevance in the real world.

1980phipsi2• 2 years ago

They don't ask for SAT because they don't want to get sued.

lxe• 2 years ago

They should "un-bin" the statistics and identify individual outliers -- those who scored poorly on the tests but hold higher GPA, and vice versa, and learn things from these individual students.

HPsquared2• 2 years ago

That's the "work ethic" part.

hodgesrm• 2 years ago

Well that's the part that deserves some additional research.

jgwil2• 2 years ago

The headline should really be, "A test-optional policy is likely a barrier to Dartmouth identifying less-advantaged students who would succeed at Dartmouth." I think it's no surprise to anyone that test scores correlate with academic success, but it might seem counterintuitive that test-optional policies actually bias admission in favor of higher-income students.

lupusreal• 2 years ago

I really wonder about the people to whom that is counterintuitive. Don't they realize that rich privileged kids can do more extracurriculars?

doctorwho423• 2 years ago

Honestly, reading this makes me think "well yeah... Why is anyone surprised?"

Being a good test taker will 100% translate to doing well in college. Almost every course I took in college had over 80% of it's overall grade being tests, with a large number of them being closer to 90-95% for how much tests+final count.

So yeah, if you are good at taking tests and/or studying for them effectively... Yeah, you will do well in college. Especially the first year.

cowmix• 2 years ago

... but it still doesn't predict actual success in my experience. I was "gifted program" adjacent growing up (all my friends were in these programs in the late 70s -> 80s) -- all earned high grades and test scores -- but many are now puzzled why they are not showered with $$ and success into present day. Well, they don't think they are successful to where they should be.

doctorwho423• 2 years ago

Well yeah, our society doesn't really reward hard work and advanced knowledge for the many. There are the rare exceptions, but for the most part to make $$$ you need to buy into the current system which is all about making spreadsheet numbers go up YoY at the cost of everything else. If you don't do that, well I hope you can find a niche somewhere that is stable and comfortable.

ccleve• 2 years ago

Here's the money quote:

> Importantly, these test scores better position Admissions to identify high-achieving less-advantaged applicants and high-achieving applicants who attend high schools for which Dartmouth has less information to interpret the transcripts.

Precisely. SATs aren't about hurting the poor or disadvantaged. It's about giving them a chance.

doctorpangloss2• 2 years ago

If the universities were sincere about that, they would figure out how to increase enrollment.

Why is every parameter of admissions open to change despite tradition, except that one?

ZeWaka• 2 years ago

You can't enroll infinite people.

Avicebron• 2 years ago

I tend to think there is an unspoken fear that if it became more accessible it loses its prestige

dghlsakjg• 2 years ago

Dartmouth in 1950 was an all mens college that enrolled less than 2,500. Women weren't allowed until the 1970s.

Dartmouth today has 6,700 students of all races, genders, religions.

Enrollment numbers can and do change, but it is much harder and slower to change something that is physically and resource constrained than it is to change policy.

subpixel• 2 years ago

Elite institutions do not exist to give more people access to anything.

And where that is a goal, there is clearly a point where you have to draw a line based on capacity, student:teacher ratios, etc.

readthenotes1• 2 years ago

Why would we want more people taking student loans to get a questionable degree?

Trade schools make a lot more sense for most. Get a good job and you can self study from a position of safety...

BeetleB3• 2 years ago

Not sure if you're referring to Dartmouth or universities in general. For state universities, I think they do increase enrollment all the time. University of Michigan, for example, increased enrollment by 25% in the last decade.

What did you have in mind?

troupe6• 2 years ago

Harvard, Yale, etc. could set a particular standard and increase enrollment to accept everyone who met that standard and (if logistics allowed) that could possibly be a good thing. In some ways the massive online classes seem to be piloting this idea.

However, if you go spend some time teaching at a community college, I don't think you'll walk away with the idea that somehow getting all those students pushed into a university would improve things for the community college students or for the ability to educate kids at the university.

Increasing enrollment might be beneficial, but past a certain point it is going to be detrimental to education.

forrestthewoods• 2 years ago

I don’t know why this comment got downvoted.

It’s an objective truth that there are more elite students than there are positions in these schools. It’s also true that schools have absolutely exploded in administration size. They just haven’t hired more teachers.

Schools can’t double capcity overnight. But they currently aren’t making student body growth a goal. They prefer to keep it small and be extremely selective and exclusionary.

The flip side to this is that undergrad doesn’t matter. These elite schools don’t actually provide an elite education. They merely serve as a talent filter.

dpweb2• 2 years ago

Increasing enrollment means increasing faculty.

Maybe controversial but I really believe this lowers overall standards. You just can't find enough faculty to meet the high standards.

Look what happened when tech blew up - they hired like crazy. I'd argue the overall level of programming skill went down.

gnicholas18• 2 years ago

Dartmouth actually has a pretty unique program that allowed them to increase enrollment substantially. Basically, every sophomore stays around for the summer semester, but takes off either fall or spring to do an internship or something. The sophomore summer is a great bonding experience for the students, and the program allows the school to enroll significantly more students than they would be able to house otherwise. To my knowledge, no other similar school has a program like this.

hardware2win• 2 years ago

Indeed.

Standardized exams are the most fair and transparent.

Materials and tutors are available easily for free or cheaply on the internet nowadays.

webdood903• 2 years ago

Standardized exams are the most fair and transparent.

perhaps the _most_ fair right now, but certainly not what I would consider fair.

consider a middle class family that has secure food, housing, and transportation vs lower class family where the child might potentially have less consistent sleep, food, or has to work to help pay bills.

one has more distractions than the other.

msravi• 2 years ago

And this:

> Third, under test-optional policies, some less-advantaged students withhold test scores even in cases where providing the test score would be a significant positive signal to Admissions. Importantly, Dartmouth Admissions uses SAT scores within context; a score of 1400 for an applicant from a high school in a lower-income community with lower school-wide test scores is a more significant achievement than a score of 1400 for an applicant from a high school in a higher-income community with higher school-wide test scores. Admissions uses numerous detailed measures of outcomes at the high school and neighborhood levels to account for these known disadvantages. As one example, Admissions computes a measure of how each applicant performs on standardized tests relative to the aggregate score of all test-takers in their high school, using data available from the College Board.

elteto3• 2 years ago

Dartmouth Admissions uses SAT scores within context; a score of 1400 for an applicant from a high school in a lower-income community with lower school-wide test scores is a more significant achievement than a score of 1400 for an applicant from a high school in a higher-income community

And this is how you again discriminate based on race in a post-Affirmative Action world. The weight they assign to each context is completely arbitrary and tuned to achieve whatever objectives they have.

godelski• 2 years ago

This is a critical part. The problem is not using metrics (like SAT/ACT), the problem is using metrics blindly. That's the thing that always bugs me is when people just go on metrics alone as if they perfectly align with what's being attempted to be measured when metrics are only guides which choices must then be made from through careful evaluation.

gnicholas18• 2 years ago

Will processes like this result in families avoiding (or at least not seeking out as much) rigorous high schools? It seems like some families might choose to send their kid to a not-super-rigorous high school — where they can coast in class and learn via outside enrichment — instead of a school with more rigorous academics where they'll be held to a higher bar by colleges.

This would allow students to have a lower-stress life and have an advantage at college admissions — even if they actually learned less academically than they would have at the rigorous high school.

trgn2• 2 years ago

It's so weird to me that all this data exists, and that's it available and munged enough to be actionable. Do colleges really keep track of this for the tens of thousands of high schools, not only that, but all the income info, the mechanics of student placement for each school district, etc... it just seems so intensely convoluted.

fudhdjdbauu2• 2 years ago

At what point can we use genetics to weight scores as well? I’m having a hard time logically justifying class factors (being born poor is not something you can control) but not genetic factors (being born dumb is not something you can control).

Jun84• 2 years ago

> Importantly, Dartmouth Admissions uses SAT scores within context; a score of 1400 for an applicant from a high school in a lower-income community with lower school-wide test scores is a more significant achievement than a score of 1400 for an applicant from a high school in a higher-income community

Since I encountered a similar approach just three years ago in my son's High School application in the Chicago Public School System I can say this leaves a lot to be desired non low-info,e families. Here's a high-level summary of how it works:

1. Chicago is neighborhoods are assigned one of (if memory serves) four levels, which represent socioeconomic levels.

2. With the thought that the economic level is proportional to school quality (this is more true in most places in the US than other countries, since a big chunk of school budget comes from property taxes, so neighborhoods with low taxes get school with lower budgets) scores from the entrance exams for kids from different levels are boosted by different multipliers.

Now, of course, (2) is a travesty of the US but that's another long discussion. The problem with (1) is that the neighborhood to level assignments are not revised each year. So you can be affluent in a district that has gentrified in the past 2-5 years and your kid will get the low-income boost unfairly.

Additionally some neighborhoods are highly bimodal, eg take Lakeview. This is considered a low-income neighborhood but there are many high-income high rises and single family homes within its borders. Alternately, there are many rental condos in our neighborhood which is the highest income level, so these middle income kids get slammed.

Still another problem is families purposefully gaming the system. We had friends who changed their rental condo during their child's 8th year so they can get a lower-info,e address sin their application. This is widely known and done.

Perhaps a better approach would be to use the family's income rather than communities.

Edit: I had more time to research and found the following, in case somebody is interested:

* What I referred to as the four levels are actually called tiers

* The tiers are actually updated every year, so I was wrong above. Still, the community average criticism stands. See the discussion about the update that was done last Feb here: https://chicagoschooloptions.com/forums/topic/when-are-cps-t...

* You can see the distribution of the tiers from a few years ago on this site: http://cpstiers.opencityapps.org, owners say it's no longer maintained. It seems currently there's no map showing all tiers, you can check only a single address at https://schoolinfo.cps.edu/schoollocator/index.html

BobaFloutist2• 2 years ago

Then they should be publicly run and free to take and send to universities.

seanmcdirmid7• 2 years ago

There is a lot of support for students who can’t afford to take the test. What isn’t really included is test prep. China has the gaokao, which everyone can take, but the prep factor is real even if they tried to outlaw it.

brigadier1325• 2 years ago

"Should". Every time I read the word "should" I cringe. Everything should be free. I should be able to go to restaurants for free and have the chefs prepare the food for me and the waiters serve me and I should do this and not have to pay money. After all food should be a right.

disintegore• 2 years ago

More precisely, it's not the standardized testing that's hurting, it's the underlying material conditions.

troupe6• 2 years ago

If a school asks for SAT or ACT scores but, hypothetically, wants to discriminate based on race or some other characteristic, doesn't requiring the ACT/SAT make them more likely to be liable for discrimination because there is an objective standard to show that they are treating some groups differently than others? I thought that some of the recent rulings against selecting students based on their race hinged on the statistics showing that some groups were able to get in with lower ACT/SAT scores, while other groups had to have a much higher score to be admitted.

retzkek• 2 years ago

The NY Times article about this addressed that concern specifically, indeed it was the key point of the study:

It’s worth acknowledging a crucial part of this story. Dartmouth admits disadvantaged students who have scores that are lower on average than those of privileged students. The college doesn’t apologize for that. Students from poor neighborhoods or troubled high schools have effectively been running with wind in their face. They are not competing fairly with affluent teenagers.
I also asked whether she was worried that conservative critics of affirmative action might use test scores to accuse Dartmouth of violating the recent Supreme Court ruling barring race-conscious admissions. She was not. Dartmouth can legally admit a diverse class while using test scores as one part of its holistic admissions process, she said.

https://www.nytimes.com/2024/02/05/briefing/dartmouth-sat.ht...

bwanab• 2 years ago

The way you phrased it, yes. But, that isn't what the report was saying. It was using a generalized criteria of disadvantaged meaning low income or non-english speaking. Race based criteria can't be considered, but income can which means poor white kids from Appalachia as well as poor black and latino kids from urban areas.

jonhohle• 2 years ago

That’s why they choose something not directly associated with race like socioeconomic environment. If the result is correlated with race, that’s not necessarily an issue with admission processes.

There are other non-academic tests like CogAT, Wechsler, etc. that may be even better since they evaluate abstract reasoning and scale from young to old without significant modification. Those are used by gifted programs and organizations like Mensa. It’s possible that these would be adequate replacements for college admissions as well, but you may get a bunch of mildly on-spectrum kids who have never had to work hard for academic success (from my personal experience).

fakedang• 2 years ago

Not just a higher score. Kids with 1600s (or close) were straight up not even considered.

gnicholas18• 2 years ago

Yes, it will be easier to sue Dartmouth (successfully) for race-based discrimination if/when they return to requiring standardized tests. For example, they won't be able to admit certain minorities with low test scores while not admitting other applicants with similar/better test scores.

However, it sounds like their system works around this problem by weighting the test score based on the average score at the school. So they can still down-weight the application of a poor Asian student from a more challenging school, compared to a different minority student who goes to a school with worse SAT performance. If Dartmouth's percentage of Asian students doesn't tick upward materially in coming years, I would assume this is what they are doing.

To steelman the argument for this process, it's effectively boosting the scores of kids whose parents didn't care as much about education, or who had no avenue into a challenging school. On the other hand, this process dings students from families that prioritize education, who work really hard. Those kids would probably succeed at Dartmouth, and beyond.

piombisallow• 2 years ago

What, you mean kids from a disadvantaged background can't build a 10 page CV filled with outreach, volunteering, expensive sports and musical practice by the time they're 18? They don't have an instinctive feel for the current pieties, bred into them since they're 3, to put in their admission essays?

laidoffamazon7• 2 years ago

This is a super goofy thing to say about Dartmouth admissions, because as of 2019 they expected admits to have all of that and have a 1550+/35+ test score.

I'm beginning to think a lot of the people commenting on these things have this weird concept of these schools admitting poor kids from the sticks and asian kids from Chinatowns with no resume items except good grades and test scores....it's true they have much lower standards for them, but they still need to have a decently "pointy" resume to get in.

The only way Dartmouth can fix it's "equity" problems is by shutting down for good.

yumraj• 2 years ago

No they are supposed to put their sob story and life’s traumas that shaped them into their college essay.

https://m.youtube.com/watch?v=MyD0m7JXgjA

titanomachy• 2 years ago

piety (n.) a belief or point of view that is accepted with unthinking conventional reverence.

TIL. I only knew the other definition, "the quality of being religious or reverent".

omgJustTest5• 2 years ago

IMO the thing that tests do not sample is persistance. Missing that degree of freedom misses important mind-sets to achieving goals.

One of the most interesting studies, not specifically focused on "success" / GPA, but on "life-satisfaction" comes from a longitudinal study "Project Talent"[1]. The key correlation to "life-satisfaction"/happiness is how many questions a person _attempts_ to answer on an otherwise un-finishable exam.

[1]https://en.wikipedia.org/wiki/Project_Talent

CharlieDigital2• 2 years ago

    > IMO the thing that tests do not sample is persistance

See Angela Duckworth's research on "grit"; it is indeed very difficult to test for.

nemo44x• 2 years ago

Wouldn't someone with grit will find a way to persevere and thrive regardless of which institution accepts them?

causi• 2 years ago

Disagree. You can retake the tests, hence they do test for persistence.

eapressoandcats• 2 years ago

Notably it’s not clear if there’s a better determinant of that. Tests at least have the advantage of being a knowable thing you can prep for, instead of whatever you have without tests.

dpweb2• 2 years ago

Always felt that the ability to think was indicative of academic success.

But also a good SAT for (those not naturally brilliant = most of us) also a positive indication of the student work ethic (to study for the test).

The problem I believe with the disadvantged students argument, which I believe to be a very real problem - is that lowering the standards wrt the test scores - is just a bad solution to a real problem.

Poor students do sit at a disadvantage - but the fix is much more difficult and complicated, and involves that child's entire life experience until they become 17 or 18 y/old. No one has a handle on how to fix it.

I get the feeling smart + hard work will come back into fashion when it comes to academic opportunities.

visarga• 2 years ago

It's also risky to take on debt if there is a higher chance of not graduating. Being admitted on a lower score might come back to bite the student. There's no substitute for actually having a handle on the subject matter. The fix is to help students before they start lagging.

nyrikki5• 2 years ago

Except that metrics like how many TVs a household has are indicators of performance on SAT scores.

Obviously this is a proxy for socioeconomic status, and people higher on that ladder will have better support structures to do better in school.

This is begging the question as socioeconomic status is the primary indicator of educational attainment in our country today.

They aren't measuring potential, but the persons ability and advantages. Just as buying more TVs won't improve your students performance.

gizmo686• 2 years ago

This point is specifically addressed in the paper. The predictive power of SAT/ACT goes far beyond its corralation to socioeconomic status.

Importantly, the relationship between first-year college GPA and SAT/ACT scores is likewise quite similar across neighborhood income and other demographic subgroups at Dartmouth. By contrast, Chetty, Deming, and Friedman (2023) show that certain non-test score inputs in the admissions process, such as guidance counselor recommendations, do not predict college performance even though they do advantage more-advantaged applicants at IvyPlus institutions, increasing their admissions chance

nostrademons3• 2 years ago

The point of the report is that SAT scores are less correlated with socioeconomic status than other metrics used by admissions like GPA, high school reputation, extracurriculars, essay content, etc. They're still correlated. One of the interesting findings of social science research is that a lot of different metrics like IQ (itself a correlation between different subtest performances), GPA, parental income, child's income, parental wealth, child's wealth, college acceptance chance, eventual education level, ability to delay gratification, general health, etc are all correlated - that's the idea behind the g-factor, a measure of general intelligence that highly correlates with wealth & income. But this research shows that among those correlates, SAT scores are actually less correlated with socioeconomic status than the other metrics that admissions committees can use.

This is also why the SAT was adopted by Harvard in 1934. Prior to that, most students attending Harvard came from private New England boarding schools that could only be afforded by the upper class. They saw an opportunity to diversify the student body by testing the general public and admitting those that scored highest. As an added benefit, the SAT was much easier to grade than previous essay-based admissions standards, and so it reduced cost for the University.

balderdash2• 2 years ago

> who attend high schools for which Dartmouth has less information to interpret the transcripts

IIRC most colleges correlate incoming and outgoing grades for specific high schools - so that a 3.0 average at a highly competitive high school (think Stuyvesant or Thomas Jefferson) is a better indicator of success than a 4.0 at a poor performing school - the SAT is very helpful to add context to a GPA for a students applying from schools where colleges don’t already have a sense for the HS program’s rigor.

I came to appreciate my high school education when I attended a highly selective university and found my grades going up from high school without actually doing any more work while seeing class valedictorians (I barely scraped by with a cum laude) from other high schools really struggling.

crazygringo• 2 years ago

Which was, of course, a major justification for creating the SAT in the first place:

First offered in 1926 by the College Board... Early proponents of the SAT argued that the admissions exam made higher education more meritocratic. Admissions officers at elite institutions like Harvard believed the test would identify talented applicants at less academically strong high schools and accelerate their journeys into higher education. [1]

[1] https://www.bestcolleges.com/blog/history-of-sat/

laidoffamazon7• 2 years ago

There was probably a time in which SAT scores are weighted highly in admissions for elite schools.

Now it's a starting level filter. You're not getting in because of top scores, but you will get rejected because of bad scores absent a massive alternative hook. People that get in have top scores and other superhuman traits.

photochemsyn• 2 years ago

A chance to acquire a massive debt (four year tuition alone is > $240,000, not counting rent and food and books). Whether or not that will be worth it (as the main reason to go to an Ivy League school is for social networking with the children of wealthy parents who might fund future endeavors) is an open question.

runarberg3• 2 years ago

You have to look at the context surrounding them though.

In today’s educational environment in America, students are rewarded heavily for belonging to an upper class, they are made to donate volunteer hours, join a sport team or a social club, participate in extracurricular activities, do homework (hopefully in a peaceful home setting with helpful parents and/or tutors), and then master the niche skill of pen and paper test taking. The SATs at the very least allow some students to skip all but last one, and have some chance of getting a decent education.

Ideally the systemic barriers to proper education should be dismantled. Remove this volunteer hour, move sports and social activities outside of the school system, remove homework, provide tutors to everyone that needs one, stop relying on pen and paper tests as the primary method of evaluation, etc. With all these systemic barriers removed, you can be sure that the SAT will have a diminished value.

mnky9800n• 2 years ago

Then why do the tests cost money?

hintymad2• 2 years ago

Is this a US thing because there has been so much debate on race, equity and etc? I thought this was obvious. Academic excellence arguably requires the least resources, which means the difference in social-economic status matters the least. All one needs is access to excellent books and internet resources, a competent teacher, and optionally a supporting community. And one driven kid can get all of them from libraries and the internet nowadays.

Besides, SAT and ACT are really simple tests. If one gets 1600 for SAT, it does not mean this person will excel in academics, but if one gets 800 for SAT, well, this person statistically is not smart enough to achieve academic excellence. Of course, this person can be smart in many other ways.

chaps• 2 years ago

"All one needs is access to excellent books and internet resources, a competent teacher, and optionally a supporting community"

That "all" is doing a heck of a lot of work, to the point of being indistinguishable from satire!

A lot of SAT success happens because of families who can afford to train and hire tutors specifically for preparation.

light_hue_13• 2 years ago

If this was a paper submitted by one of my students, they would fail. Why would these faculty members sign their names to this drivel? If you read carefully, you see how useless SAT and ACT scores really are.

Here are just a few insane things.

Figure 1 shows a linear relationship between SAT scores and cumulative first-year GPA for Dartmouth students. SAT scores are grouped into 16 equal-sized bins, and we plot mean first-year GPA (y-axis) against mean SAT score (x-axis) in the bin.4 Within any smaller range of scores, first-year GPA and the SAT maintain the same linear relationship.

Total nonsense. Figure 1 does not mean that SAT/ACT scores are predictive of academic achievement! Without seeing the variance you cannot make this statement. At all. The fact that there is a linear relationship means nothing statistically.

SAT by itself explains about 22% of the variation in first-year GPA

So we're explaining less than 1/5th of the variance. This makes SAT scores basically worthless, this account for almost none of the real world performance of people, never mind how redundant they are with GPAs.

The correct title for this would be: SAT and ACT are not highly predictive of academic achievement at Dartmouth.

There's a massive disconnect between the title and the data. The data shows there is likely some statistical effect on performance by SAT. The title would have you think that if you know a student's SAT scores, you would be good at predicting their outcomes at Dartmouth. The data shows this is wrong: if you only used SAT scores, you would almost always be wrong in predicting how well a student would do.

A lot of papers play this game, they imply that statistically significant means meaningful. Nothing could be further from the truth. Something can be statistically significant but inconsequential.

Never mind that they define as success at Dartmouth as your first year GPA.

toomim• 2 years ago

Um, 22% > 1/5.

1980phipsi2• 2 years ago

Total nonsense. Figure 1 does not mean that SAT/ACT scores are predictive of academic achievement! Without seeing the variance you cannot make this statement. At all. The fact that there is a linear relationship means nothing statistically.

You are putting words in their mouth from that quote. They didn't use that Figure to claim the test scores are predictive of academic achievement. They specifically say it is evidence of a linear relationship. They argue in the next paragraph that they are and use regression analysis to make their case.

So we're explaining less than 1/5th of the variance. This makes SAT scores basically worthless, this account for almost none of the real world performance of people, never mind how redundant they are with GPAs.

Compare that to high school GPA, which explains ~9%. The test optional movement implicitly puts more weight on high school grades by putting less weight on test scores.

I'm not saying their analysis is perfect, but your arguments aren't that strong...

At the end of the day, the analysis focuses on a subset of people who are going to Dartmouth. This is a highly filtered subsample of the total population. If you took a random sampling of 18 year olds, gave them the SAT, and then enrolled them at Dartmouth, then I hypothesize you would see a significantly stronger relationship between test scores and college grades than what is observed in their data.

light_hue_13• 2 years ago

The knowledge of statistics on HN is astoundingly bad.

Figure 1 shows nothing. At all. Without knowing if the results are statistically meaningful, showing a linear relationship means nothing. You can draw no conclusions from that figure. The only thing it does is lie with data, to imply to an observer that doesn't understand statistics that this means there is a meaningful relationship between the two variables.

I'm not saying their analysis is perfect, but your arguments aren't that strong...

They said "SAT and ACT scores are highly predictive of academic performance". This means that they can predict academic performance based on SAT. They absolutely cannot. Their own data shows this beyond any doubt.

anon2913• 2 years ago

I'm confused. The authors cite several instances of research done at other institutions as the main evidence that SAT/ACT scores are predictive of overall success. At that point, the a priori assumption is that Dartmouth would behave the same way, which they then go on to show. I agree that they could have fully analyzed graduation rates and full-term GPA, but other studies have already done that. I do believe their analysis is thorough enough (looking at first year GPA) to adequately show to me that there is absolutely no reason to believe that Dartmouth would be any different than the other institutions at which much larger / more rigorous studies have been done.

I mean... do you have a full double-blind study done when you're sick to determine the right dosage for you, or does your doctor consult an existing dosing study, make sure nothing is substantially different between you and the study group, and then decide upon an appropriate dose. There's a certain point at which over-analyzing makes little sense. The authors here ARE NOT engaged in research. This is NOT peer-reviewed. It's a letter elucidating an opinion by researchers in the field based on their knowledge of previous literature in the field and then a surface-level analysis to make sure that those results ought to apply to Dartmouth as well.

Moreover, you're hyperbolizing yourself. Namely, you criticize the title, which is not the author's title at all. Their letter is titled "Report From Working Group on the Role of Standardized Test Scores in Undergraduate Admissions"

The authors main claim (on the first page after niceties) is that ".Our overall conclusion is that the use of SAT and ACT scores is an essential method by which Admissions can identify applicants who will succeed at Dartmouth." This is not "SAT / ACT scores are highly predictive...". It's an entirely different statement. Let's not confuse HN editorializing with the contents of the letter.

The title would have you think that if you know a student's SAT scores, you would be good at predicting their outcomes at Dartmouth. The data shows this is wrong: if you only used SAT scores, you would almost always be wrong in predicting how well a student would do.

Except if you read the text, that's not at all what the authors claim. The authors do not claim the title (again... that's HN's title, not the paper's). The authors say:

In short, consideration of SAT scores allows Admissions to identify applicants who will thrive academically at Dartmouth better than does the use of high school GPA alone.

which is true. Even if you don't think it's a useful predictor, it is a more useful predictor than GPA. This is a reasonable contrast because in the absence of test scores, this would be the only data available.

As for the other critiques. I agree that Figure 1 is not useful and shouldn't have been included. However, the table at the end is pretty straightforwards and provides essentially the same conclusion as above, which is that SAT is better than GPA. Moreover, just because the figure they included is bad does not mean the data they studied was. We'd need to know more, not wholesale reject their conclusion.

The authors then go on to argue that other papers agree with these conclusions (they do), and that alternates to both SAT and GPA (letters of rec, etc) have been shown to exacerbate inequality while having very little power to predict how well a student will do.

The fact that there is a linear relationship means nothing statistically.

It does... in fact... mean that there is a linear relationship in the observed data with the given levels of significance. I'm not sure what exactly you're going at here. I agree a scatter plot of all the data might be more useful, but honestly, given the plethora of other work on the subject published by so many disparate institutions of higher ed, I really have no reason to believe dartmouth would be much different. So while the included figure is not great, I'm guessing that if we saw all the data the conclusion would probably still be the same.

light_hue_13• 2 years ago

When I talk about statistics on HN it feels like I'm taking crazy pills. The HN community pretends to be intellectual, but the level of mathematical education here is terrible.

Moreover, you're hyperbolizing yourself. Namely, you criticize the title, which is not the author's title at all. Their letter is titled "Report From Working Group on the Role of Standardized Test Scores in Undergraduate Admissions"

The authors clearly say "SAT and ACT scores are highly predictive of academic performance at Dartmouth". That is what they want to show.

And this is totally false. They cannot accurately predict high vs low academic performance based on SAT or ACT scores. Their data clearly shows this is impossible.

Except if you read the text, that's not at all what the authors claim. The authors do not claim the title (again... that's HN's title, not the paper's). The authors say:

Huh?! That's literally their claim.

They even say in the conclusion "They are significantly predictive of academic success at Dartmouth"

This is just totally wrong. They are a statistically significant but practically insignificant predictor of academic success when you define it in the most absurd and irrelevant way (first year GPA).

It does... in fact... mean that there is a linear relationship in the observed data with the given levels of significance.

Summarizing data with some line, without any statistics, and saying "look the line slopes up", does not mean there is a linear relationship. At all. You cannot conclude one from the other.

The fact that this isn't obvious is truly frightening.

gnicholas18• 2 years ago

I'm not super great at statistics, despite having taken several classes in college (which is now decades in the rear-view mirror). I am interested in your critique here but can't quite follow what you're suggesting. Can you make it in plainer terms?

I understand that HN has blindspots, but I'm a bit skeptical that HN has a blindspot and a gaggle of Dartmouth professors who were selected to do data analysis have the same one.

amusingimpala75• 2 years ago

Recently saw an article about parents spending hundreds of thousands of dollars to have their kids work with professional resume builders just to get an advantage with college applications. Now that many colleges are trying to phase out SATs because “it discriminates against poorer households”, it’s being replaced with essay and resume based things that seem to provide an advantage to the richer households who can afford the absurd amounts to very carefully cultivate the documents.

Macha• 2 years ago

Yeah, Canonical presented one of these 40 question style interviews that felt like a very thinly veiled class check in comparison to the usual interviews in my country. I'd much rather a whiteboard exercise than that.

Apparently it's been discussed on HN before: https://news.ycombinator.com/item?id=37059857

boeingUH60• 2 years ago

That was the aim always. To disadvantage the poor and middle class by disguising under “equality” and “diversity”.

Two-faced hypocrisy at its peak.

kragen• 2 years ago

this fact suggests that dartmouth is not weighting sat test scores highly enough in their admissions process

if dartmouth took sat scores into account in the admissions process by precisely the right amount, then within the set of people they actually admitted, sat scores would have no predictive value with respect to academic achievement, because the students who were admitted despite low sat scores would have been admitted due to some advantages detected by admissions that turned out to matter precisely as much as the higher sat scores would have

if, across the entire population, higher sat scores had no predictive value with respect to academic achievement, dartmouth could get this result by preferentially admitting students with lower sat scores. (for example, maybe you would need a high school gpa of 4.1 or higher to get in if your meaningless sat score were 1500, but this hypothetical bizarro dartmouth would let you in with a high-school gpa of 3.1 or higher if your meaningless sat score were only 1000; if the 3.1 students in high school continued being 3.1 students at dartmouth, it would make the meaningless sat scores highly predictive of academic achievement at dartmouth, among the people who actually got in.)

of course that would be an extremely unlikely thing for dartmouth to do, and they mention a lot of other evidence that sat scores predict academic achievement in general, not just among the students they admit

r-squared of 0.22 is still not a great predictor, though

if you want to read this report, it may be useful to open it in two separate browser tabs, scrolling the second one to the figures, which begin on p. 9

Scubabear684• 2 years ago

This is not surprising to me. Our local school district started using social promotion several years ago when a new Superintendent started. Struggling students were ratcheted down to being taught materials 1-3 grades below their level. Extra credit not related to the subject were allowed to boost grades (such as coloring assignments in 4th grade Spanish). The end result was a huge population of kids with straight B averages who performed dismally on State standardized testing.

We know because our kids were two of those students. We took them out of public school into private last year. The first six months were hell as they had to catch up from their deficits and learn how to study for real. 18 months later they are getting real A’s and B’s (and a few C’s) that are real grades.

We need standardized tests because more and more school districts are going the social promotion route

ericjmorey• 2 years ago

That seems like a poor plan to get kids "up to speed" with their peer groups. Especially if any of those students have parents who are unable or unwilling to provide a nurturing environment for their kids.

Scubabear684• 2 years ago

Standardized tests aren’t a fix, they are a form of measurement to diagnose the problem. When your son has a B average in 8th grade and scores in single digits on State standardized tests, it is clear there is a very serious problem.

The fix is to get proper leadership in the schools. Our Superintendent started his career as a gym teacher, and it shows. At a school board meeting when parents criticized the low standardized test scores, he crowed that they don’t matter because colleges stopped looking at them.

bovermyer• 2 years ago

Ah, but how well does academic achievement (at Dartmouth, or others) correlate to job/life performance?

kqr2• 2 years ago

Also listen to the Daily Podcast The Messy Fight Over the SAT: https://www.nytimes.com/2024/01/17/podcasts/the-daily/the-wa...

alberth3• 2 years ago

Anecdote: a friend of mine is an admission officer for one of the US largest public university.

They told me that if your kid has a 4.0 GPA (a perfect GPA), do not take the SAT/ACT ... because, without those standardized test scores - your kid is "perfect".

But unless your kid has a perfect 4.0 GPA, taking the SAT/ACT typically helps your child get into a better school than their GPA might indicate.

YMMV

elteto3• 2 years ago

That sounds very localized to the institution your friend works at. And public universities have very different admission procedures than private ones.

You won’t get into a high tier private university (Ivy or not) without SAT/ACT scores.

alberth3• 2 years ago

We’re saying the same thing.

99% of kids don’t have a perfect 4.0 GPA.

But the point is, if you do happen to have a kid who does - there’s more harm than good that will result from taking SAT/ACT.

For all others, it’s highly recommend to take.

verst• 2 years ago

I was born and raised in Germany where I attended an ordinary high school. Eventually I decided I wanted to attend university in the US but had no help navigating the process and requirements. Needless to say, I realized I had to take the SAT last minute and had never seen a practice test, or taken this kind of test in English before. All my instructions had been in German, and the German school system does not usually perform multiple choice tests - everything is about showing your thought process and your steps. Needless to say, I received an abysmal score. For me this test truly only tests how good you are at that test.

I applied as an international student to a top tier liberal arts college equivalent to Dartmouth. I was able to make my case and was accepted there. My academic performance in both my first year and the rest of the four years was stellar.

Perhaps the SAT can be predictive of performance for those who grew up in the system that prepares you for the SAT (or ACT). For anyone else, this is however meaningless.

gnicholas18• 2 years ago

For me this test truly only tests how good you are at that test.

International students are a completely different story. As you say, your entire educational experience had been in another language. This is not the case for students in the US. All students here have taken many multiple choice tests in English by the time they get to the SAT. For example, there are state- and federal-mandated tests that students must take.

Perhaps the SAT can be predictive of performance for those who grew up in the system that prepares you for the SAT (or ACT). For anyone else, this is however meaningless.

It is perhaps close to meaningless for an international student, but for a student who went to school in the US, it is not meaningless at all.

hirvi746• 2 years ago

everything is about showing your thought process and your steps

You probably did well at a US college because you came from a system that actually provided you with critical thinking and reasoning skills. Instead of a system, like the US, which is rife with route memorization and multiple choice tests.

I would imagine the opposite experience would be much more difficult (assuming what I remember from studying German is actually correct).

rconti• 2 years ago

NYTimes "The Daily" podcast had an informative episode on the matter[1]

I'm out of the loop since my college days are behind me, and I don't have kids.

I had heard colleges were going to rely less on standardized testing (and many dropped the requirement during Covid when SAT tests didn't take place at all). I had also heard that there was controversy over reliance on standardized tests, because the wealthy can more easily afford expensive test prep, which initially seemed reasonable to me.

But then when I listed to the podcast I was stunned that the colleges seemed to think relying much more heavily on grades and extracurricular activities was somehow an _improvement_. I thought "grade inflation" was a big worry over the past couple of decades? I thought we all joked about today's nonsensical GPAs where kids get, whatever, 4.5s or 5.0s on a scale of 0-4. And it goes without saying that it's obvious to most of us that volunteering and participating in travel sports and debate teams and so on, is FAR more accessible to those with money, with transportation, with resources, with involved parents?

Standardized tests can't tell us how diligently kids do their homework, how well they do after encountering a setback. Maybe they favor kids who are smart but lazy. They can't tell us how a kid will do when they go from being a high school standout to a below average Harvard student.

But they sure as hell tell us a lot more than utterly subjective grading systems with no standardization whatsoever, and "intangibles" that tell us only how good a kid's family (or hired college prep expert) is at identifying activities that will get them into a selective University.

1. https://www.nytimes.com/2024/01/17/podcasts/the-daily/the-wa...

dyno12345• 2 years ago

re: grade inflation, they can statistically normalize the grades

fwiw GPA and SAT scores are highly correlated anyway

tech_ken• 2 years ago

Not exactly, it's tough because every high school needs a different normalization, and it may drift from cohort to cohort. Most universities only ever see a handful of students from any given HS, so it's really tough to untangle the big spike of 5.0 students. Usually the results are weakly predictive of academic success, if the correlation even exists at all. External agencies like the CollegeBoard offer data as a service type solutions for this, but they're (A) low quality, (B) expensive, and (C) have lots of weird privacy concerns because it's data about minors.

deltarholamda• 2 years ago

People who take the SAT are self-selecting. An awful lot of kids don't take it, and a lot of schools don't require it. So the sort of kid who takes it will also be the sort of kid who gets good grades.

garyrob• 2 years ago

My own experience was that I was a mediocre high school student due to my ADHD-Inattentive, but then the SATs were interesting enough to me that I was able to really, really focus for the length of the test. And I did far better on that test that anyone would have suspected I could do based only on my grades, and it was, in fact, predictive of my college performance. In college, I was able to figure out tricks to manage my ADHD.

hirvi746• 2 years ago

Interesting, for I was quite the opposite.

I'm ADHD-C, but I always did well in school, got shit on by the ACT, went to college and did well up until my final year, but that was mainly due to some extenuating circumstances and not my drive or intelligence.

carabiner2• 2 years ago

Yep. And STEM students have higher SAT scores than lib arts. And Asians have higher SAT scores than everyone else.

breadwinner2• 2 years ago

And Asians have higher SAT scores than everyone else.

And that might be the biggest reason to prefer "holistic review" over SAT scores. Originally invented to limit the number of Jews [1], "holistic reviews" is now being used to limit the number of Asians in top universities.

[1] https://www.economist.com/united-states/2018/06/23/a-lawsuit...

carabiner2• 2 years ago

That's why you prefer holistic review?

0xbadcafebee• 2 years ago

Of course high scores on a test are going to correlate with academic achievement, they are both just a measure of ability to pass tests. This is like saying people who swim very well will have high achievement in swimming competitions.

The paper's summary implies that they should make testing mandatory, because when they made testing optional, they couldn't see the students who they'd like to admit. But they kind of ridiculously don't comment on the fact that mandatory testing might not make any difference at all.

There must be reasons students don't send in scores when testing is optional. But just making the testing mandatory doesn't address those reasons at all. So there is no reason to conclude that suddenly applicants would start sending in scores at a higher rate. It's equally likely they would just stop applying.

laidoffamazon7• 2 years ago

My ideal admissions system would consist of highly weighting standardized test scores and grades along with community impact and other extracurriculars as well as letters of recommendation from teachers and other community members.

...oh, wait, we have that admissions system. That's the admissions system of every flagship state school and the US Service Academies. Why does Dartmouth (and before it, MIT's) flip-flop on this matter to 99% of us again?

gnicholas18• 2 years ago

That’s not what the UC system does — they refuse to consider SATs.

laidoffamazon7• 2 years ago

I’ll admit I forgot they kept test optional - I’ll celebrate when they bring it back, not when elite schools whose purpose is to segregate and enrich the few do it.

costco3• 2 years ago

Extracurriculars are by and large a scam. I became vice president of a few clubs, started a UNICEF chapter at my school, etc. But I made almost 0 impact on the world because we only raised like $200, most of which was from me and my family. The proof that most people don't actually care about these things is that they quit them the day they start college. Maybe participation in something like ISEF is a better signal. But not National Honor Society membership (which I of course had, because that's what you do).

No one can verify the accuracy of letters of recommendation and teachers routinely break the rules by showing them to students. In some cases I have heard of teachers asking students to write them (!).

laidoffamazon7• 2 years ago

Yeah, that kind of extracurricular isn't enough for top schools unless you have some other kind of hook. Winning ISEF/placing high (not 100% how it works) already is a pretty good hook, I'm sure most of the MIT entering class won some equivalent national prize like this.

In some cases I have heard of teachers asking students to write them

This is very common in college, most professors don't have the time to write letters of rec for all of the students they do research with etc. They may edit it and sign off after.

TheCaptain4815• 2 years ago

It seems fairly obvious you need these standardized tests, especially when GPA inflation would be incentivized otherwise, but the SAT and ACT drive some very uncomfortable discussions involving intelligence, primarily on its hereditary nature and the possibility of "group-differences". Gets even more uncomfortable since we're talking about children. If "group-differences" exist at the genetic level for example, why shouldn't affirmative action be required since disadvantaged groups also pay taxes to these schools.

gumby• 2 years ago

MIT publicly said their stats showed that SAT was the best predictor of graduation (MIT tracks six-year graduation rate): https://mitadmissions.org/blogs/entry/we-are-reinstating-our...

They accepted me (same year as the guy who wrote that article!) from a high school than only gave letter grades, computed no class rank or GPA, and offered no "AP" classes or SAT prep. Then again, back then, I don't remember colleges caring about extracurriculars either, though I might have forgotten.

ren_engineer2• 2 years ago

The SAT was based on IQ tests used during WWI to try and find draftees with the most potential who shouldn't be wasted in the trenches. Standardized tests have always been about making things as fair as possible, and the fact that paying huge money for tutoring only results in minor score improvements should be a sign they are effective

the war against them was because people didn't like the implications that poor Asian immigrants were able to outscore other demographics regardless of income, which goes against modern academia's favored nurture over nature mindset and destroys the basis for trillions in social programs. Standardized tests effectively threaten the entire house of cards, even more than the replication crisis showing that most of the "research" used to justify modern social programs was fake in the first place

drewcoo• 2 years ago

If you're going to talk about the origins of the SAT, don't forget that it was designed by a eugenicist to try to exclude black candidates.

Ironically, the problem with them now is that they're now used to exclude white people.

https://en.wikipedia.org/wiki/History_of_the_SAT

ren_engineer2• 2 years ago

Eugenics was hot among progressive academics in those days, the founder of Planned Parenthood was one and so was the guy whose statistical methods are still the basis for all modern research

back then being against eugenics meant you were a crazy religious nutcase who couldn't accept scientific fact about evolution and genetics

codexb• 2 years ago

The war against standardized tests has far more to do with black students performing disproportionately poorer on them. Many people are unwilling to acknowledge the multitude of cultural and socioeconomic factors responsible for that and instead want to simply chalk it up to racism. And so there's a war against any admissions policy that results in disproportionate racial outcomes, until we're left with admissions policies that are openly racist in order to achieve race-based results.

fireflash38• 2 years ago

Many people are unwilling to acknowledge the multitude of cultural and socioeconomic factors responsible for that and instead want to simply chalk it up to racism.

How many of those cultural and socioeconomic factors were almost directly related to racism though?

sangnoir• 2 years ago

And so there's a war against any admissions policy that results in disproportionate racial outcomes

Except legacy admissions and pay-to-play donations. After all, pedigree and wealth are merit (or proof thereof) - no need for pesky test scores in the face of such overwhelming evidence.

csa2• 2 years ago

the fact that paying huge money for tutoring only results in minor score improvements

This might be true in aggregate, but I’ve seen data that says otherwise (buddy owns a test prep school). The keys are that the student wants the test prep and that the student has room to grow (i.e., not already near the top, where increases are capped).

Middle of the pack students can gain 50-150 points per skill (this is a lot).

the implications that poor Asian immigrants were able to outscore other demographics regardless of income, which goes against modern academia's favored nurture over nature mindset

Is this implying that Asian immigrants do better because of genetics (nature)?

The non-agenda-based research on this topic is pretty clear that Asian immigrant success is mostly about a culture of schooling and learning in Asian communities. This includes things like tutoring, “cram schools”, and test prep.

Here is an example of research/views from outside the US:

https://www.otago.ac.nz/deepsouth/vol2no1/setsuo.html

Almost all academic success is built on a culture of learning of some sort (e.g., parental involvement in schooling, reading to kids, etc.). Note that this is true regardless of race. The higher quality (non-agenda-based) research in this area all focuses on how this culture of learning can be fostered, with special focus on low SES (low socio-economic status) families regardless of race.

destroys the basis for trillions in social programs
Standardized tests effectively threaten the entire house of cards

Programs like Head Start in the US have measurable positive impact:

https://www.brookings.edu/articles/the-long-term-impact-of-t...

Fwiw, I actually agree that a lot of research in education is non-replicable, agenda-based bullshit, but your comments here go against both the better research and fairly substantial anecdotal evidence (my buddy’s test prep program).

legolas24122• 2 years ago

Programs like Head Start in the US have measurable positive impact:

Something can have an unjustified basis, but have a measurable positive impact. Just because the unjustified basis of race was correlated with income. But using the justified basis of income and wealth will not only lead to even more measurable positive impact, it will also be fair.

fngjdflmdflg• 2 years ago

Is this implying that Asian immigrants do better because of genetics (nature)?

Yes, that is very likely true. I would go as far as to say that the "nurture" side of this coin is to a large extent an indirect result of nature as well.

runarberg3• 2 years ago

That is a pretty old link which has been severed a long time ago. They’ve even changed the name from Scholastic aptitude test to just SAT, in an effort to convey that SAT doesn’t actually measure anything but it self. (This is in a stark contrast to IQ which makes ridiculous and pseudo-scientific claims that it measures something approaching what they call general intelligence)

I have many beefs with SAT and standardized testing in general, but the link with IQ is not (anymore) one of them. IQ is a horrible and racist concept that should remain along with phrenology as a pathetic attempt to use science to explain racist beliefs. SAT used to be there. But we have to give them credit for having moved on.

cm20123• 2 years ago

IQ is one of the most reliable and proven metrics in social science.

jobs_throwaway2• 2 years ago

IQ is a horrible and racist concept that should remain along with phrenology as a pathetic attempt to use science to explain racist beliefs

How then do you explain IQ's strong correlation with positive life outcomes like work success, educational achievement, happiness, etc

https://www.scirp.org/journal/paperinformation?paperid=74943....

https://www.cambridge.org/core/journals/psychological-medici...

Jun84• 2 years ago

modern academia's favored nurture over nature mindset

The is a simplified way of putting it by I would roughly agree with this statement. However, I think you're drawing the wrong conclusion from the fact that "poor Asian immigrants were able to outscore other demographics". It is absolutely about nurture, but not in a way that the "academia" wants to emphasize. I think the main reason these poor kids achieve better is the support and, yes, the tremendous push they get from their families on topics related to education. I'd wager that having a two-parent family that places importance in education correlates most with academic performance.

costco3• 2 years ago

What does the new SAT even measure? It's just some Common Core© Algebra 1 and 2 knowledge (also basic trig), reading comprehension, and a section where you read sentences in your head and see if they sound right. You can get over 10 questions wrong and still get a 1500. There's no more shape rotation questions anymore or really any question where you aren't just applying an equation.

I got a 1530 by just doing 20 practice tests from the stickied thread on r/SAT and basically learning every possible question type. 1-2 tests a week over the summer and into the early school year. Not a monumental amount of effort and didn't cost me any resources besides time and paper. I think most people just don't know how or what to study.

Kwpolska3• 2 years ago

You know what’s an even better policy? Making standardised tests the sole admission criterion. No high school grades (which are often inflated, unless your teachers hate you), no extracurricular activities, no essays. Just one set of objective numbers from the national maturity exam; which tests and with what weights depending on the field of study. This is what universities do in Poland (for most fields of study; some might have extra tests to write), and it’s great at letting people from different backgrounds into universities while filtering out people highly unlikely to succeed.

maxglute• 2 years ago

You want academic prep to drive consumption in multiple sectors silly /s.

zahma• 2 years ago

What a silly idea. Universities are not just in charge with academic education but rather with the formation of a person’s character. To succeed with the latter, academic capability must be held alongside the estimation of one’s character, for it is not only about one’s intellectual capacity but rather how one uses it.

asdff7• 2 years ago

If schools started estimating the merit of one's character they'd have to probably throw out half of their own faculty and cut off ties with many of their donors.

LudwigNagasena2• 2 years ago

If academic education is not their top priority, it would be beneficial to separate research facilities from universities and let universities be glorified fraternities. It's a waste to spend resources of top professors on "formation of a person's character" when they can help build the next generation of elite scientists and engineers.

mcmoor• 2 years ago

No, it doesn't. We already have K-12 for that and rightfully so. Academia is supposed to be a place for complete adults to be able to optionally invest to specialize. At this rate we have to admit that 12 years of general education isn't enough to build characters so we have to make K-16, which is certainly an option with its own concerns.

asdff7• 2 years ago

In theory this is what the diploma or GED is already for. These are all certified by the state to show you've completed a certain amount of education to a certain level of quality. What is an SAT or ACT supposed to be on top of all that honestly? Really its tackling a symptom of colleges for whatever reason no longer trusting the states own education departments' diploma or GED. Which is damning considering these schools point to these same states accreditation as a quality measure for their own programs, forgetting that if the diploma is untrustworthy then so are these state accreditation...

Kwpolska3• 2 years ago

Where do the grades on a US high school diploma come from? They are arbitrarily assigned by teachers from the student’s high school, right? One teacher might give everyone good grades, while another will be more strict, and yet another may give good grades to their favourite students only. Grades of two students from the same class are often not comparable, let alone grades of two students from different states.

The SAT and the ACT are standardized tests, meaning that everyone in the country writes the same test on the same day, and the tests are designed to make the test results comparable between different sessions.

alphazard2• 2 years ago

This is refreshing. I sometimes wonder if the universities understand why they have value to industry. It's not the education. Schools are rating agencies for people.

The university admission process takes a more comprehensive look at candidates than is practical during the hiring process in industry. In particular they pay close attention to proxies for general intelligence (SAT math and to a lesser extent reading) and trait conscientiousness (grades, extra-curricular, etc.). If an employer sees Dartmouth on your resume, they can make an educated guess about your IQ and your conscientiousness, both of which they should care about as an employer.

As other comments have mentioned, measuring these psychometric traits more directly has the effect of identifying top candidates in unprivileged circumstances. While diluting these psychometric scores with other measures has the effect of biasing towards affluent candidates who can afford to do things that are impressive on paper.

smeej7• 2 years ago

Are we maybe missing the forest for the trees here? Like, do the test scores or the academic achievement at Dartmouth actually matter?

By sheer dumb luck, I am exactly the kind of intelligent schools and standardized tests measure for. I went to my first choice schools on a full academic scholarship for both undergrad and grad school.

And it's meaningless in the real world.

The most important connections I've made certainly weren't at school. The academics themselves weren't terribly useful in terms of preparing me for life outside the ivory tower.

Why are we trying to sell people on this dream?

alphazard2• 2 years ago

If what you said is true, then you are likely an ideal employee for the vast majority of roles. You can read well, understand what you are supposed to do, follow instructions, etc.

It's not meaningless in the real world. You may not be satisfied with your level of success, but that's dependent on how you define success.

You won't find many dumb people among self made multi-millionaires. But being high IQ isn't enough to get you there. It also requires creativity, people skills, a network, luck, etc. We don't test for those things because the people running the show aren't interested in identifying people to compete with them, they are interested in identifying people who will work for them.

smeej7• 2 years ago

I'm not, though. I'm terrible at taking anyone's word for anything, partially (not entirely) because I was told I knew more things than my peers.

I'm also garbage at just doing what I'm told. Who's giving me the directions? Why should I trust them? What do they know?

You seem to be thinking of the people who did well in school because they worked hard. Perhaps they are excellent employees.

Those of us who never had to apply ourselves to anything and were told incessantly how much better we were than our peers make terrible employees in most atmospheres.

dfedbeef• 2 years ago

High five to all the former gifted and talenteds.

seabass-labrax• 2 years ago

Disambiguation for my fellow residents of Britain: this is about university entrance tests for undergraduate courses at Dartmouth university in New Hampshire, USA, not primary school examinations at Dartmouth, Devon (the Original and the Best™).

francisofascii• 2 years ago

SATs are the worst predictor of academic success, except for all those other forms of predictive measures that have been tried. (a variation of Churchill's democracy quote)

kebsup• 2 years ago

I've taken ACT as an exchange student when I was in the US. I didn't need it, but everyone in my grade was doing it and it was for free, so why not.

Sometimes people say, that for ACTs, one needs a specific knowledge of how to take the test, which does not teach you anything, but I did not get the feeling.

All of the questions seemed quite reasonable, no tricks and not even a need for smart ideas (unlike let's say math Olympiad). Basically know the basic math rules, and you'll be fine.

The knowledge I've gained at a Czech Gymnasium with more traditional methods (i.e. no multiple choice) translated very well into getting a good score.

hirvi746• 2 years ago

Basically know the basic math rules

I was raised in the Southeast, USA. That's a big ask of us and our education system.

In my state, we have a few state administered tests that are taking throughout one's primary schooling years. Student take these exams at periodic intervals. For example, there is one for reading comprehension and assessment taken, iirc, at grade 3, 6, and 9 or something like that.

Well, 60% of students in my state right now who took the 3rd grade reading comprehension assessment have failed to demonstrate the ability to read at a 3rd grade level.

I have a hard time accepting these test [ACT/SAT] as some kind of arbiter or one's future when the education system is failing many students long before he or she ever gets to take the SAT/ACT.

foobiekr3• 2 years ago

I feel this way about most of the standardized tests. I took the LSAT on a whim which no prep and scored 95th percentile. The whole thing was completely reasonable and mostly tested basic logical reasoning and vocabulary/reading comprehension. At the time, there was a short essay component which was entirely about the ability to express whatever thoughts you were supposed to express clearly.

The thing about these tests is that they are all perfectly reasonable. The main reason people have a test with them is they also tend to identify people who _probably_ should not be in college at all and that is not a message anyone wants to accept needs to be sent.

coolbreezetft24• 2 years ago

In Canada there are no high school SATs and university admission is mostly based on high school performance. Universities there don't seem to have the issues that American ones do.

Living in America now I see a lot of parents spending thousands of dollars on SAT tutoring & preparation.

It seems a lot of things could be fixed just by being able to support and admit more students at the "good" schools? (also not sure why there is such a massive disparity in the quality of colleges here as well)

sudosysgen4• 2 years ago

Universities in Canada definitely do have issues due to the high school GPA system.

In Quebec, it's not present, instead you get in based on a Z-score renormalized based on how good your class was at standardized tests in CEGEP, which itself disproportionately looks at your standardized tests scores and those of your peers in high school

As far as the rest of Canada, highly competitive programs have to calculate the strength of a high school to normalize for grade inflation, especially in Ontario. It's a huge problem and makes it unfair for some students who don't go to the right high school.

cubefox• 2 years ago

SAT score statistics also make for some of the most offensive graphs possible. E.g.

https://s20.directupload.net/images/240205/xfm7k5l8.png

stanford_labrat• 2 years ago

What if schools had a minimum-level lottery system based on standardized tests? E.g. "you must score XXXX to be eligible for entry to a T20 school, and you will get in randomly". XXXX+50 for a T10, XXXX+100 for a T5, XXXX-50 for a T50 etc...schools could tailor their minimum score to whatever they desired.

kbos87• 2 years ago

Of course standardized test performance correlates with test performance a year later when someone arrives at a college or university. That's the wrong question to be asking ourselves.

This discussion is maddening to me because it assumes that what colleges and universities should optimize for is test performance, full stop.

I've never been a strong test taker, but I gained a lot from my time in college and I've done objectively well in my career. The way colleges, universities, and some of the people commenting here are willing to limit access to education and life-changing opportunity based on one narrow measure is antiquated and narrow-minded.

If I had to give advice to someone who is motivated and interested in higher education but doesn't perform particularly well on standardized tests, it would be to feel no shame in doing whatever it takes to find your way through or around the arbitrary barriers like standardized tests that are going to be put in front of you.

ksoped• 2 years ago

This just makes me more sad, thinking about what could have been. I had a 1500 on SAT from a school where the average is probably at 1000.

jncfhnb• 2 years ago

My grad program was fairly open about saying the standardized tests were used to sort applicants for consideration order and other disregarded. I thought that seemed fair.

Geisterde3• 2 years ago

There is a lot of really interesting and simultaneously totally useless information in the comments here. If I ran a college, im accepting whoever I decide, because its mine. Imagine accepting random strangers in your house to level out income/racial/gender inequities.

Of course the standardized tests are the best indicator of collegiate performance, and of course you will miss inspired candidates by only looking at test scores. Its a numbers game, and people pay a high price to come to my college, because every student gets a well paying job, because every student has a good grasp of the material and we built a reputation for high standards. If I accept unqualified candidates then my reputation and profit will suffer, because im dumping thousands of subpar kids into the market with papermill degrees.

"Oh no, it was always my dream to go to harvard, its not fair!" Like fuck-all is fair in life.

BeetleB3• 2 years ago

If I ran a college, im accepting whoever I decide, because its mine. Imagine accepting random strangers in your house to level out income/racial/gender inequities.

You probably can get away with it, as long as you don't want to be credentialed and don't want any federal funds.

Geisterde3• 2 years ago

Of course, if were we in a time when those policies I would be arguing that they are immoral and regressive, but of course you are correct.

wnc3141• 2 years ago

We may nitpick at which process gives more equitable chances of admission to Dartmouth. But ultimately we are still saddled with the absurd notion (and very real prospect) that a ticket to the middle class depends on winning very costly degrees. - particularly degrees whose brand recognition enables employment with greater proximity to the deployment of capital rather than operation of capital.

Meanwhile there is little discussion of why issues of equity in admissions are zero sum rather than leveraging multi-billion dollar endowments to expand access overall.

sokoloff4• 2 years ago

Education scales to a point, but doesn’t scale infinitely. Professors are limited; labs are limited; dorms are limited; campus real estate is limited.

Some of those are amenable to short-term growth; others are not.

closeparen2• 2 years ago

Endowments are used to fund scholarships for students from middle-class and lower backgrounds. The degree is only costly to the extent that your parents are to well-off or your school's endowment is too small. (This is a serious problem if your parents are well-off but stingy, or if you get into an elite-ish but not actually top-tier college).

Endowments, like universities, are meant to be perpetual, so the amount of access they can provide is limited by the return on capital. Some people would perhaps like to see the principal spent down, which could provide dramatically more access in the short term, but then the money would be gone.

devwastaken• 2 years ago

The HN algorithm for comments is not prepared for the 90% problem. One poor quality comment receives the majority of the replies, putting the poor quality first in the list.

spoonjim4• 2 years ago

As the universities attempt their cute little social science experiments there is probably a lot of opportunity to create a new university whose admission criterion is simply a 1600 SAT score

RecycledEle• 2 years ago

Standardized test scores are objective qualifications not subject to racist prejudice.

Racist prejudice invades every step of admittance to college, subjective grading, scholarships, hiring, promotion, and firing. Therefore qualifications like education, worke experience, references, and salary history are racist and must be disregard in favor if objective qualifications.

A fair employer would only look at SAT, ACT, and industry certifications that are awarded solely based on passing an exam.

Report on the role of standardized test scores in undergraduate admissions [pdf]

Related Stories