U.S. News pulls Columbia University from its 2022 rankings

https://www.forbes.com/sites/michaeltnietzel/2022/07/07/the-fallout-continues-us-news-pulls-columbia-university-from-its-2022-rankings/

by selimthegrim • 4 years ago

440 336 4 years ago

forbes.com

dataphyte• 4 years ago

US colleges foresee a future where they are under attack for being absurdly selective, what some are now calling being "highly rejective". I think they want to avoid being targeted by the fairness wonks anymore than they already are.

Also, US News and World Report rankings have major problems, for instance, the ranking algorithm considers campus aesthetics and food quality, but doesn't account for price, giving schools incentive to raise prices to fund campus improvements that boost rank, in turn boosting applications, in turn reducing % acceptance (the principal indicator of "quality" in US schools). This was how TCU went from a meh christian school to a "selective private college" in just a couple of years. World Report now publishes a "best dollar value" report to account for this, but few read it.

duk• 4 years ago

"the fairness wonks"

peanuty12• 4 years ago

the ranking algorithm considers campus aesthetics and food quality.

wtf

Fomite4• 4 years ago

Having eaten some absolutely miserable dorm food in my time, I get it.

WillPostForFood3• 4 years ago

It is not true.

https://www.usnews.com/education/best-colleges/articles/rank...

jseliger• 4 years ago

The gap between the schools' rhetoric and reality is pretty funny; I was just writing about it the other day: https://seliger.com/2022/07/06/nonprofit-boards-of-directors...:

You can see a lot of hypocrisy that’s uncritically accepted by a lot of organizations, including nonprofits. Exclusionary higher education is a particular notable example, given the soaring rhetoric of “inclusion” spouted by some people involved with higher ed, versus the reality of those same schools seeking to reject as many applicants as possible. Princeton University’s president, Chris Eisgruber, has, for example, blathered extensively about the school’s efforts to “combat systemic racism.” Princeton has a $37 billion endowment. The school’s undergrad acceptance rate is 5.6% and it charges a sticker price of $73,000 a year (yes, the school does accept a handful of token low-income students every year, but that the school’s overall demographics reflect its target: the wealthy). Does that sound like a school devoted to combating systemic racism to you? How can people make these kinds of arguments with a straight face? Colleges and universities are run largely for the benefit of their administrators. The other exclusionary schools are doing the same things, as are their private-school feeders, despite their vigorous marketing to the contrary.

Regarding the above paragraph, let me be clear: describing how something is, is not the same thing as approving of it.

rayiner6• 4 years ago

It's truly mind-boggling. The Ivies were always bastions for maintaining the position of WASP elites. They still serve that function. Maybe there is a bit more wiggle room about skin color, but they still function to socialize the next generation of people to run WASP institutions like "JP Morgan Chase."

pyuser5836• 4 years ago

A big problem: admitting too many students actually decrease the number who apply.

kart23• 4 years ago

Is that really so wrong though? I'd like to go to a college where the food doesn't suck and the campus is nice. Those are definitely factors that went into my choice. Value was definitely the overwhelming factor in my decision though, I went to a UC.

Also, even the value rankings are mostly just Ivies at the top. Funny enough, my school is in the ~30s for overall rankings, but in the low 100s for value, since they're looking at the out-of-state cost (which for UCs is around 50k vs. 15k in-state), and they include grants, which is extremely variable by individual student. If they looked at the in-state tuition, UCs would dominate the value rankings.

https://www.usnews.com/best-colleges/rankings/national-unive...

WillPostForFood3• 4 years ago

US News rankings considers neither food nor campus aesthetics.

https://www.usnews.com/education/best-colleges/articles/rank...

burlesona• 4 years ago

Per the article this came in response to questions raised by a Columbia faculty member critiquing the university's data. I thought that report[1] was actually more interesting than the original article.

As per the quote at the top of the report:

Rankings create powerful incentives to manipulate data and distort institutional behavior for the sole or primary purpose of inflating one’s score...

It would be shocking to me if Columbia was the only institution that fudged its numbers a bit in pursuit of a higher US News ranking. Given how important that ranking has become in US culture, I wonder if it's time for the methodology to change to require audited/auditable metrics rather than continuing to allow self-reporting.

1: http://www.math.columbia.edu/~thaddeus/ranking/investigation...

hackyhacky• 4 years ago

Your suspicion is correct.

I've taught at several US universities, and it's an open secret that the administrative takes steps to manipulate numbers in order to secure better rankings.

Typical example: a small, prestigious liberal arts college always has smaller class sizes in the fall semester than in the spring semester. Why? Because classes with less than 30 students are ranked higher by US News, but they only take into consideration the fall semester. No one cares about class sizes in the spring.

CogitoCogito2• 4 years ago

Because classes with less than 30 students are ranked higher by US News, but they only take into consideration the fall semester. No one cares about class sizes in the spring.

What a bizarre and totally unnecessary restriction.

bogota• 4 years ago

Great to know many peoples graduations were delayed so someone could game some numbers

nostromo• 4 years ago

Gaming and manipulation should absolutely be addressed, but it seems Columbia was just straight up lying about their numbers, not gaming them.

lumost3• 4 years ago

Why is small class size a good thing? Anecdotally, you get better class material and TA staff in larger lectures. Why aren't Universities able to innovate here?

greymalik• 4 years ago

Goodhardt’s law in action.

pishpash• 4 years ago

What's that adage? Every metric that becomes an optimization target ...

eduction• 4 years ago

I attended a well known and regarded public university. I was admitted for spring, and for the preceding fall entered a program where I could live in a regular university dorm and take classes for full credit taught at the same university’s extension, but special extension classes just for this program (so all spring admit students). Then you stay in the dorm in spring as you become a “regular” student.

I always suspected it was a way to fudge numbers, but at the same time I knew I was genuinely on the bubble of being admitted so I took the opportunity gladly.

(My memory is that the other spring admits were, like me, from racial groups considered “over represented” so they may have been trying to affect diversity numbers too.)

bredren• 4 years ago

I TA'd in the attendance office at my high school back in the day.

The city's tracking of student attendance only lasted a portion of the year or a portion of each quarter.

During those times, much more effort went into calling the homes of missing students and finding out why they were not showing up to class.

I remember asking about why all the calls were happening or not happening and it was sort of quietly explained by the staffer: measurement of attendance affected how the school was being evaluated by the district. Presumably, budget was hooked to it in some way.

I didn't make any waves about it because "working" in this office was super powerful and the staffer always brought in cookies for us. Amazing gig.

Fomite4• 4 years ago

Similarly, at a large land grant university, we talk a lot about improving our rankings, but acknowledge that one of the major metrics (% of students not accepted) is something that's contrary to our mission.

It was an open secret when I was in Boston that Harvard actively encourages people to "give it a shot" when they know full well they're not getting in to boost that metric.

mbesto• 4 years ago

Which begs the question - if you want to get top in rankings wouldn't you simply design the school around the objective ranking parameters and nothing else?

Ripe for abuse...

nonameiguess• 4 years ago

Not sure if this is still true, but at one point, the biggest factor in the ranking was peer reputation rating. That kind of results in a self-reinforcing vicious cycle of top schools remaining top schools, but short of bribery, there isn't really any way to manipulate that at least.

colinmhayes5• 4 years ago

This is sort of what Columbia did… they juiced their faculty numbers by counting the med school which has much lower class sizes and more educated teachers.

nus07• 4 years ago

I have a friend who works at Duke University’s marketing analytics and performance metrics team. It’s filled with MBAs who’s job is to massage the numbers and metrics that are asked by US News every year and provide them to US News.

noipv4• 4 years ago

username checks out ;) you guya have a collab prog with duke

onionisafruit• 4 years ago

It's probably staffed with MBAs to boost the job placement numbers for their MBA program.

mtlmtlmtlmtl• 4 years ago

This is a good example of Campbell's Law at work: https://en.wikipedia.org/wiki/Campbell%27s_law

ethbr0• 4 years ago

An obvious takeaway in this case: US News shouldn't be the primary ranker.

If statistics were distributed in open format, transparently, and there were multiple rankers (each with their own weights and mixes), universities would be less incentivized to game "the US News metrics".

t_mann3• 4 years ago

Thanks for sharing. Sounds very similar to Goodhart's law:

https://en.wikipedia.org/wiki/Goodhart%27s_law

twobitshifter• 4 years ago

It’s like doping in cycling. Everyone was getting caught but we’re supposed to believe the couple who snuck through weren’t like the rest. Once the tests got good enough there was hardly any one left standing.

https://en.m.wikipedia.org/wiki/Doping_at_the_Tour_de_France

gofreddygo2• 4 years ago

Funny how this reminded me of a PG essay[1]

    The most damaging thing you learned in school wasn't something you learned in any specific class. It was learning to get good grades.

[1]:http://paulgraham.com/lesson.html

bdowling2• 4 years ago

The problem with that quote is that if you don't learn to get good grades (e.g., because you are actually learning and not studying for the test), then the resulting bad grades can be extremely damaging to your future.

hardtke4• 4 years ago

My kid just went through college selection process and I can assure you that the US News rankings are largely irrelevant in terms of decision making. The revealed preference score[1] or global university rankings[2] are much closer to how students, their counselors, and their parents rank the universities. For instance, any ranking that does not have Stanford first among non specialized US universities (i.e. excluding MIT, Caltech, Julliard, etc.) is suspect.

[1] https://www.parchment.com/c/college/college-rankings.php?pag... [2] https://www.topuniversities.com/university-rankings/world-un...

brianbreslin• 4 years ago

I work at a university, and honestly have never heard of this ranking that you referenced in [1]. Also seems super weighted towards military academies. What's their model? Are kids in the states now looking at more global options ?

t_mann3• 4 years ago

I think it's easy to underestimate the impact of rankings. Just from your description, it seems that one of the most important overall decision factors for you seems to have been some sort of vague personal sense of how you'd rank them. That subjective ranking is probably influenced by a lot of factors, including many rankings that you consciously claim to disregard.

rayiner6• 4 years ago

For instance, any ranking that does not have Stanford first among non specialized US universities (i.e. excluding MIT, Caltech, Julliard, etc.) is suspect.

LOL wut? Stanford is modestly well known, but obviously Harvard is the one that is unambiguously first.

gnicholas4• 4 years ago

I find revealed preference to be very useful in many situations, but a ranking that has Harvard just below Swarthmore, at #61, does not pass the smell test. I say this as a graduate of Swarthmore — I cannot take seriously a list with Harvard so far down.

pyuser5836• 4 years ago

It’s really weird Cambridge is ahead of Oxford in world rankings.

Cambridge has a reputation for strongly preferring UK students.

aiisjustanif2• 4 years ago

Adam Ruins Everything, Season 2, Episode 7 [1]

1: https://youtu.be/EtQyO93DO-Q

colinmhayes5• 4 years ago

I took an Econ class with northwestern’s president and he said that they straight up lie about their middle 50 gpa and standardized test scores. Can’t say I was super surprised. Every top 25 school is saying 25% of their students got a 35 or better on the ACT, the numbers just didn’t add up.

rayiner6• 4 years ago

Morty? He was getting pretty chatty nearing retirement.

whimsicalism7• 4 years ago

Maybe it is because they "super score" (ie. take the best of each section if they take it multiple times).

humanistbot2• 4 years ago

See https://en.wikipedia.org/wiki/Goodhart%27s_law

xdavidliu• 4 years ago

this is also paraphrased in the paper by the Columbia professor.

Almost any numerical standard, no matter how closely related to academic merit, becomes a malignant force as soon as universities know that it is the standard. A proxy for merit, rather than merit itself, becomes the goal.

lettergram2• 4 years ago

I FOIA requested a lot of universities and man do they fight to keep their stuff secret.

https://austingwalters.com/foia-requesting-100-universities/

Often I’d find the person in charge on linkedin and email them directly (work email obviously). And dig around until I found the particular person.

UIUC put up a particular fight when I FOIA requested information about their post-graduate surveys. They’re definitely manipulating there. I was in the process of filing a lawsuit when covid hit and gave up because I wasn’t sure if I’d need the funds (plus court shutdown at the time).

dragonwriter• 4 years ago

I FOIA requested a lot of universities

FOIA only applies to federal government agencies, universities (other than ones operates by the federal government, like the service academies) aren't subject to it.

State universities might be covered by state sunshine laws that are similar to FOIA, and generally the laws and policies around those only require a request from the public and aren't too particular about form or citing a completely wrong and irrelevant legal basis, so in that case an “FOIA” request might actually have some legal effect. But most private universities would just see it as a particularly pompously written discretionary request.

MatthiasPortzel• 4 years ago

From the conclusion of that professor's article:

No one should try to reform or rehabilitate the ranking. It is irredeemable. In Colin Diver’s memorable formulation, “Trying to rank institutions of higher education is a little like trying to rank religions or philosophies. The entire enterprise is flawed, not only in detail but also in conception.”

A lot of HN commenters here trying to "fix" the details of the rankings.

bagels• 4 years ago

I'm really curious what the personal cost to that faculty member is. Surely this report was not well liked by the administration.

hprotagonist• 4 years ago

tit for tat: http://www.math.columbia.edu/~thaddeus/ranking/investigation...

function_seven• 4 years ago

In the same way that one restaurant flouted Yelp ratings[1], couldn't all the Ivy League schools just refuse to participate in US News' annual rankings? What would happen if Harvard, Yale, Princeton, Columbia, et. al. decided that this ranking is not helpful, so they won't supply any info?

Readers of US News would more likely start to lose trust in the rankings rather than move their assessments of the Ivies downward, right?

[1] https://arstechnica.com/information-technology/2014/09/why-t...

ketzo2• 4 years ago

But it’s a prisoner’s dilemma situation, right?

If none of the Ivy’s are on US News, well, it’s probably a shitty ranking.

But if e.g. just Penn falls off the list… most people are just gonna assume that Penn got worse.

mlinsey• 4 years ago

This is why it is really just Harvard (likely Stanford, Yale, and Princeton too, but definitely Harvard) that can make a statement boycotting the list, just like a billionaire can get away with wearing a plastic $10 watch but mere millionaires feel the need to wear a Rolex to show off their money. In fact, it wouldn't surprise me if Harvard did the least to game the US News metrics of any top 25 university, because the US News staff would just tweak their formula to keep Harvard in the top 5 regardless.

lbarrow• 4 years ago

These rankings consistently tell people that Harvard, Yale, Princeton, etc are the best schools in the country. Why would they boycott rankings that praise them?

genericone2• 4 years ago

Because it puts other 'less desirable' non-elite schools in the same lists for top twenty/thirty/forty, etc.

bee_rider6• 4 years ago

Yeah they should stick all the ivies, MIT, etc... in a top "non-rankable" rank. Like realistically a student will be happy to go to whichever one they get into, and if someone gets into multiple, they probably won't pick based on position in some list.

ketzo2• 4 years ago

if someone gets into multiple, they probably won't pick based on position in some list

Not trying to be contrarian, but I had two friends literally pick between Ivies based on this very list (in 2015).

It’s generally accepted as “The List” by a lot of people. If you’re a parent with no other frame of reference, The List has a serious impact.

twblalock• 4 years ago

The US News rankings were already a bit of a mess, but leaving out one of the Ivy Leagues is only going to be bad for US News.

An Ivy League school does not need help from the rankings -- but rankings that leave out one of the most prestigious and well-known universities in the country are useless.

prepend• 4 years ago

An Ivy League school does not need help from the rankings

I tend to agree, but why then did Columbia fudge the numbers?

I think there’s some envy from the “lesser ivys” that they wish they were Harvard or Princeton. A high USNews ranking proves little, but I guess Columbia wanted to be tied with Harvard.

rr8882• 4 years ago

Ivy league is kinda obsolete now, esp with places like Stanford, Duke, John Hopkins better known than Dartmouth, Brown, Cornell.

genericone2• 4 years ago

Columbia incentivized administration MBAs to increase the rankings, perhaps attached to bonuses. They did it unscrupulously by fudging the numbers and so Columbia will hopefully will dis-incentivize the behavior in the future, and so perhaps will other schools... doubt it.

bee_rider6• 4 years ago

I wonder how much these measurements actually matter, for a place like Columbia. I mean, I can see why, like, people might want to look at the list and see "hey this school I've never heard of is actually OK."

But like, people don't find out about Columbia from this list, right? It is, uh, older than the country "U.S. News" is named after and an Ivy League school. Not having them on the list just reflects poorly on the list.

willhinsa• 4 years ago

On the other hand, Coca-Cola spends 4 billion of dollars every year on advertising.

bee_rider6• 4 years ago

Yeah.

I dunno, nothing about advertising makes sense to me, so I guess I have trouble evaluating this kind of thing.

Common sense would tell you that you don't give 4 billion dollars to the guy who says "I'm really good at convincing people to part with their money -- whether they should or not!" But then I don't run Coca Cola, so I guess they've figured out something I've missed.

toast0• 4 years ago

In 2010, Pepsi switched at least their superb owl advertising budget to feel good community projects and lost sales at 6% vs a 4.3% overall decline for soda. Advertising drives sales, even if you can't tie a sale directly to an ad.

peanuty12• 4 years ago

I imagine there is value in being in the same company of more prestigious schools like Harvard and Princeton.

ladberg• 4 years ago

I absolutely know people who have had the choice of multiple top-tier schools and chose the one that ranked higher on U.S. News, so it definitely matters.

zactato• 4 years ago

Why?

MIT vs Harvard vs Stanford are all amazing programs but are better for different types of people/interests/goals

rel2thr• 4 years ago

Not really , Knowing about Columbia is kind of a new york, upper class thing

They don't have good sports teams, no way a person in middle america would have heard of them if not for school rankings

bigDinosaur• 4 years ago

Interesting, because they're a fairly famous university internationally - Obama was a graduate from there etc.

bee_rider6• 4 years ago

It might not be a good spot for people who pick their school based on sports leagues, but I mean... surely that's not everybody in Middle America.

throwawayzXwEt• 4 years ago

These rankings are rampant with manipulation. For instance, I heard of classes at Hopkins being taught half in-class, half pre-recorded, so that they could say the "classroom" size was half its actual enrollment. It's especially frustrating when this kind of manipulation actually hurts students' education.

endisneigh• 4 years ago

Unless a school is willing to accept a entirely random set of students all schools success or lack there of can be chalked up to selection and survivor bias.

I’d have to find it but someone did an analysis of folks who got into elite schools and didn’t go for whatever reason and saw that by mid career they had more or less the same outcomes.

Thiel Fellows entire premise is around this effect, as well.

labster2• 4 years ago

One down, 99 to go. These university rankings are absurd, and just a status game that school administrators feel they are forced to play. Individual programs may be outstanding at otherwise middling colleges — it’s better to learn about specific departments than try to reduce the school as a whole to a number.

kupopuffs3• 4 years ago

Aren't there rankings for specific schools in schools? like Engineering, etc...

labster2• 4 years ago

Sure, but excellence in computer engineering does not imply excellence in chemical engineering does not imply excellence in civil engineering.

unosama2• 4 years ago

Universities are a status game in general. Reputation and networking opportunities matter more than education quality.

culi• 4 years ago

One of the heaviest part of their grading is just the opinions of academics. They just send a survey out and ask academics to rank universities. Of course they end up reproducing and reinforcing the existing rankings as always.

dang• 4 years ago

An Investigation of Columbia's U.S. News Ranking - https://news.ycombinator.com/item?id=30603287 - March 2022 (46 comments)

somethoughts• 4 years ago

Somewhat related - How Northeastern Gamed the College Rankings

https://www.bostonmagazine.com/news/2014/08/26/how-northeast...

There are probably some significant parallels to any executive who manages to game the compensation structure at any large corporation to their financial benefit.

kens• 4 years ago

There's a very interesting article "How to Game the College Rankings" about how Northeastern University's president focused on improving the university's ranking. In 1996, it was a "third-tier, blue-collar, commuter-based university" rated #162. The new university president had a singleminded goal: to improve the ranking. He got the university into the top 100 in 2006 and into the top 50 by 2013.

https://www.bostonmagazine.com/news/2014/08/26/how-northeast...

saalweachter• 4 years ago

Now the interesting question: did Northwestern^H^H^H^H^H^H^Heastern get meaningfully better for students between 1996 and 2013?

Did they graduate in higher numbers? Did they have better career outcomes? Did more get into better(?) grad schools? What is their median income? Are they happier, did they enjoy college more?

myko• 4 years ago

Northeastern.

Northwestern was already relatively prestigious.

zerocrates2• 4 years ago

Just a note that this is Northeastern (in Boston) not Northwestern (in Illinois). Northwestern was and is typically a very high ranker.

As for what the improvements are like... hard to say. Some ways to improve the rankings like getting more students you don't want to come to apply (so you can appear more selective) don't actually help at all, while things like small classes even if done to "game" the rankings could actually help.

There's also just the cyclical aspect of things: the quality of your experience is going to be significantly determined by the quality of the student body, so if you have a better applicant pool because of your better rankings you maybe have a bit of a virtuous cycle that's somewhat disconnected from whether the things you changed actually improved anything real on their own other than the ranking.

kaesar14• 4 years ago

Was it that hard to read and get my alma mater right? ;)

In my opinion as a student who entered Northeastern in the mid 2010's the answer to all your questions is an unequivocal yes. More competitive student body, better profile to attract employers, more student amenities to enhance on campus life (we could have more in this regard imho). Compared to the commuter school of yesteryear it's for sure a much better educational experience.

ameliaquining• 4 years ago

Almost certainly yes, because the school could presumably attract a higher-quality applicant pool in 2013 than in 1996. This, of course, makes it impossible to tell how much of that value was produced by the university and how much was just selection effects.

pyuser5836• 4 years ago

Most colleges have the ability to graduate close 100% of their students.

If a school only admits students who have at least two years at another college with high grades, their graduation numbers will skyrocket.

A highly selective college, like Harvard, could even refuse to grant credit for those two years.

There are less nefarious ways of selecting a class of likely graduators: leadership in extracurriculars is highly correlated with post-secondary graduation.

Foreign students with limited English fluency have very low graduation rates. They could (should?) be excluded.

nerdponx• 4 years ago

Subjectively, I imagine that it did. Northeastern has had a good reputation for as long as I've been aware of it.

raygelogic• 4 years ago

I've worked with a number of northeastern university grads, and they are consistently well prepared. NEU has co-op program, a standard part of their undergrad, where they get like a year of experience before graduating (5-year undergrad is the norm). who knows if the school is what creates the difference, but grads are really well prepared in general.

throwaway0a5e• 4 years ago

Did they graduate in higher numbers? Did they have better career outcomes? Did more get into better(?) grad schools? What is their median income? Are they happier, did they enjoy college more?

And make sure you control for your inputs. It's harder for the #2 campus of state school to wring "good outcomes" from the children of plumbers and truck drivers than it is for big brand name schools that have plenty of kids of doctors and lawyers in there to drag their average up because even if you do a middling job educating them and have squat for industry connections they will still land internships and go on to get good jobs.

pcurve• 4 years ago

#49 now. I guess it's harder to game the system past certain threshold

ausbah• 4 years ago

speaking as a as-of-this-year Northeastern CS graduate

the university definitely has done things to improve its rankings over the past ~30 years, some legitimately good (hiring higher quality staff, putting more emphasis on research, building more student accommodations like on campus housing, expanding the co-op program) and also less than savory moves (the NUIn program which admits more freshman students without having to record their statistics in the incoming class record, international campuses to "build the brand"). I think, at least the CS and co-op programs are great, and - minus the cost - is a overall great school. they definitely had the aim of improving the ranking, but in doing that they did improve the quality and size of the school along many important fronts. my read is then that USNews still has use in measuring universities, up to a certain point (say top 50 or top 100)

beowulfey• 4 years ago

I was at NEU from 2007-2012. Aoun, who became president in 2006, was not popular amongst the students. It was pretty apparent that he was making changes simply to make things appear better on paper, and some of them actually made the students really angry. A big one that I remember was that they eliminated the mandatory co-op requirement (which means most students graduate in 5 years instead of 4), which is a core tenet of NEU's philosophy!

The changes NEU made between 1996 and 2006 were seemingly genuinely good for students. My dad had graduated from NEU in '86, and he was really impressed with how far the school had come. But I wouldn't say anything that changed while I was there had any real impact on my experience, so my gut instinct says they were more gamey than anything.

dwater• 4 years ago

For context, U.S. News and World Report used to be a trusted periodical that reported national and international news. In the 80's they started ranking colleges and selling it as a separate publication, which turned out to be pretty profitable, so they repeated the trick by ranking hospitals (1990), cars (2007), and then states (2017). The news business went online only in 2011, then shuttered in 2015. Since then they've just been a rankings company, but they kept the name to trade on their reputation.

cuteboy192• 4 years ago

I feel like it would not be possible to sell such a static and easily leaked thing as a ranking list

uptownfunk• 4 years ago

Wow fascinating - so US News is now a rankings company.

xanaxagoras• 4 years ago

If you think of clickbait as a crowdsourced ranking of emotionally manipulative hyperbole, that could be said of any corporate press rag.

UncleOxidant• 4 years ago

U.S. News and World Report was a highly respected & trusted news magazine in the 70s-90s, but it's online incarnation now is just another clickbait site (similar for Newsweek).

bluedays• 4 years ago

They’re the #1 best rankings company.

kristopolous• 4 years ago

I remember reading the paper magazine for school in the 1990s. Teachers would assign parts as homework sometimes. It was on that level of trusted journalism

I don't know what the equivalent today would be. The Reuters website maybe?

rendall• 4 years ago

I had first heard of them because Zork had a US News and Dungeon Report magazine, and it was such a specific name I knew it must refer to something in the real world. I never did get a chance to read the actual magazine, though.

data4lyfe• 4 years ago

The fact that U.S. News and World Report can pull a school from it's own rankings and it makes such a strong impact is really a strange phenomenon.

epgui• 4 years ago

For all of the ways we can easily criticize the incentive structures and possible intentionality behind these bad numbers (which I agree merit scrutiny), I think it's worth noting and remembering that the issue was flagged by a mathematics professor of the same university: there is at least some well-aligned incentives, some measure of self-regulation, with respect tenured positions and academic freedom.

danielmorozoff• 4 years ago

Having pursued a PhD at Columbia and having taught classes there. I am surprised it took this long for someone to speak up. Also the fact that there are no checks in place to certify top ranking academic institutions is fascinating.

xiaolingxiao• 4 years ago

could you expand on what you mean please?

taveras• 4 years ago

If folks are curious about how schools are ranked, I highly encourage you to listen to the following podcast:

https://www.pushkin.fm/podcasts/revisionist-history/lord-of-...

saagarjha• 4 years ago

Kind of sad that US News decided to unrank just Columbia rather than suspending the whole thing. If one college games your ranking to the point that a professor from the university is like "hey you guys this seems pretty sus", it's probably true that all the other universities have the same problem and just don't have anyone willing to talk about it. As it stands it seems like just Columbia did something wrong, when they are the only university that did something wrong but also had someone who did something right.

wombat-man• 4 years ago

Well, it's kinda their main business at this point.

naniwaduni3• 4 years ago

Suspending the whole thing would be suspending pretty much US News's entire remaining raison d'être at this point.

lofatdairy2• 4 years ago

If I'm not mistaken the news that Columbia "won't participate" in the rankings came a bit sooner[^1], but if anything that struck me as a "you can't fire me, I quit" sort of preempting of what was inevitable.

Honestly though, college rankings are toxic, as is this whole prestige economy we've built around them. I don't even really care about colleges faking numbers, so much as other shady practices like sending students false promises to get their hopes up - pretty much for the express purpose of rejecting them down the line. As any unfortunate student who has either had to apply to college, or been the parent of someone applying, you can't avoid the "chance me" threads or the endless HYPSM dick measuring contest while trying to get legitimate information.

[^1]: https://www.nytimes.com/2022/06/30/us/columbia-us-news-ranki...

raldi• 4 years ago

Columbia announced they wont participate in next year’s rankings. The delisting was for this year’s report.

lpolovets• 4 years ago

The influence of US News on colleges is wild. One absurd tidbit that I've heard: the rankings weigh freshman stats much more than transfer student stats, so a lot of competitive schools that won't let you in as a freshman if you would hurt their SAT/GPA/etc averages will happily let you in as a transfer because they'll make more revenue and your stats no longer hurt their rankings. *

The ranking formula is also changed arbitrarily based on the reactions of colleges and readers. My school went from #9 to #1 one year, there were a lot of "wth??" reactions, and the following year the formula was tweaked so that our ranking dipped down to #4. I'm pretty sure nothing changed materially in the school over those 3 years, but the ranking moved around a lot.

The whole system is a farce.

* I haven't researched freshman vs transfer data myself, but it's something I've heard from multiple startup founders in the college education space.

ffguuficc• 4 years ago

For UW this is 100% true. A 4.0 is like a 50/50 to get in the CS program but for transfers it’s easier.

whimsicalism7• 4 years ago

Depends on the school. At a place like Harvard or Yale it is incredibly difficult to get in as a transfer, much harder than to get in in the first place.

caddemon2• 4 years ago

Yeah but that's Tier 0, it's not surprising that HYPSM don't feel the need to engage in as many number fudging tactics. It's a lot more impactful to Tier 1 top schools (e.g. a Dartmouth) where USN ranks them.

rr8882• 4 years ago

Yeah this was one of the whistleblower topics. Columbia has highest number of transfers http://www.math.columbia.edu/~thaddeus/ranking/investigation...

oneplane• 4 years ago

Wouldn't it make more sense to just make sure that all universities just pass the same requirements to be a university, and beyond that not add some ranking or other entertainment nonsense in the mix?

This whole entertainifying business really isn't good for anyone, except perhaps for people making money off of numbers, which is still not good for most.

gumby3• 4 years ago

I really wonder why Columbia even cares: it's a globally well known ivy league school obviously producing a good education and is in NYC.* Doesn't US News need Columbia more than the other way around?

* No I have no connection to the school except I do have some friends who teach there and some others who attended decades ago.

lofatdairy2• 4 years ago

I think professors and the actual scholars there don't really care. However, administration cares because ranking is tied to both how many undergraduates students apply and how much they can charge these students. If college rankings didn't matter financially then I hazard that Columbia would've had questionable numbers to begin with.

That said I agree that being an Ivy in NYC certainly was a major contributor to its more recent rise in the rankings and applications.

colinmhayes5• 4 years ago

Having high ranking is like the entire job of administrators. Especially at Ivys they’re selling prestige, being highly ranked gives you prestige.

gumby3• 4 years ago

If you're in the small top 'n' I think the prestige goes the other way. Will anyone care if MIT is +1 or -1 some year? I assume those top schools can just run themselves the way they want and still get highly ranked.

And if US news were serious they could simply have kicked columbia out for a year, but it would have hurt them to do so. They'd probably do it for a small college ranked 100 or below though.

I'd be embarrassed for Columbia that they gamed the numbers. Outside that small n though, I presume gaming the numbers is commonplace.

sam-2727• 4 years ago

The beginning of the conclusion of the original study [1] is worth repeating:

No one should try to reform or rehabilitate the ranking. It is irredeemable. In Colin Diver’s memorable formulation, “Trying to rank institutions of higher education is a little like trying to rank religions or philosophies. The entire enterprise is flawed, not only in detail but also in conception.”

Students are poorly served by rankings. To be sure, they need information when applying to colleges, but rankings provide the wrong information. As many critics have observed, every student has distinctive needs, and what universities offer is far too complex to be projected to a single parameter. These observations may partly reflect the view that the goal of education should be self-discovery and self-fashioning as much as vocational training. Even those who dismiss this view as airy and impractical, however, must acknowledge that any ranking is a composite of factors, not all of which pertain to everyone. A prospective engineering student who chooses the 46th-ranked school over the 47th, for example, would be making a mistake if the advantage of the 46th school is its smaller average class sizes. For small average class sizes are typically the result of offering more upper-level courses in the arts and humanities, which our engineering student likely will not take at all.

[1] http://www.math.columbia.edu/~thaddeus/ranking/investigation... (section 8)

lazyjeff2• 4 years ago

I've been looking at the bias in rankings for a little while. I think one way to identify and raise awareness of the biases, is just put rankings together side-by-side. I did this for computer science programs, and there's some interesting differences that I noticed:

https://jeffhuang.com/computer-science-open-data/#bias-in-co...

evouga• 4 years ago

The focus on best paper awards is odd as the major conferences of some CS subfields dole out awards as if they were party favors, and those of other subfields don't have best paper awards at all.

For instance SIGCHI 2021 had 28 best papers out of 747 accepted papers (or 3.7%) whereas CVPR 2021 had one best paper out of 1660 accepted papers (0.06%).

I have no opinion about whether it's "better" to be stingy or generous with best paper awards. But obviously any kind of ranking that doesn't account for differences between conferences and subfields is going to be quite suspect.

cuteboy192• 4 years ago

This is just the wrong way to look at it. Clearly, there is real demand for rankings by students. No one is stupid enough to think that there is some real difference between #47 and #48. But obviously #47 is very different from #26.

Just because you can't get an exact measurement does not mean that a metric does not exist or is not useful.

brewdad7• 4 years ago

I would argue that the difference between 47 and 26 comes down largely to field of study or cost of attendance.

The top 10-15 offer an almost indisputable advantage with the top 5 or so being a tier unto itself. Anything outside of those groups is largely "it depends" and 50-100 forms another tier where total cost of attendance largely dictates whether one school is "better" than another.

thomasahle• 4 years ago

Trying to rank institutions of higher education is a little like trying to rank religions

Did anyone try to make a ranking for religions similar to those for universities? Sounds like it would be a fun project making a great point.

Measures like "Fees", "Diversity", "Alumni Salary", "Financial Aid Provided" would all be interesting to see for each religion.

physicles• 4 years ago

I’d love to see that too. Mormonism for example would rank very high in “Alumni Salary” and “Quality of Alumni Network”, but low in “Diversity”, “Fees”, and “Amount of Bullshit You Need to Swallow”.

btheshoe• 4 years ago

Here's the response:

Universities are mostly unimportant in terms of what they offer: class sizes, education, curriculum, etc... All mostly bs that doesn't matter.

The real role of universities is to gather together smart people as they develop. This requires mostly a sort of self-selection of applicants, who need to agree independently to go to the same university. Hence rankings, prestige, and all that nonsense.

waylandsmithers2• 4 years ago

I couldn’t agree more. The schools I got into were ranked something like 12, 13, 25, and 30. I went to 12 for only that reason and always regretted it. Was it my own dumb fault? Of course. I was 17.

HWR_142• 4 years ago

This is completely true of undergraduate studies. There is a very real reason to think that department (not university) rankings in graduate studies matter.

aiisjustanif2• 4 years ago

This has been brought up multiple time now over that last decade. At least this is one step in the right direction.

Adam Ruins Everything, Season 2, Episode 7 [1] 1: https://youtu.be/EtQyO93DO-Q

willcipriano• 4 years ago

If colleges manipulate rankings like this en mass I think we have a solution to the student debt issue.

A FTC review of college advertising would be interesting, if they are selling you a degree with basically no career prospects while proudly displaying these gamed rankings, that sounds like fraud and misrepresentation. Perhaps the students can get a refund?

NelsonMinar• 4 years ago

I'm proud of my alma mater, Reed College, for refusing to participate in the US News rankings since 1995.

resoluteteeth• 4 years ago

It's interesting to think what most universities would do to a student who tried to falsify their data like this.

deepzn• 4 years ago

A reminder that the growth and cost of College administration is one of the primary causes of the massive increase in College tuition across the U.S.

conqrr• 4 years ago

Explains how ASU has always been number 1 in innovation. It's a great school, but no.q for innovation is pushing it too much.

foolfoolz• 4 years ago

a huge shift over the last 10 year and most visible in technical roles is the elimination of college rankings. i used to work at a company that had a ceo mandate “all resumes must come from top 20 schools”. saying something like that now as an engineering manager would raise questions and probably make you look dumb. there’s many reasons for this but the biggest is it’s impossible to do this with a global workforce. this is great for high school students, one less area of pressure

onemoresoop• 4 years ago

https://www.theblueandwhite.org/post/michael-thaddeus

geebee• 4 years ago

One interesting thing to come of these discussions - a lot of the problems with US News methodology arise from the attempt to push certain metrics from 90% to 100%. Class size, faculty with a "terminal" degree, class size, admissions rate. Many of these things are treated by the rankings as "the higher (or lower) the better." As an analogy, I'd use lean muscle mass percentage for elite athletes. Generally speaking, lower is better, and in many sports, all elite athletes are all under 10% body fat. But much below this starts to become harmful, and would eventually be fatal.

I think this is kind of what's happening with Columbia here. For example, Columbia claimed that 100% of the faculty had a terminal degree. Prof Thaddeus (if you read his full blog post) questioned this from two angles. The first was to say it can't possibly be true. The second (which I found more interesting) was: why would you even want this to be 100%? Seems like Columbia has valuable faculty who don't have a PhD or MFA or whatever is the "terminal" degree int he field.

The harm here isn't in wanting a terminal degree for a high percentage of faculty, it's trying to get this to 100%. I'd say that it probably is a good sign, overall, that most of the faculty has a doctoral degree, as this is the main training degree for research and teaching. But the way US news does it, you get more points for 95% than 90%, more points for 99% than 95%. Seems like a lot of harm comes from trying to wring out that last bit, since even if you do generally agree with the value of a PhD for research (or other terminal degree), it does seem like you'd build a stronger faculty overall with the ability to hire 1 in 10 or 1 in 20 based on some other credential.

So much else fits this pattern. Small class sizes.. kind of the same thing. Sure, it's a good sign, but they're only an indicator, not a goal in and of themselves. Low admissions rates, high test scores. All good metrics, but things that can become not just harmful but harmful and maybe even fatal to a university if pursued with a single minded intensity as if total purity in the metric is the goal.

Honestly, overall we just have to reject the US news rankings. I appreciate what Prof Thaddeus did here, and it was a useful and very well reasoned and sourced takedown. But now and then, I realized that there isn't much to be added, just new and interesting ways to say what everyone knows at this point - yousnoozeandworlddistort rankings are pretty well idiotic. I actually think college rankings in general can be useful if managed cautiously and read critically, but this one is just a turkey.

aj72• 4 years ago

Columbia’s downfall began in a single moment in the early 50’s. When I.I. Rabi refused to let a poor Mr. Bader go to grad school in Physics part time so he could help his family out by working. Mr. Bader later became Feynman’s high school physics teacher, and taught him the principle of least action. So everyone knows that the world will still turn without Columbia.

srl• 4 years ago

This is a bit of a non-sequitur (not to mention silly), but I'm curious about the details. Do you have any links to this story of Rabi refusing to let Bader study part-time?

The problem, of course, it that the timing doesn't make sense at all. You say this happened in the early 50's. That can't be the early 1850's, since Bader wasn't born yet. But by the 1950s, Feynman was already long gone from high school. Indeed, he'd already devised path integrals, and so he certainly knew about the principle of least action!

Perhaps you meant 30's?

aj72• 4 years ago

You’re right somewhere in here. I’m certainly wrong about the 50’s. Columbia was a bastion of antisemitism until the 70’s, it had the world’s best Physics Dept. for a long time, but frittered it away. Meanwhile, Cornell built a world-class research university, partly on Columbia’s leavings.

dnissley• 4 years ago

Polymatter did a great episode on the pressures that create outcomes like this in regards to rankings:

https://youtu.be/cQWlnTyOSig

justinzollars• 4 years ago

Rankings do not matter. Unless you are going into Big Law, or if you are focused on staying in academia.

ken47• 4 years ago

What is the point of university rankings? To gratify the human need for someone to dumb down a complex system to a simplistic measure, just so we don't have to live with the "unbearable" uncertainty that one university may not be strictly superior to another?

Once everyone starts optimizing for a metric, that metric ceases to be useful. It's only a matter of how long it takes people to realize that this has happened.

skinnymuch• 4 years ago

just so we don't have to live with the "unbearable" uncertainty that one university may not be strictly superior to another?

Unfortunately, yes

superior

Key word unfortunately

jdnordy• 4 years ago

Just when you think it's time to count hire education out because of bloated administrative spending, overwhelming student debt, and poor technical training for real world jobs, a light shines in the darkness.

I guess there's still hope if higher education can learn from Professors of the likes of Michael Thaddeus. He and others like him make pursing a degree valuable. The administration can go...

Note: read the original document: http://www.math.columbia.edu/~thaddeus/ranking/investigation...

spoonjim• 4 years ago

If Harvard, Yale, and Princeton mutually agreed to withdraw from these rankings it would go a long way towards killing them.

synergy20• 4 years ago

How to stay objective on these rankings?

What about a list of independent|objective tests(for the 100 different majors out there) to rank students from any schools when you graduate that acts like SAT|ACT for jobs? do make it optional though, if employer finds that valuable, more college graduates will take it, it will work better than leetcode I hope.

waylandsmithers2• 4 years ago

I honestly hope this starts an upheaval against the US News and World Report college rankings. That stupid magazine has way too much power in higher education for no reason .

fujidust• 4 years ago

I have an interest in researching CS programs within a specific geographical area in the states but am having trouble finding good data on student outcomes and associated metrics. If this ranking data is bs or gamed, then what’s the best available resource for helping to determine what programs are good, bad or otherwise?

y0ungarmanii• 4 years ago

Lol colleges are more expensive country club than place of higher education. We really shouldn't be surprised with the corruption anymore after we found out how easy it was to get on the rowing team of most of these places.

alexpotato• 4 years ago

This story is not necessarily related to fudging of numbers/stats etc but it does illustrate to what level schools will go to attract students:

- Small liberal arts school

- Is ranked Division 3 in the NCAA

- Realized they are losing students

- They start a football team!

- Why? For a layout of $500,000/year for a coach + staff, using existing facilities (e.g. soccer stadium) etc they get ~100 additional students who want to be able to say "yeah, I played football in college"

100 students * full boat tuition is nothing to sneeze at and is a pretty good deal from an ROI perspective. It's not a small leap from "let's drop some cash for a very obvious improvement to our attendance" to "let's fudge some numbers that no one may ever really look at" for the same outcome.

equivocates• 4 years ago

Cool, now do all the Ivys.

fsckboy• 4 years ago

misleading headline (on the original article)

from TFA: "on Thursday of last week, just before the July 1 deadline for submitting new data to U.S. News... Provost Mary Boyce, said [Columbia] would not submit data this year, adding that the university had “embarked on a review of our data collection and submissions process”

and that is why they will not be in the rankings. It seems Forbes filled out this submission just like Columbia did theirs, to skew overall appearance.

the "scandal" could very well still be the scandal everybody thinks it is, but it seems to be an overstatement to draw the bold lines that Forbes is juicing their article with. And as others have said, unless we know the extent of fudging at other schools it's difficult to judge the severity of this case.

full disclosure, I have no Columbia or ivy affiliation, but I perceive it to be easily within the top 10

WillPostForFood3• 4 years ago

The article is correct. US News kicked Columbia out of the 2022 rankings. Columbia subsequently withdrew from submitting data for the 2023 rankings.

oldstrangers• 4 years ago

I worked for a marketing agency that gets paid a shit ton of money to run ads targeted at the US News voters. Always seemed so stupid to me, its just a popularity contest.

throwawayarnty• 4 years ago

University rankings are mostly bunk because they weigh mostly on perception and lagging indicators. A university can deteriorate for decades while remaining high on ranking.

The only criterion that matters for a student that wants to select a university is the fraction of graduates that go on to do impressive things.

Ice_cream_suit• 4 years ago

"The picture coming into focus is that of a two-tier university, which educates, side by side in the same classrooms, two large and quite distinct groups of undergraduates: non-transfer students and transfer students.

The former students lead privileged lives: they are very selectively chosen, boast top-notch test scores, tend to hail from the wealthier ranks of society, receive ample financial aid, and turn out very successfully as measured by graduation rates.

The latter students are significantly worse off: they are less selectively chosen, typically have lower test scores (one surmises, although acceptance rates and average test scores for the Combined Plan and General Studies are well-kept secrets),43 tend to come from less prosperous backgrounds (as their higher rate of Pell grants shows), receive much stingier financial aid, and have considerably more difficulty graduating.

No one would design a university this way, but it has been the status quo at Columbia for years. The situation is tolerated only because it is not widely understood. "

https://www.math.columbia.edu/~thaddeus/ranking/investigatio...

mulmen• 4 years ago

Sure. But, why not doubt everything else US News said? It’s not like they uncovered their own incompetence. The takeaway here isn’t that Columbia cheated. Everyone cheated. US News failed in their duty and aren’t a worthy authority.

nottorp• 4 years ago

Nice cookie dialog you have there, Forbes. And I was mildly interested in the article. Not any more.

U.S. News pulls Columbia University from its 2022 rankings

Related Stories