FiveThirtyEight articles on the Internet Archive

https://fivethirtyeightindex.com/

by ChocMontePy • 2 months ago

380 74 2 months ago

fivethirtyeightindex.com

ChocMontePy• 2 months ago

Github link:

https://github.com/palewire/fivethirtyeightindex.com

nl• 2 months ago

This is because whoever owns Fivethirtyeight now (ABC?) deleted the whole archive of articles on the site.

bombcar2• 2 months ago

Don't we need more than an index of Archive.org because whomever controls the domain could robots.txt these out of existence if they wanted to?

ycombinete• 2 months ago

Archive.org mostly ignores robots.txt

https://blog.archive.org/2017/04/17/robots-txt-meant-for-sea...

tardedmeme• 2 months ago

It's not about robots.txt but yes, the owners of 538 can just send a cease and desist letter to get them all immediately removed. Many sites that don't want to preserve history have done this already.

nomilk2• 2 months ago

Couldn't figure out why archiving FTE aricles matters, but a quick search yields:

Thousands of FiveThirtyEight articles seemingly vanish from the internet

https://www.editorandpublisher.com/stories/thousands-of-five...

And discussions here on hn:

ABC News has taken all FiveThirtyEight articles offline https://news.ycombinator.com/item?id=48152553

Disney erased FiveThirtyEight (article by Nate himself) https://news.ycombinator.com/item?id=48197703

ricardobeat• 2 months ago

I don’t really know much about it, but remember it as being _fantastic_ journalism every time I encountered one of their articles. As a bonus, great infographics and interactive data visualizations.

defrost3• 2 months ago

For any, like myself, wondering "Who is Ben Welsh" ?

  Hello. My name is Ben Welsh. I'm an Iowan living in New York City.

  I am a reporter, an editor and a computer programmer. My job is to use those skills, together, to find and tell stories.

  I work at Reuters, the world's largest multimedia news provider, where I founded the organization's News Applications Desk. In that role, I lead the development of dashboards, databases and automated systems that benefit clients, inform readers, empower reporters and serve the public interest.

  [...]

~ https://palewi.re/who-is-ben-welsh/

yogorenapan• 2 months ago

Can't believe Ben Welsh is not Welsh, and FiveThirtyEight has nothing to do with Wales

simonw• 2 months ago

Ben is one of my favorite people in the world of data journalism. He's the author of many excellent training courses in the field, including:

- https://github.com/palewire/first-python-notebook

- https://github.com/palewire/first-web-scraper

- https://github.com/palewire/first-graphics-app

dang• 2 months ago

(Submitted title was "Ben Welsh made an index of all FiveThirtyEight articles on the Internet Archive" - we've since changed it)

defrost3• 2 months ago

Cheers for the clarity, that'll help me look less weird wrt above comment to future historians of archived HN threads :-)

TBH I enjoyed looking up Ben and finding out what he's about and done in the past far more than I did just knowing there's a 538 archive on IA.

gib444• 2 months ago

Don't click that link if you have any feelings of inadequacy about your achievements LOL

ChrisArchitect2• 2 months ago

Love Ben but title can simply be: Index of FiveThirtyEight articles preserved by the Internet Archive

buildsjets• 2 months ago

But that would be a false attribution. The Internet Archive did not create the index, Ben did. And the Internet Archive is not hosting the index, Ben is.

ChrisArchitect2• 2 months ago

Ah, yes, could be worded better, fairplay. Point is the Ben attribution isn't needed in that place to avoid unnecessary confusion about who that is etc.

3eb7988a1663• 2 months ago

If I wanted to get the complete WARC archive of 538 - how do you do this in a friendly way? No interest in history tracking, just want the last available version from Internet Archive.

palewire4• 2 months ago

The site includes a CSV of everything I found. You could hand that to Claude Code and ask it to pull down the WARCs

internet2000• 2 months ago

I'm seeing a lot about this. What makes this situation different than any other website going offline?

patcon• 2 months ago

I think it's the fivethirtyeight of of historical significance, and Disney is one of the largest and wealthiest companies on the planet. So it's just kinda like "whoa, this is stratospheric negligence" or "whoa, what is the reason for this... assuming they are not idiots?"

materielle2• 2 months ago

Also, they don’t any plans for the IP, and Nate would’ve paid above-market rate just to take over and preserve the content for posterity. He estimates that they deleted 200,000 hours of human labor.

This is just some Disney suits being extraordinarily petty.

f311a• 2 months ago

https://www.natesilver.net/p/disney-erased-fivethirtyeight

        Here are some numbers roughly in the right ballpark: during the Disney era, which lasted about 10 years, FiveThirtyEight published about 20 stories a week. Let’s say that each story took about 20 hours to produce between research, writing, graphics and editing.3 Do the math, and that works out to about 200,000 person-hours of work that ABC News just deleted.

mold_aid• 2 months ago

In a sense, nothing - and any other website should be archived, too.

In another sense, it's a journalistic source with information and commentary on past elections. Even aside from the political context that muddies the waters around or outright denies results, matters of public discourse on the web should not be ephemeral or subject to the decisions of the publication - they should be archived.

arlattimore• 2 months ago

I'm not a soccer guy, but I still think the piece on Lionel Messi was awesome

https://web.archive.org/web/20140701122958/http://fivethirty...

stinkbeetle8• 2 months ago

Those 2015-16 ones sure aged poorly, I'm reminded of this https://i.imgur.com/6Z9QQj3.jpeg

This is why people don't really buy the "but he had Trump at 30%, you just don't understand statistics" apologist line. Sure he hedged in the dying days of the campaign (a cynic might think to try to protect his credibility), but the tone overall was of a person who comprehensively failed to understand the mood of the country from beginning to end.

Which is a problem because these election predictions are not just pure "mathematical models" and "data driven" like 538 would have had you believe. What mathematical model should be used? What data should and should not be used? At some point those things are based on the modeller's understanding of reality.

f1ay• 2 months ago

I think Nate did a phenomenal job calling out pollsters in that time. Since 538 was predominately a poll aggregator that did tricky stats to rank the reliability of each poll. I remember specifically an interview with him griping about some of the unusual data he was seeing from pollsters that made it look like, and I quote, 'Someone has their finger on the scales'

stinkbeetle8• 2 months ago

Perhaps critiquing statistical methods used by polling was something he was good at. I have no real opinion of his work there, which I didn't pay attention to.

But predicting an election requires a lot more than polling datasets and statistics textbooks. That's the problem that he made himself out to be an election prediction wizard, but really that was off the back of his good prediction in quite a bland and conventional election.

When things got slightly more spicy and reality diverged from his vaunted "models", his "data science" predictably fell in a heap. The worst thing is almost not even that he got it wrong, it's that he seemed incapable of recognizing that present reality was quite significantly different from the past data he had used to build his models. Even after being wrong in so many of these predictions. He just kept churning out these pieces about how Trump was probably finished this time.

materielle2• 2 months ago

He didn’t hedge at the end. Nate always writes the models before election season then doesn’t touch them apart from actual bug fixes. The model actually organically predicted 30%.

I still think that’s about accurate. Maybe it should’ve been 40%.

Everyone forgets that it was a pretty close election. Clinton could’ve won without the Comey announcement.

stinkbeetle8• 2 months ago

I think he did hedge (or "strategically bug fix"). The prediction for Trump went from IIRC around 15 to 30 in the last week or so. It was a big swing, IIRC with a lot of waffle around why it happened but not a lot of verifiable fact.

I still think that’s about accurate. Maybe it should’ve been 40%.

It wasn't accurate. This is something people misunderstand about these predictions. If the 2016 election was held 100 times, Trump would have won 100 times. It's not the same as rolling dice.

These election predictions don't say that. They say something like "the observations I have agree with scenarios that have Clinton winning, 70% of the time". Which is fine and correct as far as his data and model goes, but none of those scenarios were the reality he was trying to predict. They are all just figments of the model though. Getting down to the brass tacks, he predicted Clinton would win, and he was wrong.

Which is fine, we just can't know anything about his process from that failure. Certainly we can't conclude that it was "accurate", since it was not. If we had a good sample of elections where he used the same process and built up a good record then sure.

culi• 2 months ago

Unfortunately most of the most important visualizations are broken in the archived version. Including the gun deaths visualization and I think the P-hacking interactive

https://web.archive.org/web/20230205124354/https://fivethirt...

It's kinda sad to know no one else will get to experience those interactive visualizations. Though its nice to see the approval comparison page still works

https://web.archive.org/web/20241031232233/https://projects....

nomilk2• 2 months ago

Curious why they're broken, as the wayback machine seems to be able to run javascript. Do the visualisations rely on a server (or some other assets not included in wayback machine's crawl)?

palewire4• 2 months ago

I fixed that:

https://fivethirtyeightindex.com/graphics/ https://fivethirtyeightindex.com/illustrations/

7777777phil• 2 months ago

Related: Disney erased FiveThirtyEight

https://news.ycombinator.com/item?id=48197703

alberto-m• 2 months ago

I am certainly missing a lot of nuance here, but it seems to me Nate Silver managed to have his cake and eat it too. He surely got good money for selling FiveThirtyEight, and now that the buyer has erased the product, Nate can get back a huge chunk of its readers since he offers very similar analyses on his personal site. Sure, natesilver.net has less brand recognition than fivethirtyeight.com, but it's still decently well-known and can only go up from here.

its-summertime• 2 months ago

I think the nuance is that it is notable historical articles about predictions and discussions of political elections, during a time when politics is quite at the fore-front of many people's minds

xattt2• 2 months ago

It’s possible NS may have signed a contract saying that he cannot engage in elections prediction for X YZ months to same extent that he did with FiveThirtyEight.

derektank• 2 months ago

He didn’t. He retained the IP of his election forecast models and he publishes the results from them on his newsletter, The Silver Bulletin

JumpCrisscross• 2 months ago

Nate can get back a huge chunk of its readers

The downside is this furthers the divide between folks who pay for subscriptions and masses who get shoveled ad-powered slop.

jnovek• 2 months ago

He put up a substantial number of articles for free during the 2024 election, it felt less paywalled than much of the MSM these days.

shagie• 2 months ago

https://electoral-vote.com is still up, running, and showing data. It has always been free and ad free.

https://en.wikipedia.org/wiki/Electoral-vote.com

Electoral-vote.com is a website created by computer scientist Andrew S. Tanenbaum. The site's primary content was originally poll analysis to project election outcomes. Since the 2016 elections, the site also has featured daily commentary on political news stories.

iamnothere• 2 months ago

He may also finally shake off the comment trolls who piled onto him after 2016, seemingly blaming him for the election results (absurd but people are absurd).

After that election, a certain group would tirelessly work to discredit him any time his election predictions were not entirely one-sided.

beepbopboopp• 2 months ago

It was so funny, he mostly predicted the election correctly, but people confused probabilistic forecasting for sports lines.

gaws• 2 months ago

He surely got good money for selling FiveThirtyEight

Yes, he did: https://www.crainsnewyork.com/real-estate/residential/cny-na...

petros• 2 months ago

This is a great service for everyone who appreciates thoughtful analyses about politics. Losing FiveThirtyEight was a big loss, but this archive helps. Bravo!

aeternum• 2 months ago

Did FiveThirtyEight really get that much right other than the 2016 election?

I remember thinking they were the best data journalists out there, and they had some nice visualizations but did their other predictions actually hold up?

akio• 2 months ago

Yes, they forecast thousands of U.S. elections and thousands of sports games, and their forecasts had excellent calibration—e.g., events they said would happen 30% of the time actually happened about 30% of the time.

https://archive.is/sId82

https://github.com/fivethirtyeight/checking-our-work-data

palewire4• 2 months ago

FWIW, the site now indexes 13,000 graphics and 3,000 illustrations, which I clawed out of the Wayback Machine's HTML and preserved separately in an archive.org collection.

https://fivethirtyeightindex.com/graphics/ https://fivethirtyeightindex.com/illustrations/

The archive.org backend is at https://archive.org/details/fivethirtyeight-collection and will almost certainly outlive my site.

FiveThirtyEight articles on the Internet Archive

Related Stories