Nate Silver has made a career out of predicting things

http://www.slate.com/articles/business/books/2012/10/nate_silver_s_book_the_signal_and_the_noise_reviewed_.html#

by barredo • 14 years ago

64 89 14 years ago

SkyMarshal• 14 years ago

<3 Nate's blog. Only two election sites I bother with nowadays are Nate's [1] and Andrew Tannenbaum's [2], both data-oriented.

[1] http://fivethirtyeight.com

[2] http://electoral-vote.com

sendos3• 14 years ago

FYI, I use the data from electoral-vote.com to turn it into a probability of winning the election at http://prespredict.com

Thrymr• 14 years ago

I like that your site shows the 2008 history right below the 2012 (although it is distracting that the scale in time is different between them). I couldn't even find that comparison when I looked for it on Nate Silver's site.

I'm curious about the methodology as well. Why didn't you choose more standard gaussian statistics for the differences, instead of the combination linear/step model? How much difference would it make?

tricolon• 14 years ago

Nice site! On the front page, when I hover of the 2008 graph, I see stats for Obama/Romney rather than Obama/McCain.

andrewcooke• 14 years ago

he's good because he's careful, he does the maths right, and he's doing meta-analysis, so he has more data than the people he's being compared to.

but he's popular mainly because he can communicate so well. doing the maths right is not that hard.

andrewljohnson5• 14 years ago

Doing the math as "right" as he does it is incredibly hard. You can tell because academics do it totally wrong on a regular basis, which leads to many garbage papers, which thankfully sometimes get caught in peer review.

It's true he's a good writer, but the proof is in the pudding. He could be foaming at the mouth, but if he's predicting the outcomes, people will tune in.

tptacek12• 14 years ago

The math he's using is not hard, but the data sets he's working with are annoying.

ithcy22• 14 years ago

"Predictions are hard—especially about the future."

And here I was under the impression that ALL predictions are about the future.

textminer2• 14 years ago

Today's the day you learn about Yogiisms.

tptacek12• 14 years ago

Sure it's Yogi? It's also attributed to Samuel Goldwyn and Mark Twain. (I doubt the Twain association).

ithcy22• 14 years ago

Well, I suck.

andrewljohnson5• 14 years ago

Republicans say Nate Silver is a biased liberal, and reading his blog this election cycle makes it seem that way to me even.

But I would like to know - is there a Republican analog to Nate Silver? Someone who predicts 90% of the races at the end, does the meta-math, and has a computer model?

If there is no Republican analog, can you possibly say Nate Silver is liberal or biased? Does using a computer model and math, and blogging about your results make you a Democrat?

derleth4• 14 years ago

Does following the scientific consensus on issues like anthropogenic global warming, evolution, and sex education make you a Democrat?

andrewljohnson5• 14 years ago

It correlates!

Global warming, 2/3 Republican deny it, more than 3/4 Democrats believe it: http://www.businessweek.com/news/2012-09-30/global-warming-l...

Creationism is 38% for Democrats, 60% for Republicans: http://www.cultureofscience.com/2011/08/19/republicans-and-d...

I think Republicans are more likely not to want to mention or distribute condoms to teens in sex education, but I don't regard this as quite the same general disregard for science and math.

anamax7• 14 years ago

> Does following the scientific consensus on issues like anthropogenic global warming, evolution, and sex education make you a Democrat?

The correlation is with misunderstanding all three and coming to bogus conclusions.

Let's take evolution. There are lots of folks, typically Democrats, who insist that it is fundamental to all of science. (Hint - that's clearly false. Disagree? How does evolution matter outside biology? Too hard? Name three biology subfields that don't work without evolution? Too hard? Name three things in biology that were predicted using evolution?)

Democrats like to think that they're scientific, but ....

Let's take the suggestion that state of mind affects conception in humans. Clearly false when Akin said it, but what does everyone say to a couple trying to get pregnant? Yup - "you've got to maintain a positive mental attitude." (I don't know whether mental attitude matters - I'm just pointing out the double-standard.)

joshuahedlund• 14 years ago

Yes, but only if you also ignore the scientific consensus on genetically modified food, vaccines, nuclear energy, and hydraulic fracking :)

FreeKill• 14 years ago

It's just a time tested tactic of discrediting a result you dislike in order to prevent your constituents from becoming apathetic and potentially not voting.

If Republicans actually concede that Nate Silver's results are legitimate and unbiased (if they are), than the party would have a much harder time raising money and getting out the vote.

It's better to frame it as a conspiracy so less people take it seriously.

sharkweek• 14 years ago

If you follow much of his writing/tweeting, he doesn't disguise much opinion, which has a liberal slant - but the pure math of his project is pretty hard to argue against.

sxcurry• 14 years ago

I don't think he has a leaning one way or the other - he tries to be unbiased, and arrive at data-driven conclusions. What have you seen from Nate that makes him seem a "biased liberal" to you? I'd be very interested to know. Also, we'll see in about a month how good his meta-analysis is.

andrewljohnson5• 14 years ago

I can't put a finger on it, but you know which way he hopes things go, and I think that shows through, despite his attempts to stay unbiased, and despite the NYT style guide. I think a TV news anchor probably gives away his bias with body language too.

I like to tell myself that too, so I regard his work with at least a little skepticism, even though I gobble it up and bet on it.

tptacek12• 14 years ago

He leans Democratic and has said so on more than one occasion. He supported Obama in '08.

One gets the impression that his professional reputation is more important than his ideological affiliation, and so I find him pretty credible.

kmfrk• 14 years ago

I don't really see why it's wrong that Nate has political opinions. The alternative is Jim Lehrer who values his "objectivity" so much that he doesn't even vote at elections.

Sometimes, facts favour liberals, sometimes they don't.

mtgx• 14 years ago

Must be because "reality has a liberal bias".

smacktoward• 14 years ago

No, but the GOP side does have a Bizarro version of him: http://www.businessinsider.com/unskewed-polling-dean-chamber...

nnethercote• 14 years ago

His methodology seems to be "ignore any poll that is bad for Republicans".

wilfra2• 14 years ago

He is a biased liberal and reading any of his commentary should be viewed through that lens. He doesn't even make much of an effort to hide it, in 2008 on 538 he was basically rooting for Obama.

However that doesn't mean his models are wrong. When they show that Republicans are going to win (or have improved), he states that just as clearly as when the same is true for Democrats.

Alex39173• 14 years ago

"He is a biased liberal and reading any of his commentary should be viewed through that lens."

Are we reading the same blog? I've never seen him take sides on any issue.

arrrg• 14 years ago

Humans who are not biased probably do not exist. Or at least do not care or know about politics in the slightest. I think there is nothing wrong with that. That doesn't invalidate his work.

I know too much about human biases to even try to claim that his do not influence his interpretations and work at all – I just don't think that's possible – but that is general and in all likelihood inevitable problem. I think it's pretty clear, however, that it's in his best interest to make accurate predictions, not partisan predictions.

I do not think it's really helpful to start out with that question. It puts too much focus on what's actually important. Yeah, bias can be hard to detect, but a little bias does not make models useless and when there is a lot of bias you should actually be able to point at what's wrong with the model instead of having to resort to some diffuse claims of bias.

Writing this made me think of Elisabeth Noelle-Neumann (http://en.wikipedia.org/wiki/Elisabeth_Noelle-Neumann). I disagree with many of her views and I think it would be fair to call her biased after reading her most famous work (The Spiral of Silence), but I do respect her work. There is a difference.

nhebb• 14 years ago

> is there a Republican analog to Nate Silver?

No, not directly. RealClearPoltics (http://www.realclearpolitics.com) aggregates polling data, but it doesn't analyze it. Sean Trende does some analysis of the polls, but he is a lawyer and political scientist, not a mathematician.

Other than that, pollster Scott Rasmussen (http://www.rasmussenreports.com/) has a solid track record forecasting outcomes and is the closest thing to a Nate Silver analog.

Edit: > does the meta-math

BTW, I think one of the most interesting things that will come out of this election is the impact of cell phones on polling data. The meta-math is based on the sampling models for each pollster, and the models have to take land line / cell phone demographics into consideration. Many pollsters only call land lines and try to accommodate for the demographics in their polling, but it's not certain how accurate their models are.

There has been a shift in land line demographics in the last four years. Lower income people used to have land lines only and mid to upper income people had land lines plus cell. land line usage has dropped a lot. In recent years, lower income people tend to have cell phones only. It will be interesting to see how the the election results compare to the polling data.

mkramlich2• 14 years ago

Reminds me of the meme "reality has a well-known liberal bias". I don't know or suspect whether that's true or not. But I have noticed personally in life that whenever someone is both intelligent, educated, is compassionate/empathic and does long-term thinking, that person also seems to be "liberal", progressive or left-leaning, politics wise. Perhaps not always. But most of the time, based on my observations. I think if you have those traits but take away compassion/empathy, and focus more on short-term/short-range thinking, you're more likely to be "conservative", or right-leaning.

waterlesscloud6• 14 years ago

What would you say about the tens of millions of Americans who are educated, religious, and give hundreds of millions of dollars to build schools, clinics, and missions around the world?

Are they short-sighted or lacking compassion?

I think this kind of binary political thinking is the most insidious thought virus we face. Far more dangerous than whether or not someone believes in evolution, because it separates us and makes us believe that the "others" have traits that they really don't have. It's awful.

spamizbad• 14 years ago

Well the proof is in the pudding. His performance on the 2008 and 2010 election indicate to me he's not some partisan hack. He accurately predicted all of the EVs Obama obtained in 2008 with the exception of Indiana (which was quite close - only a few hundred votes in the margin IIRC). He did exceptionally well in 2010, coming in 3rd for accuracy behind Survey USA and PPP (A democratic shop, incidentally) in house race prediction.

Silver is politically liberal but he's a stats guy at heart.

The republican equivalent would be Unskewed Polls http://unskewedpolls.com/ which re-weights partisan affiliation of polls to match Rasmussen (The result: Romney is WAY ahead). It's author isn't a big stats guy and goes more off gut. This comparison isn't entirely fair though. Republican polsters and aggregators are probably private, working inside campaigns and not for a public news organization.

You can be politically liberal or conservative and not have it directly spill into your work.

adestefan• 14 years ago

Rasmussen is well known to have a decent conservative slant because of the way they select their population and perform their polling. I'll leave it as an exercise to the reader to decide if this is intentional or not.

seunosewa• 14 years ago

I'm lost. What are some of his latest predictions that came true?

paulgb• 14 years ago

From Wikipedia[1]:

> Silver's final 2008 presidential election forecast accurately predicted the winner of 49 of the 50 states as well as the District of Columbia (missing only the prediction for Indiana). As his model predicted, the races in Missouri and North Carolina were particularly close. He also correctly predicted the winners of every U.S. Senate race. The accuracy of his predictions won him further acclaim, including abroad,[49] and added to his reputation as a leading political prognosticator.[50]

[1] http://en.wikipedia.org/wiki/Nate_Silver#2008_U.S._elections

wtallis2• 14 years ago

Why do you only care about his recent predictions? What's unconvincing about the track record of the rest of his career?

spindritf• 14 years ago

If he has a good method, if he's on a right track, that method should generally be improving with more work and data so recent predictions should generally be better than distant ones.

Also, "You're only as good as your last envelope."

tatsuke952• 14 years ago

Nate Silver came from the fantastic world of the statistical analysis of baseball, specifically Baseball Prospectus. You can check out his body of work here (some articles require a subscription):

http://www.baseballprospectus.com/author/nate_silver/

His big contribution was a model used to predict player statistics. When he first started writing about politics, I assumed the name was a coincidence.

wilfra2• 14 years ago

Actually he came from the fantastic World of poker :)

He gravitated to Sabermetrics from there.

tatsuke952• 14 years ago

There you go. Dynamic guy. Almost like he enjoys numbers.

tptacek12• 14 years ago

The book is pretty good, sort of like a cross between Moneyball and Freakonomics. It's less political than you'd think it'd be, and as an attempt to preach the gospel of Bayesianism, should be congenial to most HN readers.

Nate Silver has made a career out of predicting things

Related Stories