Interpreting Noise

http://robjhyndman.com/hyndsight/interpreting-noise/

by siddg • 12 years ago

47 32 12 years ago

mbillie1• 12 years ago

This is particularly evident during presidential election season, when the mainstream political news will do things like show a real-time opinion poll accompanying speeches or debates. The article hits the nail on the head at the end: this phenomenon isn't going anywhere unless people start getting a whole lot less gullible.

mtimjones• 12 years ago

Does anyone remember the unemployment rate dropping just before the 2012 Presidential election (and then rising again immediately afterwards)? These numbers can't be trusted, and are nothing more than a tool for party control of government.

batoure4• 12 years ago

that is simply untrue. If you spent any time trying to understand the data and the way it is produced you would understand why your comment is so outlandish. The BLS that produces these figures is basically a cult of numbers. they spend a huge amount of time correcting and reworking out of fear of this type of contamination and rigorously work to keep those numbers accurate.

sd8f9iu• 12 years ago

Which statistic are you interpreting this from? The Bureau of Labor Statistics' publications? In glancing at their unemployment rate estimates, it looks like the overall trend throughout 2012 and continuing into 2014 is negative, with some noise in between. Though the methods used to calculate unemployment statistics are frequently discussed (such as their inclusion or exclusion of workers who give up searching for a job), it seems like you are interpreting the noise for some mischievous activity (ironically, in a way similar to what this post discusses), for which you provide no evidence.

mikeyouse• 12 years ago

Seriously?

http://research.stlouisfed.org/fred2/graph/?g=xNq

001sky• 12 years ago

http://www.bls.gov/opub/btn/volume-2/revisions-to-jobs-numbe...

Table 1 shows the revisions.

October spiked month over month only to be reversed. And the 1st relase numbers show a "uptrend" for aug, sep, oct while the final numbers show a distinct "downtrend" for same.

crossreference: p17, http://www.gallup.com/strategicconsulting/163007/state-ameri...

batoure4• 12 years ago

I think that there are two things at work here.

1. Who is the designed consumer of this data.

In the US the CEW (census of employment and wage) is a data set that is primarily aimed at economists or modelers. The startling thing is how these data-sets became mainstream. I work in economics and for years no one was particularly conscious of when these figures are released(non quantitative people i mean) but now I am often surprised how many regular people know that the first Friday of the month is unemployment data day. The danger here is that modelers planning to work with the data are well aware of the signal to noise ratio and take it into account with everything they do. Unemployment data in the US at a granular level is also heavily redacted for a variety of reasons which makes this problem even worse.

2. The second issue that I would have loved to have seen this article elaborate on is what are our core metrics as a society. This is something that we still haven't nailed yet.

I have friends who work at the government institutions the produce this data and they can tell you that most of this work is still done with a very mid 20th century mindset.

On the other hand I know people who work with large business data-sets such as supply chain data from Amazon Walmart and others. This data is far more robust and could be used to give a much truer picture of how the economy is doing but it is currently locked up under the veil of corporate secrecy. if gov't statistical agencies could figure out how to work with this data we could do significantly better.

monochr• 12 years ago

"What is going on here is that the commentators are assuming we live in a noise-free world. They imagine that everything is explicable, you just have to find the explanation."

They know full well that there isn't anything behind it. The problem isn't the news organization, it's the media consumers. People want stories to understand the world in easy bite sized pieces. Unfortunately what affects us doesn't come with a simple story line, it's just one damned thing after another. But no one would sell any news with "Stuff happens, we have no idea why. More stuff to happen later."

batoure4• 12 years ago

The problem isn't the news organization, it's the media consumers.

I've always been somewhat disheartened by comments like this. People will blame consumers for how media is but then say this about technology:

"If I had asked people what they wanted, they would have said faster horses."

At some point someone sold the idea of sound bite consumable news to customers and convinced them that it was good. the way that we know that is because news had never looked that way until cnn and networks like it started.

consumers either know what they want or they don't know what they want... but I hardly think that issue is cut and dry

ZoF2• 12 years ago

At some point someone sold the idea of sound bite consumable news to customers and convinced them that it was good.

What makes you say this? This seems pretty false in my opinion.

It seems much more likely that sound bite consumables are the outcome of testing/feedback and iteration. It seems entirely possible to me(if not probable) that consumers(for the most part) don't know what they truly 'want' in terms of news.

We live in a world of(largely) instant gratification; the fact that 'news' would have to adapt to fit that lifestyle makes perfect sense to me.

People will blame consumers for how media is but then say this about technology: "If I had asked people what they wanted, they would have said faster horses."

What point are you trying to make here? I'm having difficulty parsing your meaning. Humans, when they aren't versed in the underlying operations of a system, WILL make broad assumptions(about said operations) and WILL have irrational expectations and 'wants' based on those assumptions. What would my grandmother(or even mother for that matter) ask me to improve about her spyware-ridden current-gen laptop? "Make it faster." If I asked her how to improve the news on the other hand I guarantee "make it faster" would not enter the conversation.

Soooo, what's your point here? That consumers do want 'faster' news? And why would a consumer's desire for more succinct news imply that they are okay with less accurate news...?

Obviously the issue isn't cut and dry, everyone is a consumer of the news and we all have our own opinions. That said, I have a hard time disagreeing with the parent comment. Have you watched the news recently? 'News' stories are NOT given time based on their importance to the world as a whole; and for good reason. When a media company(whose purpose is to provide fiscal return) is met with positive feedback for stories pertaining to popular culture, or pandering to popular opinion, they will invariably continue to publish in that vertical. The result is a 'news' which consists mainly of stories which are either racially/politically charged or major events(read: actual news) which are strung along for weeks(natural disasters, terrorists actions, etc...)

hawkharris• 12 years ago

On a related note, we could take another simple step toward improving the way people communicate about statistics: ban point estimates in news reports. Using a point estimate, I can accurately claim that there's a 50% chance heaven and hell exist. This is technically true because my confidence ranges from 0% to 100%.

I get that people often need so-called "killer facts" — quick statistics that make an impression in press releases — but confidence intervals are more honest and they don't take much longer to communicate. Instead of allowing people to say that "x% of users prefer...", let's say that "x-y% of users prefer."

yummyfajitas• 12 years ago

I get the point you are making, but confidence intervals are completely the wrong solution. They do NOT mean what you probably think they mean (where "you" is a generalized "you the reader").

http://www.ejwagenmakers.com/inpress/HoekstraEtAlPBR.pdf

Credible intervals are probably what you want. A lot of the time people misunderstand confidence intervals and treat them like credible intervals.

http://en.wikipedia.org/wiki/Credible_interval

highfreq• 12 years ago

While I think the articles most point is reasonable. The choice of stocks as an example is a poorly one. Because prices are reacting in large part to news. To the degree that the news presents something material and unsuspected it influenced peoples trading. Of course good news doesn't always lead a stock to increase, because sometimes most investors are expecting better news. Certainly there is a lot of randomness, but for heavily traded stocks the impact of news is real and substantial. Unexpected good news most often leads to an increase in stock price, and unexpected bad new most often leads to a decrease.

SixSigma• 12 years ago

Here's an example: http://m.bbc.co.uk/news/business-27058146

Google's share price changes the same day as an announcement. The BBC attributes the change to the news as though they are monotonic. But who is to say that the news didn't stop the share price falling twice was far?

Also, every trade requires two parties, so one side thought they were getting a good deal.

I'm also amused that the share traders are called "investors" rather than "gamblers".

MaggieL• 12 years ago

Warmists take note.

ISL2• 12 years ago

Errorbars are largely absent from public dialog. Can you imagine a world in which the weather report said:

"Tomorrow's high will be 30±3 degrees, and the low will be 25±2 degrees. We expect light precipitation, but our confidence is low."

That world is coming, I hope.

shanemhansen• 12 years ago

30±3 degrees... With what confidence?

dredmorbius4• 12 years ago

Where error bounds are stated, they are generally given for a 95% confidence interval, by convention.

thaumasiotes4• 12 years ago

That world seems pretty pointless to me. What should the public do when tomorrow's high is forecast at 30±3 degrees but it's actually 35 degrees? Why was the hedge an improvement?

dredmorbius4• 12 years ago

30±3 is different than, say, 30±6 or 30±1.

If the temperature range is, say, close to freezing, or some maximum safe temperature for an activity or crop, then the error bound tells you how certain the forecast is, and whether or not an adverse condition might be experienced, for which you might want to take precautions (cover plants to protect them from freezing, ensure that they're watered for heat, say).

TheLoneWolfling2• 12 years ago

If your crops will be killed by frost, 3+-1 is rather different than 3+-6.

TheLoneWolfling2• 12 years ago

Or:

"Tomorrow's high will be 27-33 degrees, and the low will be 23-27 degrees"

vijucat• 12 years ago

I have a principle to explain this, and much more : The universe sides with stupid.

There, there, take a look at Wild Poppies, near Argenteuil : http://i.imgur.com/1bSp0TJ.jpg

plg• 12 years ago

An example: a phenomenon that I think has come into play gradually over the past 3-5 years: national news broadcasts lead stories are based on the weather.

"3rd highest temperature seen in 5 yrs"

"4th coldest winter since 2001"

"2nd highest rainfall in March in the past 12 years"

etc etc

PS on a secondary note: shame on the news services for leading with weather so often. I guess it's laziness. No need to spend time, resources, expertise on investigating actual news.

Interpreting Noise

Related Stories