What is a standard error? [pdf]

http://www.stat.columbia.edu/~gelman/research/published/standarderror.pdf

by luu • 3 years ago

45 13 3 years ago

stat.columbia.edu

To give a little preview of the content of the (very short) paper:

In statistics, the standard error has a clear technical definition: it is the estimated standard devi- ation of a parameter estimate. In practice, though, challenges arise when we go beyond the simple balls-in-urn model to consider generalizations beyond the population from which the data were sampled. This is important because generalization is nearly always the goal of quantitative studies. In this brief paper we consider three examples:
What is the standard error when the bias is unknown and changing (my bathroom scale)?
How do you interpret standard errors from a regression fit to the entire population (all 50 states)?
How should we account for nonsampling error when reporting uncertainties (election polls)?

happytoexplain2• 3 years ago

This and "standard deviation" are a couple of those terms for which I'd be interested in a definition that didn't assume quite so much domain knowledge (maybe one step above a totally naive layman's metaphor). I hear the terms all the time, but I only have intuitive statistics knowledge (averages, curves, controls, etc). I recall doing a brief Wikipedia dive once and coming up dissatisfied.

kkoncevicius• 3 years ago

Maybe this (and the answers therein) can help: https://stats.stackexchange.com/q/550619/18417

jldugger• 3 years ago

My statistics textbook at home covers this in pretty clear detail. I bought mine from abebooks for like 10 bucks, seems money well spent. Stats Wikipedia is written by and for stats PhDs. My recollection on standard deviation:

1. It's a descriptive statistic; we want to summarize how closely data clusters around its central ("mean") value.

2. We could just calculate the average distance, but we kind of want to penalize large outliers. We want the sequence [50,50,50,50,47,53] to have a higher deviation than [51,49,51,49,51,49], even though they both have a total distance from 50 of 6.

3. To achieve this, we use "root mean squared" -- square each sample's distance from average, then average those values, and finally take the square root of that average. In this manner, samples with large distances have a greater effect on the outcome.

anonymousiam• 3 years ago

"Standard error" could also be stderr (fd 2) in C.

mike_hock• 3 years ago

stderr is the obvious pun thread starter for this post and you've preempted it with this comment :(

o11c• 3 years ago

In C and similar languages, a standard error is more often known as a "defect report" (DR).

fsckboy• 3 years ago

stderr in C is actually stderr in unix, "file handle" 2 which is an integer 2, and is a "property" of a unix process.

C implementations on non-unix platforms do attempt to simulate the "meaning" of file handles, especially 0, 1 and 2, on non-unix systems

the term "file descriptor" tends to be used to describe FILE structs which are a software layer on top of file handles—which include buffering to improve efficiency in some circumstances and mask the difference between a character stream and a block device—and FILE descriptors are a property of userspace software

HPsquared• 3 years ago

Is it true that the standard deviation is the "expected" distance from the mean of a randomly selected value?

Like, if the mean is 1 and the SD is 1, and I was to take a sample and take the average of the absolute values of the samples, would that tend towards 1 as I take more samples?

Edit: I think I've just described "mean absolute error"... seems to be related but the "root mean square deviation" is the one more closely related to SD. I think... Not sure where the "n" comes in.

LogicalRisk• 3 years ago

"Where does n come in?"

I *think* this might help to answer your question for where n comes from. It helps me at least think about it.

The definition of the variance of the standard error V[\bar{x}] = V[X]/n. You can back this out from the definition of variance, the property that V[aX] = a^2V[X], and that variances are additive with independent draws. Take the square root of that and you have the standard error.

Why this "feels" right to me via an example.

Suppose we want to know the average height of the US population. Intuitively, we think that (assuming a representative sample) we'll do "better" in the sense of a tighter distribution around our best guess (mean of sample) of the population value if we sample 1000 people as opposed to sampling 10.

This is related to the distribution and would function as our "best guess" about the dispersion of a variable in the same units as the original. Both of them are sampling to try and guess the average. Since \bar{X} is itself a random variable, it has a distribution, and that distribution should probably include something about the sampling process we used to characterize it.

Mean absolute error would be E[|X-mu|] since the true mean of the distribution is a constant.

atkindel• 3 years ago

Your thinking here is pretty close to right, but it sounds like you may be a bit confused about the difference between "standard deviation" and "standard error". They're closely related, but it's very important to keep them distinct.

In your example, you have a random variable X that comes from some distribution parameterized by its mean m and its variance s^2. We can write this random variable as x ~ ?(m=1, s^2=1). Continuing your example, suppose we sample a sequence of n draws of this random variable X = [x1, x2, ... xn] and then take their sample mean f(X) = 1/n * sum(X). The value of the sample mean will vary depending on the sample we drew, so we want to know the properties of the sampling distribution of f(X), particularly as we increase n. The standard error is the standard deviation of the sampling distribution of f(X).

What is a standard error? [pdf]

Related Stories