Lies. Damned Lies. P-value thresholds

https://www.newyorker.com/magazine/2019/09/09/what-statistics-can-and-cant-tell-us-about-ourselves

by fghorow • 2 years ago

60 24 2 years ago

newyorker.com

brudgers• 2 years ago

Previous discussion under original title, https://news.ycombinator.com/item?id=20978134

chkgk• 2 years ago

http://archive.today/yju8K

sdwr2• 2 years ago

This piece is basically set up to fail. It's historical, based on math, and doesn't venture towards any drama or stakes. I can easily imagine it being a farticle on a third-rate SEO webzine.

So why does it work okay as a New Yorker piece? How is their writing consistently good?

I think their secret sauce is the (implied) perspective. The impression that the author has a unique, complete, accurate take on the subject, and is letting you in on it piece by piece, in a meandering way.

tucnak• 2 years ago

You're exactly right: what you refer to as "perspective" is easily faked.

martindale• 2 years ago

Astroturf on HN. Never thought I'd see the day.

sdwr2• 2 years ago

Yes I'm a shill for big "New Yorker magazine". You are so smart how did you figure it out.

moonlion_eth• 2 years ago

P values are why llms hallucinate

ellisv• 2 years ago

Would you elaborate?

svnt• 2 years ago

If I take it at face value they seem to be blaming LLM hallucinations on questionable training data from published science done badly while focused on p-values.

klysm2• 2 years ago

I have no idea how this could possibly be true

zrm2• 2 years ago

LLMs are essentially predicting what token could plausibly come next. They sort the possible next tokens by probability, often throw out the the ones with very low probabilities (p-value too low) and then use a weighted random number generator to choose one from the rest.

Sometimes that means you exclude a good next token, or include a bad one, so a bad one gets chosen. And then once it has, the thing is going to pick whatever is most likely to come after that, which will be some malarkey because it has already emitted a nonsense token and is now using that as context. But whatever it is, it will still sound plausible because it's still choosing from the most likely things to follow what has already been emitted.

seanhunter5• 2 years ago

I can tell by your response that you are burdened by understanding at least one of a)what a p-value is b)What an “LLM hallucination” means or c)what actually causes llms to hallucinate.

If you set yourself free from the meaning of all the nouns in the sentence then you can get there.

flobosg• 2 years ago

(2019)

Hilift• 2 years ago

Harold Shipman with the NHS killed about one person per month for 30 years, and the response was to ask if doctors could be monitored to discover this earlier. However, the systems they conceived "eventually cast suspicion on the innocent". 25 years later the NHS is still struggling to answer these questions.

dccsillag• 2 years ago

If thresholding of P-values is the issue, E-values -- a recent, much more elegant, easier to work with, and more robust alternative to P-values -- solve this.

https://arxiv.org/abs/2312.08040 https://arxiv.org/abs/2205.00901 https://arxiv.org/abs/2210.01948 https://arxiv.org/abs/2410.23614

alphan0n• 2 years ago

Thank you, excellent reading.

Lies. Damned Lies. P-value thresholds

Related Stories