Math That Matters: The Case for Probability over Polynomials

https://anandsanwal.me/math-eduction-more-probability-statistics-less-calculus/

by herbertl • 1 year ago

24 33 1 year ago

For a moment there I was excited because I though the article would be about an extension of probability theory to use a polynomial basis instead of real numbers, with an analogy to how Quantum Mechanics is essentially just probability with complex numbers.

Hmm... this gives me an idea.

magicalhippo• 1 year ago

Quantum Mechanics is essentially just probability with complex numbers

For some more on that topic:

https://arxiv.org/abs/quant-ph/0101012

https://www.scottaaronson.com/democritus/lec9.html

tgv6• 1 year ago

One who grasps statistical significance can better interpret health studies ...

Ok, that's where I draw the line. Statistical significance is bullshit. Learning about it is as useful as learning about phlogiston. Its biggest failure is that it gives a false sense of security. It's what lead us to the reproducibility crisis. This should not have a place in an article that purports to emphasize the importance of statistics in the school curriculum.

taejavu• 1 year ago

Any chance you could expand on this, or point me to resources that do? It’s a very interesting take and not one I’m familiar with.

tgv6• 1 year ago

There's a lot of technical stuff, but you could look into Ioanides articles [1], and the book Bernoulli's Falacy by Clayton [2] is a good introduction, also for people less involved in statistics.

[1] https://medium.com/@d.incecushman/the-p-value-problem-how-dr... (this is just an example; I haven't read it)

[2] https://cup.columbia.edu/book/bernoullis-fallacy/97802311999...

TrackerFF• 1 year ago

Search for "p-hacking", that will give you a nice starter on the topic.

blackbear_• 1 year ago

Unfortunately we will not get rid of statistical significance anytime soon. Teaching it and explaining its shortcomings is much better than sweeping it under the rug, and perhaps it will help the next generations of scientists not falling into the same trap.

reedf14• 1 year ago

No no no. You couldn't have gotten this more wrong. The replication crisis comes from soft sciences under-powered statistical training.

jfengel4• 1 year ago

It comes from the fact that soft sciences deal with vast numbers of variables, far too many to control. They actually get a ton of statistics, trying to find a signal in so much noise.

It would be great if human beings were more amenable to rigorous experiment. Failing that, we at least need to understand what these things do and don't actually mean. It's either that or give up trying to study people entirely.

llamaimperative• 1 year ago

No it doesn’t. Common meme around here but the hard sciences are also rife with unreproducible studies. It’s an incentives problem.

tgv6• 1 year ago

I don't even know what you mean by "under-powered statistical training."

But about the harder sciences: when this ball got rolling, I attended a lecture of a statistician, who explained that basically all genetic results preceding were likely wrong, unless they showed something like 6 sigma significance. That's because H0 is so easy to reject when you base H0 and H1 on different measurements. The result is that every theory is true.

jasode• 1 year ago

Statistical significance is bullshit. Learning about it is as useful as learning about phlogiston.

The term-of-art "statistical significance" isn't bs because we ultimately have to choose what to pay attention to. Removing "statistical significance" from the vocabulary doesn't change the underlying reality about people deciding what to do based on a threshold of a number.

The observed effect of <something> is either:

(1) appearance of cause & effect is actually not there and just random chance or coincidence

(2) a real cause & effect and not random chance

Yes.. but "statistical significance" leads to "p-hacking" and other academic fraud! (https://en.wikipedia.org/wiki/Data_dredging)

That's true but the abuses still doesn't eliminate the need to name the concept of real cause & effect vs random chance. Whatever alternative mental framework one uses to decide which of those scenarios is happening, you will arrive at something that looks like "statistical significance" even if you don't call it that.

As an analogy, even though "averages" in math is misused, it doesn't mean "averages are bs".

tgv6• 1 year ago

Sorry, but statistical significance has a pretty narrow meaning, and it's wrong.

Removing "statistical significance" from the vocabulary doesn't change the underlying reality about people deciding what to do based on a threshold of a number.

I'm not arguing to remove it from the vocabulary. I'm arguing that that bloomin' threshold is the source of many problems.

The article also isn't about methodology, it's about the secondary school curriculum. To provide pupils with a false understanding of statistics is just bad. And to teach them why it's bad, is hard.

sweezyjeezy• 1 year ago

Statistical significance is bullshit. Learning about it is as useful as learning about phlogiston.

Ok, that's where I draw the line - statistical significance is not "bullshit" - however as you say, leaning on it too hard can cause things to break quite badly. Scientists misusing it do not negate all the medical advances we have made from moving to a significance-based system. It is an absolutely essential tool for people using statistics to understand, but its limitations must be emphasised when taught, and it must be understood that it is a tool, not a conclusion. Also other alternatives should be taught more widely (e.g. Bayesian inference).

tgv6• 1 year ago

We would have made better progress if we had skipped NHST. Mind you, nor Fisher, nor Pearson were in favor of it. It is a tool that should be taught after better ways, and then only to understand the past. Like phlogiston.

roenxi2• 1 year ago

You seem to be arguing that a grasp of statistical significance helps you interpret health studies.

eesmith• 1 year ago

Some background might be useful.

In the 1800s, geometry was seen as the best math to train one's mind. Abraham Lincoln, for example, "kept a copy of Euclid in his saddlebag, and studied it late at night by lamplight; he related that he said to himself, "You never can make a lawyer if you do not understand what demonstrate means; and I left my situation in Springfield, went home to my father's house, and stayed there till I could give any proposition in the six books of Euclid at sight"." (quoting https://en.wikipedia.org/wiki/Euclid%27s_Elements ).

This held until around WWII. One way to see how curriculum has changed over the last 75 years is in Sheldon Glashow's autobiography. He graduated from the Bronx High School of Science in 1950. Quoting https://www.nobelprize.org/prizes/physics/1979/glashow/biogr... , "High-school mathematics then terminated with solid geometry."

In the post-war era, the government needed a lot of people trained in the physical sciences and engineering fields where applied skills in calculus was more important than abstract ideas of how to think.

In the last 25 or so years there's been an effort to de-emphasize calculus as the end-goal progression for high school math, and to teach more statistics. AP Statistics wasn't offered until 1997, for example: https://en.wikipedia.org/wiki/AP_Statistics

This essay is part of that modern movement.

That said, I think the author has almost dismissed the classic idea of a mathematical education - to learn logical thinking. The only mention of "proof" is in reference to Gelman's "restructuring mathematical education around understanding uncertainty rather than abstract proofs."

I know nothing of Gelman's writing, but as the Lincoln example shows, abstract proofs are useful in constructing and analyzing legal arguments (and programming, IMO), in way I think should not be so readily set aside.

jfengel4• 1 year ago

Law seems an odd application of proofs. I've never seen anything in law to which mathematical rigor could apply.

Learning to reason is always good as a brain exercise. But legal "proof" has always struck me as laughably vague.

milesrout3• 1 year ago

As someone with degrees in both law and mathematics I was struck by the similarities in the subjects when I learn law. Everyone was surprised when I told them that... except for my cousin, a mathematician. It didn't surprise him at all.

Law (analytically - practically it is about problem solving, strategy, tactics, understanding your client's problems, clear presentation of ideas, etc. like any other business) is all about logic, reasoning, thinking clearly, separating precedent and antecedent, the logical flow of ideas, etc.

In practice, legal reasoning is more inductive than deductive (not entirely), which means if has a different flavour in practice. It is still just as much about logic.

But don't get distracted by things like the standard of proof etc. That has much more to do with the fact that you are making decisions based on unreliable and imperfect information, as opposed to reasoning about entirely abstract ideas like a mathematician.

jfengel4• 1 year ago

Polynomials had always struck me as begging for the question "when are we ever going to use this"? It eventually makes sense when you get to linear algebra and calculus, but for students not on that track, it seems designed to make them hate math.

roenxi2• 1 year ago

The path is polynomials -> matricies -> multivariate statistics. And multivariate statistics are actually the natural home of probability, although it requires too much setup for most people to learn anything about it.

milesrout3• 1 year ago

You use them at the same time you learn them in physics, no? Also, you sometimes need to learn things on a bit of faith it will eventually be useful. This stuff is so fundamental that it is difficult to motivate it except to say: you will use it so pervasively if you do any technical, scientific, mathematical or computational subject that it will seem like asking "when are we ever going to use this 'algebra' thing?"

I never had much motivation for much of maths presented to me and I never hated it. In fact, I got bored when teachers tried to justify what they were teaching with boring physics examples etc.

I think the sentiment you are talking about has a risk of going too far and making teachers afraid that if they teach something without justifying it to their class that they will lose their attention immediately and justifiably slow. It risks normalising the idea that teachers aren't entitled to the attention and hard work of their classes unless they give the kids a good reason to pay attention: you have to make it relatable, relevant to them, or it is your fault when they don't pay attention.

What ever happened to: you are a child, you are at school to learn, you learn what you are taught and that is that? Obviously sometimes it is good to motivate abstract concepts with concrete examples but if you take it too far you'll end up teaching calculus before you teach algebra because you are trying to justify the algebra! And you'll need to teach differential equations before you teach calculus because you need to motivate calculus. And what is the point of differential equations anyway? Before long you're teaching undergraduate physics to 11 year olds.

jfengel4• 1 year ago

Not everybody is going to take physics, either. It's necessary for students on a science track, but only a fraction of students are going to take science.

Even the polynomials we do use are very limited, until much later. We don't factor polynomials; we don't compute residues. They're just formulas with numbers in and numbers out. We may solve for various parts, but nearly always just for a single variable of power 1.

nh23423fefe• 1 year ago

polynomials are generalizations of number. you use them to solve equations just like you use all numbers. polynomials add the notion of an "indeterminate"

in the same way that you use integers to solve equations like 3 + x = 2 (x = -1), or rationals to solve 3x = 2 (x = 2/3). you use polynomials to model equations like x^2 = x + 1 (x = phi). or to perform polynomial division. or to do algebraic geometry (x^2 + y^2 - 1 = 0 => unit circle)

bsenftner• 1 year ago

Real world education story: I struggled with Calculus, having to take it 3 times before passing. I went to my professor and explained to him that I was simply unable to conceptualize what we did to solve calculus problems as not being arbitrary steps. I showed him a software program I wrote that translated my Latin homework into English, and remarked how the rules we use to solve calc problems feel like just another Latin translator. Then I showed him that I was finally getting the calc problems right by doing them with statistics first, and then knowing the statistical answer I'd play with the calc rules until I got the same answer as I got from statistics.

He was quite surprised, and we discussed what I was doing at length. He asked me what I wanted as a career, and I told him that I already had a video game studio, products in the market, and simply wanted to get better, maybe pursue that new field of 3D computer graphics. (This was '83). My professor completely changed his course and nature of his homework, it was still Calculus 1 and 2, but he was using video game situations and simulations and illustrating the duel nature of solving with statistics or with calculus and when one is the choice over the other. That class, that single professor's insight to modify his course set me on a track that led to a career many dream about, some disbelieve when I relate it, my career has been stellar.

vlasky• 1 year ago

Base rate fallacy is a key part of many erroneous human decisions or judgements. This could easily be taught in high schools:

https://en.wikipedia.org/wiki/Base_rate_fallacy

wakawaka28• 1 year ago

Interesting idea but math educators put a LOT of thought into what should be taught and how to help students understand. If you don't understand polynomials then you're going to be extremely limited in terms of what kinds of probability problems you can solve. Calculus is also used in statistics very heavily. "I will NEVER need to do that!" you say? Well, how sure can you be, and how many people say that and change their minds? Teachers can't just take everything at face value. They try to educate you up to a level that won't leave you in a bad spot if you want to continue some day. Learning about polynomial factoring never hurt anyone. There are lots of basic problems you can't solve without knowing how to do that stuff. You can self-study less difficult things on your own, as well.

Chinjut• 1 year ago

Are polynomials not used in probability?

Math That Matters: The Case for Probability over Polynomials

Related Stories