What Are the Odds?

https://terrytao.wordpress.com/2022/10/03/what-are-the-odds/

by jjgreen • 4 years ago

145 88 4 years ago

terrytao.wordpress.com

unphased• 4 years ago

This article started off really clear and straightforward, then it dove into stuff that I clearly wasn't going to be able to follow, and upon skipping it I arrived at the explanation which was was plainly obvious from common sense.

rubatuga• 4 years ago

Yeah, that article really took a sudden nosedive

minihat• 4 years ago

such a number is often dutifully provided to such journalists, who in turn report it as some sort of quantitative demonstration of how remarkable the event was.

Apparently the conclusions are not common sense if you are a journalist.

andreareina• 4 years ago

Common sense can mislead, so it's instructive to make it rigorous to see whether it does indeed hold up. Not for everything or even most things, otherwise you'd be analysis paralysis sets in.

jiggywiggy• 4 years ago

In this case the author was a bit silly.

He also used common sense to determine he had no way of determining the probability of a rigged event. And therefore using Bayesian probability looked fancy but didn't help learn anything new. I remember reading first paragraph and thinking wow, bayesian has a way of helping determining the odds of that, but no.

Either he wanted to educate or is so involved with statistics that it became his default way of framing problems.

Rioghasarig• 4 years ago

Well that's what mathematicians like to do.

fny3• 4 years ago

Most useful takeaway from this article: If you choose to play the lotto, don't use cute numbers. Odds are high that other people will too, and you'll have to split the pot.

Part of the explanation surely lies in the unusually large number (433) of lottery winners
But on the previous draw of the same lottery ...
the unremarkable sequence of numbers {11, 26, 33, 45, 51, 55} were drawn ...
and no tickets ended up claiming the jackpot.

Semaphor6• 4 years ago

That is actually commonly recommended for exactly that reason. Often right after recommending to not play the lottery at all ;)

katrotz• 4 years ago

Remember reading in some book on probability and statistics that one will be better off betting insignificant amounts of money rather than not playing at all.

fny3• 4 years ago

It actually depends on the size of the pot. There have been times where there was positive expectancy.

[0] https://uncommondescent.com/intelligent-design/saturday-fun-...

checkyoursudo• 4 years ago

I never understood the admonishment not to play the lottery at all.

I waste lots of money doing things that are fun experiences, or buying unnecessary foods or drinks, or gadgets or toys, etc.

I can understand recommending not to get addicted to playing, or not to spend money that should be going elsewhere, right? But when that admonishment is not qualified, then it is a little silly. Might as well say, never buy extra dip for your french fries or something.

It's true that I haven't bought lottery tickets in years, but spending a few hundred cents on some longshot jackpot can be its own kind of fun.

yeellow2• 4 years ago

I read once about a famous mathematician who claimed that whenever he plays lottery he chooses the sequence 1,2,3,4,5,6 because nobody else would. Also it seems strange that so many people would choose a sequence that most people would call "not random" and suspicious (hence the article). Maybe those people follow the advice of this famous mathematician. Authorities should check the occurrence of this series in the previous games to see if that many people really like this sequence or if it was a one time event. Thinking about this, it would be interesting to see what numbers people choose, if there are some outliers preferred by many. I wonder if there are lotteries that publish such data?

jmharvey2• 4 years ago

I've bought the number 1,2,3,4,5,6 on numerous occasions, but my lottery play isn't really representative of the typical player.

A few years back, the Quebec Lottery published its most frequently played numbers. It seems the original press release has been lost to the sands of time, but IIRC 1,2,3,4,5,6; 1,7,13,19,25,31; and 4,8,15,16,23,42 were the most common sequences (corresponding to the top row of the play slip, the first column of the play slip, and the mystical numbers from the TV show Lost, respectively).

Oh, and if you're interested in digging more, in my experience, the Texas lottery is quite forthcoming with their open records; you could probably find out what's popular in 2022.

thegabriele• 4 years ago

From an algorithmic information theory, i wouldn't choose any easily reproducible sequence.

If you were presented with two array of integers:

- 1, 2, 3, 4, 5, 6

- 95, 10, 39, 45, 6, 47

...which one would you say it's random?

Still, from a frequentist point of view both are.

Where's my mistake?

geysersam• 4 years ago

I wonder if the lottery companies takes this into consideration when setting the price of their tickets.

If all your players pick the same numbers, you as the lottery seller is in a much better spot, and you could safely lower the prices or increase the pot to draw more people into the game.

However, if you do that and people start picking random sequences you risk losing a lot of money.

ethanwillis• 4 years ago

And yet what Mr. Tao misses is his own assumption that the drawing of each number is truly independent.

FabHK2• 4 years ago

They’re not independent. If the 9 is drawn, the next ball won’t be the 9.

rippercushions• 4 years ago

Mr Tao accounts for this, since

   (55 6) = (55/6)*(54/5)*(53/4)*(52/3)*(51/2)*(50/1)

Tainnor2• 4 years ago

Yes, surely fields medalist Prof. Terence Tao must have fundamentally misunderstood basic probability.

sohkamyung• 4 years ago

An alternative take via The Conversation [1]:

Based on anecdotal evidence from other lotteries, this number may not at all be unusual. We also need to consider the many thousands of similar lotteries drawn around the world each year, almost all of which receive no international press. While such outcomes are highly improbable for any given draw, the huge number of total lotteries means it’s actually quite likely at least one of them will produce a remarkable outcome by chance alone.

[1] https://theconversation.com/433-people-win-a-lottery-jackpot...

AdamH12113• 4 years ago

The multiple endpoints fallacy also comes into play here. Instead of multiples of 9, it could have been multiples of another number, or any group of sequential numbers, or sequential even numbers, or prime numbers, or...

The odds of a particular remarkable thing happening are very low, but the odds of some remarkable thing happening can be much higher.

tnuc• 4 years ago

Maybe they just got the numbers from a fortune cookie?

idiocrat• 4 years ago

That reminds me that I finally need to start participating in lotto, just to have a skin in the game.

Not participating - zero chance of winning.

Participating - 1 in 30 million chance of winning 4m USD.

tomcatfish• 4 years ago

0.000000000000001 chance - cost $0 0.000000033333333 chance - cost $4

Which one comes up on top, expected value wise?

Epa095• 4 years ago

Your evaluation of the price might not scale linearly with the money amount, making it a fault to focus to much on the expected value.

Example 1: you own the mafia 1 million dollar, pay tomorrow or die. You have absolutely no way to get the money before the deadline. You have some money though, and the winner of the lottery gets 1 million. Your expected value(in $) is negative, but the chance of surviving is no longer 0 if you play the lottery.

Example 2: you are a normal lower income person. You have no real chance of getting rich, or even comfortable. You can save the 4$, and maybe buy a slightly better TV every 3 years, but it will make no real difference in your life. Winning e.g 2 million on the other hand would lead to a qualitatively different (and better) life. Its not that 2 million is 2millon/4 times better than 4$, it is just incomparable. One leads to a completely different life.

andirk2• 4 years ago

I don't play and instead write down my selection of numbers. If I lose, I won the $3 I would have spent. If I win, well, that hasn't happened so we'll see.

FabHK2• 4 years ago

The beginning of the article is an excellent (though conventional) introduction to Bayes and Bayesian updating, but what I found original and have never seen expressed so clearly were the remarks about dishonest discussion tactics, especially the first one:

Formulate a vague hypothesis H1 so broad that it is not extremely unlikely (thus has non-negligible prior, P(H1) > 0), then in the updating step sneakily introduce a much more specific hypothesis H1’ that is far less likely a priori, but that yields a high probability for the evidence P(E|H1’).

If the audience doesn’t notice the bait and switch, it’ll come away with the impression that the evidence strongly supports H1/H1’ vis-à-vis H0, when it actually doesn’t.

Reminiscent of the Motte and Bailey fallacy.

https://en.m.wikipedia.org/wiki/Motte-and-bailey_fallacy

mistermann2• 4 years ago

It's even more complicated than that though: "dishonest discussion tactics" suggests that the person is doing it with fully conscious awareness and intent, but this is far from a safe assumption - and, it isn't a simple binary "is or is not", and it isn't only the usual suspects who commit this technique.

simonh2• 4 years ago

Maybe so, but ultimately it's up to us to take responsibility for the techniques we use in argumentation. I would agree that it's not generally a good idea to try to use an attack on the rhetorical techniques someone uses in order to win an argument. It's best to politely find a way to challenge their argument without attacking them personally for the way they put their case.

pc86• 4 years ago

Maybe it's just my liberal arts education but discussions like this and comments like "hypothesis H1’ ... that yields a high probability for the evidence P(E|H1’)" or "evidence strongly supports H1/H1’ vis-à-vis H0" mean absolutely nothing to me without a concrete, real-world example of what H1, H1', and H0 are.

time_to_smile• 4 years ago

It's not the presence of a liberal arts education, it's the absence of a mathematical one (and technically mathematics is one of the 'liberal arts'). I have a liberal arts background, but also a statistics background and reading this is no problem for me.

Reading and thinking mathematically is a literacy the same way reading sheet music is a type of literacy. Just like there are degrees of literacy for reading and writing same with these disciplines. Some people can't read sheet music at all. I can name the notes when reading sheet music, and know where they are on a piano, but, unlike many musician friends, I can't "hear the music" when I read.

Similarly it takes time and practice to "read mathematics". To someone with no practice the notation appears as scribbles. A more experienced mathematician will pause and work through what's happening. At a certain point, you start to experience the same "hearing the music" with mathematical writing.

defen• 4 years ago

Formulate a vague hypothesis H1 so broad that it is not extremely unlikely (thus has non-negligible prior, P(H1) > 0), then in the updating step sneakily introduce a much more specific hypothesis H1’ that is far less likely a priori, but that yields a high probability for the evidence P(E|H1’).

H0 is "the lottery is fair" (null hypothesis). H1 is "somebody rigged the lottery". It's not outrageously unlikely, it's happened many times before and it's something people can easily understand, and it's not specific about the details. So maybe someone would say "Without knowing more, I think there's a 1% chance that happened". H1’ is "The lottery is rigged by a cult that worships the multiples of {9}" - This is the sneaky step. If it weren't such an obviously outrageous example, you might not notice that this specific hypothesis is very unlikely. However, H1’ perfectly predicts the evidence that we did observe! So basically, if you plug in the numbers, without adjusting for the fact that H1’ does not have a 1% chance of being true, you turn "I think there's a 1% chance someone rigged the lottery" into "The lottery was almost certainly rigged - 99.9997% chance!"

williamcotton• 4 years ago

He defines those as concrete real-world examples!

Event {E}: on October 1, 2022, the PCSO Grand Lotto in the Philippines, which draws six numbers from {1} to {55} at random, managed to draw the numbers {9, 18, 27, 36, 45, 54} (though the balls were actually drawn in the order {9, 45,36, 27, 18, 54}). In other words, they drew exactly six multiples of nine from {1} to {55}. In addition, a total of {433} tickets were bought with this winning combination, whose owners then had to split the {236} million peso jackpot (about {4} million USD) among themselves.

Null hypothesis {H_0}: The lottery is run in a completely fair and random fashion.

Alternative hypothesis {H_1}: The lottery is rigged by some corrupt officials for their personal gain.

Alternative hypothesis {H'_1}: The lottery is rigged by a cult that worships the multiples of {9}, and views October 1 as their holiest day. On this day, they will manipulate the lottery to only select those balls that are multiples of {9}.

So then with substitution:

Remark 1: The contrast between alternative hypothesis that "the lottery is rigged by some corrupt officials for their personal gain" and alternative hypothesis that "the lottery is rigged by a cult that worships the multiples of {9}, and views October 1 as their holiest day. On this day, they will manipulate the lottery to only select those balls that are multiples of {9}" illustrates a common demagogical rhetorical technique when an advocate is trying to convince an audience of an alternative hypothesis, namely to use suggestive language (“`I’m just asking questions here”) rather than precise statements in order to leave the alternative hypothesis deliberately vague. In particular, the advocate may take advantage of the freedom to use a broad formulation of the hypothesis (such as "the lottery is rigged by some corrupt officials for their personal gain") in order to maximize the audience’s prior odds of the hypothesis, simultaneously with a very specific formulation of the hypothesis (such as "the lottery is rigged by a cult that worships the multiples of {9}") in order to maximize the probability of the actual event of "they drew exactly six multiples of nine from {1} to {55}" occurring under this hypothesis. (A related technique is to be deliberately vague about the hypothesized competency of some suspicious actor, so that this actor could be portrayed as being extraordinarily competent when convenient to do so, while simultaneously being portrayed as extraordinarily incompetent when that instead is the more useful hypothesis.) This can lead to wildly inaccurate Bayesian updates of this vague alternative hypothesis, and so precise formulation of such hypothesis is important if one is to approach a topic from anything remotely resembling a scientific approach. [EDIT: as pointed out to me by a reader, this technique is a Bayesian analogue of the motte and bailey fallacy.]

andirk2• 4 years ago

There have been a few national American lotteries in the past handful of years with a jackpot so big ($1.5+ billion) that it was statistically advantageous to buy every number sequence; the total price of the lottery tickets was less than the jackpot. The problem with this lotto hack was that it was not humanly possible to buy that many tickets. Also, if even 1 other person won, the person would be in the hole some 1/2 a billion $ after splitting the winnings.

jmharvey2• 4 years ago

Somewhat counterintuitively, the billion-dollar jackpot lotteries' expected values often shrink as the jackpots grow, because the media circus pulls in a lot of players and increase the likelihood of a split jackpot.

It's not that uncommon for a smaller lottery to have a positive expected value. I think the logistics of buying every combination in a pick-6 lottery (on the order of 10 million combinations) would be rather unwieldy, but a pick-5 (on the order of a few hundred thousand combos) could be doable with a small team.

lupire9• 4 years ago

The "broken" lotteries aren't for that kind of jackpot, for the reason you say. The broken lotteries are counting -caeds based: when a fixed priced pool is printed on the cards, and then part of the deck is exposed as losers so the rest of the deck has positive EV.

tzs• 4 years ago

Virginia lottery around the end of 1991: pick 6 from 1 to 44 giving 7 059 052 possible combinations.

The challengers: International Lotto Fund (ILF), an Australian investor group with 2500 investors.

ILF did't quite manage to cover all 7.1 million combinations. They only managed 5 million. They fell short because they underestimated the time it would take to buy the tickets, which they got from around 8 grocery and convenience store chains.

And no, I'm not some kind of lottery history buff. This is one of those weird bits of trivia one picks up in law school. Specifically in the class I took on transnational tax around 1995, where the taxing of ILF's winnings from this hit some edge cases in US and Australian tax law that were interesting and instructive enough for the case to make the next edition of the textbook.

eyelidlessness• 4 years ago

The odds of N-1 numbers being “remarkable” diminishes for each number drawn. Unless remarkable can be conjured and retconned for every number combination or every Nth number drawn. Each combination is equally likely before the first number is drawn. And equally likely at the end of the draw. But odds change when you have a partial result that’s already ruled out possibilities for each drawing.

You can’t have 9, 18, 27, 36, 45, 54 if you draw an 8. The odds of that are zero.

lqet• 4 years ago

Comment under the article:

  In particular, the geometry of the lottery tickets seems important. It is plausible that 55
  numbers are set up in a 7 x 8 matrix pattern, with one wildcard (to produce 56 objects). Imagine 
  that it is done as such:
  
   *,  1,  2, .. ,  7
   8,  9, 10, ..,  15
  16, 17, 18, … ,  23
  24, 25, 26, 27…, 31
  ..
  
  then multiples of 9 are the main diagonal. That would explain frequency in an easy way.

ginko• 4 years ago

Looks like that's the case.

See: https://primer.com.ph/tips-guides/wp-content/uploads/sites/5...

jzer0cool• 4 years ago

Interesting question for this lottery draw.

1 - How about past lottery dates. Is there also a high purchase of these same sequence of numbers?

2 - Who are the 433 winners? Is there an unusual clustering of the winners? Was it mostly purchased by a single entity?

anonymous344• 4 years ago

This is actually the only smart question(s) to ask to find out was it a cheat

apienx• 4 years ago

ELI5 Bayesian theory. https://youtu.be/HZGCoVF3YvM

botencat• 4 years ago

It’s an interesting read but the author could have just used Kolmogorov complexity to explain all this.

mvcalder• 4 years ago

H''''': The lottery is run by corrupt officials, who want to cover their tracks. In order to do so they want to have many associates win so no single winner is subject to scrutiny. They need to communicate to this diverse group of conspirators the correct numbers to choose in a simple way, say: multiples of 9, or anti-diagonal of ticket.

Remark 4: The human mind is an amazing hypothesis generating machine. If it also knows about Bayesian statistics, it is capable of accounting for it. Paranoia is a bitch.

graycat2• 4 years ago

For any six numbers that can exist or be drawn, once they are drawn can find something about them that is improbable.

Sooooo, that found something improbable about the numbers, once they have been drawn, means essentially nothing.

kgwgk3• 4 years ago

On the other hand, those numbers were "special" before they were drawn - which is why more people played those numbers than would be expected for "non-special" combinations.

lupire9• 4 years ago

"Essentially" is load-bearing there. You can compute the statistics to determine what number sets are more suspicious.

graycat2• 4 years ago

I don't think we are on the same page:

There have to be some uncountably infinitely many statistical tests can apply to six numbers, and given any list of 6 numbers it will have to fail, be very "suspicious", according to some of the tests.

So, in this example, they took six numbers and found a test that it failed. Of course. Can always do that -- given 6 numbers, knowing the numbers, can always find a test that it fails.

There is an old remark that some probability theory guys "didn't believe in statistics". Hmm

simonh2• 4 years ago

A few other ways to thing about the lottery question. What are the chances this sequence would occur in the entire history of the lottery to date? What are the chances that any sequence of multiples of 9 would get drawn? What are the chances that any sequence of multiples of any number would be drawn? What are the chances that any simple mathematical progression of numbers of any kind would be drawn, again in the history of the lottery not just on one day?

I play an online game called Axis & Allies 1942 Online. It uses dice for resolving battles, and in a typical game you might roll many dozens of battles and several handfuls of virtual dice for each battle. We regularly see people go on Discord or the Steam forums to complain that some extraordinarily unlikely outcome happened to them. The thing is with thousands of people playing the game, typically in several games at a time, each rolling hundreds of handfuls of dice every day, one in 10,000 odds outcomes that seem extreme are going to happen on a daily basis to someone. Often several someones a day. Every now and them one of those people is likely to go and complain about it online, so IMHO this is an expected outcome. So far this argument doesn't seem to have convinced many of the 'victims' though.

MarkusWandel• 4 years ago

There's a lot of math here, but what stands out for me is very simple. If you're going to play the lottery, play numbers that have the lowest odds of others playing them too. So any cute sequences, valid dates and so on are right out. You have the same chance of winning with truly random, weird numbers. But you increase your odds of not having to share.

What Are the Odds?

Related Stories