An OpenAI model has disproved a central conjecture in discrete geometry

https://openai.com/index/model-disproves-discrete-geometry-conjecture/

by tedsanders • 2 months ago

1429 1045 2 months ago

openai.com

https://x.com/wtgowers/status/2057175727271800912, https://xcancel.com/wtgowers/status/2057175727271800912

aurareturn2• 2 months ago

One thing seems for certain is that OpenAI models hold a distinct lead in academics over Anthropic and Google models.

For those in academics, is OpenAI the vendor of choice?

FloorEgg3• 2 months ago

Gemini seems better trained for learning and I think Google has made a more deliberate effort to optimize for pedagoical best practices. (E.g. tutoring, formative feedback, cognitive load optimization)

As far as academic research is concerned (e.g. this threads topic), I can't say.

aurareturn2• 2 months ago

Yes, I meant academic research.

cute_boi• 2 months ago

Gemini is like someone with short-term memory loss; after the first response, it forgets everything. That being said, I have checked multiple model and gemini can sometime give accurate answer.

snaking0776• 2 months ago

Agreed I usually use Gemini for explaining concepts and ChatGPT for getting things done on research projects.

astrange4• 2 months ago

Gemini the chatbot has a very strange personality that intensely overindexes on your user profile and absolutely loves insane mixed metaphors.

Its explanations are quite good but they're also hard to understand because it keeps trying to relate everything back to programming metaphors or what it thinks it knows about the streets in the neighborhood I live in.

causal• 2 months ago

A simpler explanation is that more people are using ChatGPT

karmasimida4• 2 months ago

I think the mathematicians on X are all using GPT 5.5 Pro

ai_fry_ur_brain4• 2 months ago

And they all think its garbage. This is a publicity stunt.

bayindirh4• 2 months ago

From my limited testing, Gemini can dig out hard to find information given you detail your prompt enough.

Given that Google is the "web indexing company", finding hard to find things is natural for their models, and this is the only way I need these models for.

If I can't find it for a week digging the internet, I give it a colossal prompt, and it digs out what I'm looking for.

senrex• 2 months ago

This is my experience too. Gemini and Gemini deep research are awesome. Claude's deep research is pretty bad really relative to ChatGPT or Gemini. Overall, I still love Claude the best but it is not what I would want to use if I wanted to really dig into deep research. The export to google docs in Gemini deep research is tough to beat too. I haven't used Gemini since January but have probably years of material from saved deep research in google docs. Almost an overwhelming amount of information when I dive into what I saved.

Jcampuzano2• 2 months ago

OpenAI specifically targeted Academia a lot and gave out a lot of free/unlimited usage to top academics and universities/researchers.

They also offer grants you can apply for as a researcher. I'm sure other labs may have this too but I believe OpenAI was first to this.

tracerbulletx• 2 months ago

Hasn't AlphaFold been used to make real discoveries for a few years now?

KalMann5• 2 months ago

I think he's talking about reasoning models.

logicchains2• 2 months ago

OpenAI models seem to have been trained on a lot of auto-generated theorem proving data; GPT 5.5 is really good at writing Lean.

empath752• 2 months ago

Important note: this was not done with a special mathematics harness or specialized workflow.

dwroberts3• 2 months ago

How/why should we know this, it does not explain the process in the text?

horhay7• 2 months ago

This part of the announcement holds no value besides maybe taking a shot at the Deepmind Co-Mathematician paper. Nearly every mathematical success they've achieved around the GPT 5.2 generation has been done with general (and even public) models. Their last bountied problem solved was done with 5.4 Pro, also a general model.

Jeff_Brown• 2 months ago

Can anyone find (or draw) a picture of the construction?

pradn• 2 months ago

They have a "before" picture but not an "after"!

ninjha• 2 months ago

They only proved that one exists; computing the actual construction is non-obvious (the naive way to construct it is computationally infeasible).

gibspaulding• 2 months ago

This only a proof that a field with more connections is possible, not what it looks like.

I’m very out of my depth, but the structure of the proof seems to follow a pattern similar to a proof by contradiction. Where you’d say for example “assume for the sake of contradiction that the previously known limit is the highest possible” then prove that if that statement is true you get some impossible result.

paulddraper4• 2 months ago

Yeah, unfortunately, they just proved there existed a better solution, they didn't construct it.

(Though in some ways that's actually more impressive.)

krackers4• 2 months ago

See https://news.ycombinator.com/item?id=48233034

alansaber• 2 months ago

AI isn't going to supercharge science but I wouldn't be as dismissive as other posters here.

OldGreenYodaGPT• 2 months ago

Isn’t that a joke? It already has supercharged science

datsci_est_2015• 2 months ago

Where are the second order effects of this supercharging of science? Or has it not been enough time for those to propagate?

ks20482• 2 months ago

Since "supercharged science" is as ill-defined as AGI, ASI, etc., people will be able to debate it endlessly for no reason.

vatsachak14• 2 months ago

I absolutely believe that AI will supercharge science.

I do not believe it will replace humans.

seydor3• 2 months ago

replace, no. obsolete, yes

unsupp0rted2• 2 months ago

I absolutely believe that AI will supercharge science and replace humans.

Why shouldn't it? Humans are poorly optimized for almost anything, and built on a substrate that's barely hanging together

renegade-otter2• 2 months ago

It will notice things that humans may have missed. That said - it can only work off the body of work SOMEONE did in the past.

throw-the-towel4• 2 months ago

it can only work off the body of work SOMEONE did in the past.

And so do humans. Gotta stand on these shoulders of giants.

bel8• 2 months ago

Can't the previous body of work be from AI too?

karmasimida4• 2 months ago

To be strict, Math is not Science.

But AI is supercharging Math like there is no tomorrow.

anthk6• 2 months ago

LLM's? I doubt it. Systems with Prolog, Common Lisp and the like with proof solvers? For sure.

LLM's are doomed to fail. By design. You can't fix them. It's how do they work.

comboy2• 2 months ago

Not only it supercharged science it supercharges scientist. Research on any narrow topic is a different world now. Agents can read 50 papers for you and tell you what's where. This was impossible with pure text search. Looking up non-trivial stuff and having complex things explained to you is also amazing. I mean they don't even have to be complex, but can be for adjacent field where these are basics from the other field but happen to be useful in yours. The list goes on. It's a hammer you need to watch your fingers, it's not good at cutting wood, but it's definitely worth having.

dvfjsdhgfv4• 2 months ago

It's a very heavy hammer. I used it in the way you describe and after double-checking noticed some crucial details were missed and certain facts were subtly misrepresented.

But I agree with you, especially in areas where they have a lot of training data, they can be very useful and save tons of time.

tombert2• 2 months ago

I'm not a scientist but I like to LARP as one in my free time, and I have found ChatGPT/Claude extremely useful for research, and I'd go as far as to say it supercharged it for me.

When I'm learning about a new subject, I'll ask Claude to give me five papers that are relevant to what I'm learning about. Often three of the papers are either irrelevant or kind of shit, but that leaves 2/5 of them that are actually useful. Then from those papers, I'll ask Claude to give me a "dependency graph" by recursing on the citations, and then I start bottom-up.

This was game-changing for me. Reading advanced papers can be really hard for a variety of reasons, but one big one can simply be because you don't know the terminology and vernacular that the paper writers are using. Sometimes you can reasonably infer it from context, but sometimes I infer incorrectly, or simply have to skip over a section because I don't understand it. By working from the "lowest common denominator" of papers first, it generally makes the entire process easier.

I was already doing this to some extent prior to LLMs, as in I would get to a spot I didn't really understand, jump to a relevant citation, and recurse until I got to an understanding, but that was kind of a pain in the ass, so having a nice pretty graph for me makes it considerably easier for me to read and understand more papers.

kingkongjaffa• 2 months ago

One heuristic I used during my masters degree research thesis was to look for the seminal people or papers in a field by using google scholar to find the most cited research papers and then reading everything else by that author / looking at the paper's references for others. You often only need to go back 3-4 papers to find some really seminal/foundational stuff.

horhay7• 2 months ago

It's a very complicated matter honestly. This is a new height that AI has reached, even though it follows the usual methods of success that it has had.

What strikes me as unusual though is that they do make a point of saying things like "this is a general purpose model that wasn't trained on the problem" among a few other things as if that's new. The last bountied problem they accomplished used a public model that ALSO didn't rely on specialized training. And that didn't make their blog.

vatsachak14• 2 months ago

As I have stated before, AI will win a fields medal before it can manage a McDonald's

A difficult part was constructing a chess board on which to play math (Lean). Now it's just pattern recognition and computation.

LLMs are just the beginning, we'll see more specialized math AI resembling StockFish soon.

soupspaces2• 2 months ago

Lee Sedol, Move 37 https://www.reddit.com/r/singularity/comments/1l0z5yk/the_mo... Edit: I wasn't necessarily disagreeing. But on second thought the chessboard in this math analogy is being built, not just played in. This Hardy quote comes to mind https://www.goodreads.com/quotes/902543-it-proof-by-contradi...

vatsachak14• 2 months ago

My claim is that we haven't even witnessed the move 37 of math yet. I am claiming that math AI is going to get even better

Lerc• 2 months ago

I disagree. It will be able to perform work deserving if a fields medal before it is capable of running a McDonalds. I think it will be running a McDonalds well before either of those things happen, and a fields medal long after both have happened.

edbaskerville• 2 months ago

I just visited a McDonald's for the first time in a while. The self-order kiosk UI is quite bad. I think this is evidence in favor of the idea that an incompetent AI will soon be incompetently running a McDonald's.

c7b• 2 months ago

One could hardly ask for a task better suited for LLMs than producing math in Lean. Running a restaurant is so much fuzzier, from the definition of what it even means to the relation of inputs to outputs and evaluating success.

vatsachak14• 2 months ago

Not necessarily. Obviously playing Kasparov on the board requires more planning ability than managing a McDonald's but look at where chess bots are now.

There's much more to being human than our "cognitive abilities"

sigmoid106• 2 months ago

Managing a McDonalds is a question of integration and modalities at this point. I don't think anyone still doubts that these models lack the reasoning capability or world knowledge needed for the job. So it's less of a fundamental technical problem and more of a process engineering issue.

throw-the-towel4• 2 months ago

The capability they lack is being able to be sued.

andy12_2• 2 months ago

I disagree. Even frontier models still achieve way worse results than the human baseline in VendingBench. As long as models can't manage optimally something as simple as a vending machine, they have no hope of managing a McDonalds.

dap• 2 months ago

Have you not seen:

https://www.anthropic.com/research/project-vend-1 https://www.wsj.com/tech/ai/anthropic-claude-ai-vending-mach...

(Two different examples of a similar idea)

forinti• 2 months ago

AI is already too old for that.

Terr_• 2 months ago

manage a McDonald's

Dystopia vibes from the fictional "Manna" management system [0] used at a hamburger franchise, which involved a lot of "reverse centaur" automation.

At any given moment Manna had a list of things that it needed to do. There were orders coming in from the cash registers, so Manna directed employees to prepare those meals. There were also toilets to be scrubbed on a regular basis, floors to mop, tables to wipe, sidewalks to sweep, buns to defrost, inventory to rotate, windows to wash and so on. Manna kept track of the hundreds of tasks that needed to get done, and assigned each task to an employee one at a time. [...]
At the end of the shift Manna always said the same thing. “You are done for today. Thank you for your help.” Then you took off your headset and put it back on the rack to recharge. The first few minutes off the headset were always disorienting — there had been this voice in your head telling you exactly what to do in minute detail for six or eight hours. You had to turn your brain back on to get out of the restaurant.

[0] https://en.wikipedia.org/wiki/Manna_(novel)

kmeisthax2• 2 months ago

Casual reminder that the author's proposed solution to the labor-automation dystopia is to invent a second identity-verification dystopia. Also casual reminder that the author wanted the death penalty to anyone over the age of 65.

tomjakubowski• 2 months ago

Amazing bit of trivia that the founder of HowStuffWorks.com was named Marshall Brain.

evenhash• 2 months ago

The proof is not written in Lean, though. It’s written in English and requires validation by human experts to confirm that it’s not gibberish.

vatsachak14• 2 months ago

Yeah, but I wouldn't be surprised if they train the model on verification assisted by Lean

whimsicalism• 2 months ago

the only thing keeping the mcdonalds from happening will be political, likewise the same with fields medal

trostaft5• 2 months ago

A difficult part was constructing a chess board on which to play math (Lean). Now it's just pattern recognition and computation.

However, this was not verified in Lean. This was purely plain language in and out. I think, in many ways, this is a quite exciting demonstration of exactly the opposite of the point you're making. Verification comes in when you want to offload checking proofs to computers as well. As it stands, this proof was hand-verified by a group of mathematicians in the field.

vatsachak14• 2 months ago

Yeah, but I wouldn't be surprised if they train the model on verification assisted by Lean.

ComplexSystems• 2 months ago

That may be true for now, but it seems clear enough that letting the model use Lean in its internal reasoning process would be a great idea

NooneAtAll3• 2 months ago

how would they calculate "probability of solving" without automated verification?

ken47• 2 months ago

However, this was not verified in Lean.

This is the caliber of thinking in unimpaired AI bullishness.

segmondy• 2 months ago

our local AI models are already capable of running McDonalds.

hoppyhoppy2• 2 months ago

Why aren't they doing so?

ric2b2• 2 months ago

No they aren't, they can't even manage a vending machine.

volkercraig• 2 months ago

we'll see more specialized math AI resembling StockFish soon

Heuristically weighted directed graphs? Wow amazing I'm sure nobody has done that before.

vatsachak14• 2 months ago

My claim is that LLMs waste a lot of time training on all available data.

Math is a sequence of formal rules applied to construct a proof tree. Therefore an AI trained on these rules could be far more efficient, and search far deeper into proof space

ori_b8• 2 months ago

We're automating art and science so that we can flip burgers. This future sucks.

vatsachak14• 2 months ago

Math is a very specialized subset of art and science more amenable to automation.

dyauspitr4• 2 months ago

No, we’re not going to be flipping burgers either, they will have physical robots for that. 20 years down the line I wonder what work all of us will be doing.

dyauspitr4• 2 months ago

Nonsense. Have you been watching the figure live stream? Or the Unitree video from yesterday with real time novel action generation? We’re less than a year away. If you can cook a burger, assemble a sandwich and clean up surfaces you’re all of the way there.

vatsachak14• 2 months ago

Fair. Let's see in a year. I'm willing to bet that nothing happens.

KalMann5• 2 months ago

I think your analogy is good but I don't believe modern LLMs use Lean or any lean-like structure in their proofs. At least recent open source ones like DeepSeek can do advanced math without it (maybe the most cutting edge ones are doing it I can't say).

vatsachak14• 2 months ago

They are most likely using them in training. I doubt their IMO team are show ponies

auggierose2• 2 months ago

A difficult part was constructing a chess board on which to play math

We have that chess board for quite a while now, over 40 years. And no, there is nothing special about Lean here, it is just herd mentality. Also, we don't know how much training with Lean helped this particular model.

fapjacks• 2 months ago

I dunno. Is AI less than forty years old?

brikym• 2 months ago

Hey ChatGPT, if a person spills hot McCoffee on themselves who is at fault?

brookst2• 2 months ago

Well, brikym, exactly how hot is this hot coffee? If it’s within normal expectations for coffee it is likely that person’s fault. If it is 210 degrees F, it is likely McDonald’s fault.

energy1238• 2 months ago

The issue with this prediction is the gulf between problem-solving using known tools, versus creating new concepts for problems where existing tools aren't enough.

All AI proofs so far, including this one, are using existing tools in new ways, rather than inventing new tools. This is not surprising if you know how these models are trained. These existing tools are in distribution. New tools are not.

Problems worth of a Fields Medal likely require new tools to be invented. Thus it is not clear whether progress within the confines of the current paradigm is enough.

We could get this weird spiky situation where the AI is insanely superhuman at all problem solving, but completely incapable of coming up with a single new tool. It discovers everything there is to discover, subject to existing axioms and concepts.

Timothy Gowers gives some commentary on this in the attached PDF.

jeremyjh2• 2 months ago

Stockfish did not teach itself to play chess. You are probably thinking of Leela Chess Zero - an open re-implementation of AlphaZero - both were given nothing but the rules of chess and a board and played millions of games against themselves until they were the strongest engine available at the time.

Stockfish's neural net evaluation model was trained on millions of its positions with its own original algorithmic evaluation function (entirely developed by humans) and search tree. The result was a much smaller model than Leela's that requires little computation (not even a GPU), paired with its already extremely efficient search/pruning algorithms that made it stronger than Leela in competitive play. Leela's evaluation function is much stronger (at one ply it has an ELO of around 2300, Stockfish is probably closer to 1800), but it requires vastly more resources and those are always bounded in a match.

Humans haven't learned as much new information about chess from Stockfish as we have from Leela.

xigoi4• 2 months ago

at one ply it has an ELO of around 2300, Stockfish is probably closer to 1800

Nitpick: it’s Elo, not ELO. The name comes from the inventor’s surname and is not an acronym.

yusufozkan2• 2 months ago

"The proof came from a general-purpose reasoning model, not a system built specifically to solve math problems or this problem in particular, and represents an important milestone for the math and AI communities."

Kwantuum• 2 months ago

I trust openAI's marketing team 100%

krackers4• 2 months ago

It seems plausible given that people have been using off the shelf 5.5 xhigh to decent success with some erdos problems. There is likely still some scaffolding around it though (like parallel sampling or separate verifier step) since it's not clear if you can just "one shot" problems like this.

seydor3• 2 months ago

all reasoning is .. well problem reasoning. restricting black-box AIs to specific human-defined domains because we believe that's better is such a human-ist thing to do.

horhay7• 2 months ago

The accomplishment is cool. But all Erdos problems and other complicated mathematical problems they solved were accomplished with general-purpose models too. In fact for some of those problems, including bountied ones, they were public models. So I don't get saying this

endymi0n• 2 months ago

To paraphrase Gwynne Shotwell: “Not too bad for just a large Markov chain, eh?”

rhubarbtree• 2 months ago

Erdos, or the model?

phkahler• 2 months ago

I would have thought a triangular grid works better than a grid of squares. You get ~3n links vs ~2n for the square grid. Curious what the AI came up with.

comboy2• 2 months ago

Yes, not providing visualization of the solution seems criminal.

kmeisthax2• 2 months ago

Knowing OpenAI, the solution's probably being withheld as a trade secret, lest it fall victim to distillation attacks (i.e. exactly the same shit they did to the open Internet).

red_admiral• 2 months ago

Unless it's a non-constructive proof.

kilotaras• 2 months ago

Both 3n and 2n are linear, the broken conjecture is that you can't do better than linear.

bustermellotron• 2 months ago

The grid of squares actually gets > Cn for any C. (More in fact… C can grow like n^a/loglog(n).) The AI proved > n^{1 + b} for some small b > 0, which a human (Will Sawin) has now proved can be about b = 0.014. The grid can be rescaled so the edges are not necessarily length 1, but other pairs will have length 1; that is necessary to get more than 2n unit distances.

dadrian• 2 months ago

While the result is impressive, this blog post is extremely disappointing.

- It does not show an example of the new best solution, nor explain why they couldn't show an example (e.g. if the proof was not constructive)

- It does not even explain the previous best solution. The diagram of the rescaled unit grid doesn't indicate what the "points" are beyond the normal non-scaled unit grid. I have no idea what to take away from it.

- It's description of the new proof just cites some terms of art with no effort made to actually explain the result.

If this post were not on the OpenAI blog, I would assume it was slop. I understand advanced pure mathematics is complicated, but it is entirely possible to explain complicated topics to non-experts.

Al-Khwarizmi• 2 months ago

Indeed, it's a pity. While many advanced math problems are highly abstract or convoluted to explain to a layman audience, this one in particular is about points in a 2D plane and distances. A drawing would have been nice.

changoplatanero• 2 months ago

apparently the proof is not constructive in the sense of not giving an easy to compute recipe for generating a set of points that you can plot on a 2d plane

solomatov• 2 months ago

How central is it in the discrete geometry? Could anyone with the knowledge in the field reply?

energy1238• 2 months ago

There's pages of comments from like 8 mathematicians in the attached pdf

sigmar• 2 months ago

The blog post links a pdf that OpenAI put together of nine mathematicians that commented on the proof. Each is quite brief and written in accessible terms (or more accessible terms, at least). https://cdn.openai.com/pdf/74c24085-19b0-4534-9c90-465b8e29a...

horhay7• 2 months ago

There may be years of investigation as to how far you can generalize these methods. As to how central it is, it's a longstanding problem that Erdos loves to cite for that branch of math.

The thing is is that it seems a lot of the effort through the years (which is unquantifiable in scale as to how much time was spent and how many people focused their entire worklives on it if any) has gone for trying to look for the proof, and the search for the disproof seems minimal.

Lockal• 2 months ago

No separate Wikipedia page -> just another Erdős problem.

There is no universally agreed-upon "central" conjecture (like "P vs. NP" in CS), but here are some pillars:

1) https://en.wikipedia.org/wiki/Happy_ending_problem

2) https://en.wikipedia.org/wiki/Hadwiger_conjecture_(combinato...

3) https://en.wikipedia.org/wiki/Hirsch_conjecture

reactordev2• 2 months ago

I dunno, I'm skeptical without proof. I've had the MAX+ plan for a while and I'm sorry, the quality between GPT vs Claude is night and day difference. Claude understands. GPT stumbles over every request I give it.

nathan_compton3• 2 months ago

Weird thing to say about a report which literally has the attached mathematical proof.

reactordev2• 2 months ago

Except its not a proof. It's an existential proof of what? Projecting points and loosing density? Nah, it's wrong. At least with Edros you could solve f(x) or not solve it (inf). You can not with this. All they did was balance a really fancy quadratic equation. The projection from C^f to R² doesn't demonstrate geometric injectivity, so nⱼ = |X| isn't established, and the bound collapses.

0x5FC32• 2 months ago

Is there a reason why we only hear of Erdos problems being solved? I would imagine there are a myriad of other unsolved problems in math, but every single ChatGPT "breakthrough in math" I come across on r/singularity and r/accelerate are Erdos problems.

tonfa• 2 months ago

Afaik this is because there is a community and database around them.

0x5FC32• 2 months ago

Interesting. OpenAI could also be trying to solve other problems, but Erdos problems maybe falling first?

throw-the-towel4• 2 months ago

They're just famous because Erdos was a great mathematician, kinda like the Hilbert problems a century earlier.

empath752• 2 months ago

It's a large set of problems that are both interesting and difficult, but not seen as foundational enough or important enough that they have already had sustained attention on them by mathematicians for decades or centuries, and so they might actually be solvable by an LLM.

1qaboutecs• 2 months ago

Also fewer prerequisites to understand the statement than the average research problem.

bananaflag8• 2 months ago

Erdos problems are easier to state, thus they make a great benchmark for the first year of AI mathematics.

famouswaffles4• 2 months ago

It's not just Erdos problems - https://news.ycombinator.com/item?id=48213189

jltsiren• 2 months ago

Erdős problems form a substantial fraction of all mathematical problems that have been explicitly stated but not solved; are sufficiently famous that people care about them; and are sufficiently uninteresting that people have not spent that much effort trying to solve them.

Solving problems people have already stated is a niche activity in mathematical research. More often, people study something they find interesting, try to frame it in a way that can be solved with the tools they have, and then try to come up with a solution. And in the ideal case, both the framing and the solution will be interesting on their own.

edanm2• 2 months ago

and [Erdős problems are] sufficiently uninteresting that people have not spent that much effort trying to solve them.

Note that this is not really true of this problem in particular.

dandaka4• 2 months ago

Do we have a list of open problems? Would love to see a chart, where AI solves such problems one by one in the upcoming years.

odie5533• 2 months ago

I was promised a cure for cancer, but all I got was this disproof of an Erdos problem.

xyzsparetimexyz3• 2 months ago

The models can't actually so good work on practical problems so openai tasks them on stuff nobody cares about

TrackerFF• 2 months ago

As others have written, Erdős was a lifelong curator of mathematical problems, from high-school level problems to the types that will land you a Fields medal. Like the Collatz conjecture.

Most new math problems appear in other papers, doctoral dissertations, etc. Usually you'll find them in the "future work" / "future research" section.

So obviously in order to present and formalize these problems, you either need the author(s) to do it, or some reader. At this level of math, there are many extremely niche fields, where the papers might only be read by a small amount of people.

In short, it is a visibility problem.

But, I figure, there's some potential use in AI models to extract and present these problems, which would make them available to a larger audience.

That is exactly what Erdős did. His life revolved around math, and seeking mathematical questions.

cold_harbor• 2 months ago

Erdos problems are well-posed for AI — elementary statements, exact counterexample targets, extensively catalogued. selection bias: these are exactly the problems AI can actually search

bradleykingz• 2 months ago

ok. so what are the implications of for math

jaimex2• 2 months ago

absolutely none

Fraterkes• 2 months ago

I guess if this stuff is going to make my employment more precarious, it’d be nice if it also makes some scientific breakthroughs. We’ll see

ausbah• 2 months ago

shame we won’t see any of these medical breakthroughs when we all lose our jobs and thus our healthcare

karmasimida4• 2 months ago

There is a world that AI makes medical breakthroughs, but there is 0 guarantee it is going to be affordable

cubefox3• 2 months ago

Breakthroughs in pure mathematics aren't scientific though. They say us nothing about the world, and they are not useful.

zozbot234• 2 months ago

The summarized chain of thought for this task (linked in the blogpost) is 125 pages. That's an insane scale of reasoning, quite akin to what Anthropic has been teasing with Mythos.

estetlinus• 2 months ago

Today I generated the equivalent of two LOTR books just to fix three missing rows in my SQL models (and open a PR), so +1

wayeq• 2 months ago

or put differently, you melted x cubic meters of polar ice

devttyeu2• 2 months ago

That's here for anyone wondering - https://cdn.openai.com/pdf/1625eff6-5ac1-40d8-b1db-5d5cf925d...

gilgoomesh• 2 months ago

I'm disappointed only that the chain of thought needed to be rewritten. Need to train these LLMs to natively communicate in LaTeX research paper format.

noobermin• 2 months ago

So, I'm not really a mathematician, but the first 3-8 pages reads like nonsense and a bunch of unrelated facts. A bit surreal may be, but if this the norm for this kind of thing, I'm amazed it arrives at any useful result at all.

Chamix• 2 months ago

I note that (though summarized), this is ~100k tokens. Anyone who routinely works with Codex (or any agentic harness really) can tell you how trivial it is to eat up 100k tokens doing complex work. I've personally had plenty of codex 5.5 xhigh sessions where just the pure chain of thought token count in a single turn exceeds 200k (and I assume doesn't go further only due to compaction meta-guidance; the harness will push the model to stay under 256k per turn/thinking block) .

I think the more interesting question is how many tokens were spent all told; the most interesting graph in the article imo is the success rate by log test-time compute: how many tokens are being spent on the right of the graph to hit a winning CoT/solution like this >50% of the time?

stratos123• 2 months ago

AFAIK, extremely long CoT is fairly routine for those math-finetuned private models. Also, is Mythos unusually verbose compared to Opus?

m-hodges• 2 months ago

To the “LLMs just interpolate their training data” crowd:

Ayer, and in a different way early Wittgenstein, held that mathematical truths don’t report new facts about the world. Proofs unfold what is already implicit in axioms, definitions, symbols, and rules.

I think that idea is deeply fascinating, AND have no problem that we still credit mathematicians with discoveries.

So either “recombining existing material” isn’t disqualifying, or a lot of Fields Medals need to be returned.

dvt4• 2 months ago

I think that idea is deeply fascinating, AND have no problem that we still credit mathematicians with discoveries.

Most discoveries are indeed implied from axioms, but every now and then, new mathematics is (for lack of a better word) "created"—and you have people like Descartes, Newton, Leibniz, Gauss, Euler, Ramanujan, Galois, etc. that treat math more like an art than a science.

For example, many belive that to sovle the Riemann Hypothesis, we likely need some new kind of math. Imo, it's unlikely that an LLM will somehow invent it.

Tenobrus• 2 months ago

what basis do you have for assuming an LLM is fundamentally incapable of doing this?

pulkitsh1234• 2 months ago

Creation is done by humans who have been trained on the data of their life experiences. Nothing new is being created, just changing forms.

A scientist has to extract the "Creation" from an abstract dimension using the tools of "human knowledge". The creativity is often selecting the best set of tools or recombining tools to access the platonic space. For instance a "telescope" is not a new creation, it is recombination of something which already existed: lenses.

How can we truly create something ? Everything is built upon something.

You could argue that even "numbers" are a creation, but are they ? Aren't they just a tool to access an abstract concept of counting ? ... Symbols.. abstractions.

Another angle to look at it, even in dreams do we really create something new ? or we dream about "things" (i.e. data) we have ingested in our waking life. Someone could argue that dream truly create something as the exact set of events never happened anywhere in the real world... but we all know that dreams are derived.. derived from brain chemistry, experiences and so on. We may not have the reduction of how each and every thing works.

Just like energy is conserved, IMO everything we call as "created" is just a changed form of "something". I fully believe LLMs (and humans) both can create tools to change the forms. Nothing new is being "created", just convenient tools which abstract upon some nature of reality.

kenjackson• 2 months ago

"new kind of math"

Well I think the point is there is no "new kind of math". There's just types of math we've discovered and what we haven't. No new math is created, just found.

bbor5• 2 months ago

  math more like an art than a science.

That’s a fun turn of phrase, but hopefully we can all agree that math without scientific rigor is no math at all.

  we likely need some new kind of math. Imo, it's unlikely that an LLM will somehow invent it.

Do you think it’s possible/likely that any AI system could? I encourage us to join Yudkowsky in anticipating the knock-on results of this exponential improvement that we’re living through, rather than just expecting chatbots that hallucinate a bit less.

In concrete terms: could a thousand LLMs-driven agents running on supercomputers—500 of which are dedicated to building software for the other 500-come up with new math?

Someone• 2 months ago

I think “new math” is ‘just’ humans creating new terminology that helps keep proofs short (similar to how programmers write functions to keep the logic of the main program understandable), and I agree that is something LLMs are bad at.

However, if that idea about new math is correct, we, in theory, don’t need new math to (dis)prove the Riemann hypotheses (assuming it is provable or disprovable in the current system).

In practice we may still need new math because a proof of the Riemann hypotheses using our current arsenal of mathematical ‘objects’ may be enormously large, making it hard to find.

black_knight5• 2 months ago

It could be that RH is independent of current mathematical axiom systems. We might even prove that it is some day. But that means we are free to give it different truth values depending on the circumstances!

This is also true for established theorems! We can can imagine mathematical universes (toposes) where every (total) function on the reals is continuous! Even though it is an established theorems that there are discontinuous functions! We just need to replace a few axioms (chuck out law of the excluded middle, and throw in some continuity axioms).

voooduuuuu3• 2 months ago

I think you are conflating composition and prediction. LLMs don't compose higher abstractions from the "axioms, symbols and rules", they simply predict the next token, like a really large spinning wheel.

adampunk• 2 months ago

How sure are you that this is correct?

peterlk• 2 months ago

Yes they do…? Who cares if they just predict the next token? The outcome is that they can invent new abstractions. You could claim that the invention of this new idea is a combination of an LLM and a harness, but that combination can solve logic puzzles and invent abstractions. If a really large spinning wheel could invent proofs that were previously unsolved, that would be a wildly amazing spinning wheel. I view LLMs similarly. It is just fancy autocomplete, but look what we can do with it!

Said differently, what is prediction but composition projected forward through time/ideas?

sunshowers• 2 months ago

One might argue that the composition of higher abstractions is the next token predicted after "here is a higher abstraction:"

frozenseven2• 2 months ago

Show me on the anatomical prop where the magical "real reasoning" gland is.

umanwizard5• 2 months ago

"Predicting the next token" is meaningless. Every process that has any sort of behavior, including a human writing, can be modeled by some function from past behavior to probability distribution of next action. Viewed this way, literally everything is just "predicting" the next action to be taken according to that probability distribution.

The most likely series of next tokens when a competent mathematician has written half of a correct proof is the correct next half of the proof. I've never seen anyone who claims "LLMs just predict the next token" give any definition of what that means that would include LLMs, but exclude the mathematician.

adam_arthur• 2 months ago

Pretty much everything that appears novel in life is derivative of other works or concepts.

You can watch a rock roll down a hill and derive the concept for the wheel.

Seems pretty self evident to me

paulddraper4• 2 months ago

"LLMs just interpolate their training data"

Cracks me up.

What exactly do we think that human brains do?

omnimus2• 2 months ago

That has been the question since the beginning of humans.

Maybe computers can help understand better because by now it's pretty clear brains aren't just LLMs.

gpugreg• 2 months ago

Maybe the human brain also does other things besides interpolation?

ActorNightly• 2 months ago

I love this comment because it so clearly highlights the difference between intelligence and reasoning.

A lot of people across all fields seem to operate in a mode of information lookup as intelligence. They have the memory of solving particular problems, and when faced with a new problem, they basically do a "nearest search" in their brain to find the most similar problem, and apply the same principles to it.

While that works for a large number of tasks this intelligence is not the same as reasoning.

Reasoning is the ability to discover new information that you haven't seen before (i.e growing a new branch on the knowledge tree instead of interpolating).

Think of it like filling a space on the floor of arbitrary shape with smaller arbitrary shapes, trying to fill as much space as possible.

With interpolation, your smaller shapes are medium size, each with a non rectangular shape. You may have a large library of them, but in the end, there are just certain floor spaces that you won't be able to fill fully.

Reasoning on the flip side is having access to very fine shape, and knowing the procedure of how to stack shapes depending on what shapes are next to it and whether you are on a boundary of the floor space or not. Using these rules, you can fill pretty much any floor space fully.

charlie902• 2 months ago

I agree. Humans are given a body that lets them "discover" things on accident, test out ideas, i.e. randomness.

As in, I would hazard a guess the discovery of the wheel wasn't "pure intelligence", it was humans accidentally viewing a rock roll down a hill and getting an idea.

If we give AI a "body", it will become as creative as humans are.

__s• 2 months ago

Creativity is hard. Pretty much needs a fuzzer process to generate new strings, mostly nonsense, & pick up when that nonsense happens to be correct

oh_my_goodness2• 2 months ago

We don't know what human brains do.

slashdave3• 2 months ago

You have to define what you mean by "interpolate". The mechanisms that LLM use are not mysterious, and they are not the same as used by humans.

sillysaurusx2• 2 months ago

It’s easy to see that LLMs don’t merely recombine their training data. Claude can program in Arc, a mostly dead language. It can also make use of new language constructs. So either all programming language constructs are merely remixes of existing ideas, or LLMs are capable of working in domains where no training data exists.

baq7• 2 months ago

LLMs ingest and output tokens, but they don’t compute with them. They have internal representations of concepts, so they have some capability to work with things which they didn’t see but can map onto what they know. The surprise and the whole revolution we’re going through is that it works so well.

HarHarVeryFunny• 2 months ago

They recombine and reuse the patterns in their training data, not the surface level training data itself.

An LLM generating Arc code is using the LISP patterns it learnt from training, maybe patterns from other programming languages too.

bsder2• 2 months ago

So either all programming language constructs are merely remixes of existing ideas, or LLMs are capable of working in domains where no training data exists.

And yet LLM/AIs can't count parentheses reliably.

For example, if you take away the "let" forms from Claude which forces it to desugar them to "lambda" forms, it will fail very quickly. This is a purely mechanical transformation and should be error free. The significant increase in ambiguity complete stumps LLMs/AI after about 3 variables.

This is why languages like Rust with strong typing and lots of syntax are so LLM friendly; it shackles the LLM which in turn keeps it on target.

nomel2• 2 months ago

I feel this is the case whenever I "problem solve". I'm not really being creative, I'm pruning a graph of a conceptual space that already exists. The more possibilities I see, the easier it is to run more towards an optimal route between the nodes, but I didn't "create" those nodes or edges, they are just causal inevitabilities.

HDThoreaun• 2 months ago

I dont know this sort of just seems like youre really stretching the meaning of "creative". The conceptual space of the graph already exists, but the act of discovering it or whatever you want to call that is itself creative. Unless youre following a pre-defined algorithm(certainly sometimes, arguably always I suppose) seeing the possibilities has to involve some creativity.

awesome_dude6• 2 months ago

There was a project long long ago where every piece of knowledge known was cross pollinated with every other piece of knowledge, creating a new and unique piece of knowledge, and it was intended to use that machine to invalidate the patent process - obviously everything had therefore been invented.

But that's not how new frontiers are conquered - there's a great deal of existing knowledge that is leveraged upon to get us into a position where we think we can succeed, yes, but there's also the recognition that there is knowledge we don't yet have that needs to be acquired in order for us to truly succeed.

THAT is where we (as humans) have excelled - we've taken natural processes, discovered their attributes and properties, and then understood how they can be applied to other domains.

Take fire, for example, it was in nature for billions of years before we as a species understood that it needed air, fuel, and heat in order for it to exist at all, and we then leveraged that knowledge into controlling fire - creating, growing, reducing, destroying it.

LLMs have ZERO ability (at this moment) to interact with, and discover on their own, those facts, nor does it appear to know how to leverage them.

edit: I am going to go further

We have only in the last couple of hundred years realised how to see things that are smaller than what our eye's can naturally see - we've used "glass" to see bacteria, and spores, and we've realised that we can use electrons to see even smaller

We're also realising that MUCH smaller things exist - atoms, and things that compose atoms, and things that compose things that compose atoms

That much is derived from previous knowledge

What isn't, and it's what LLMs cannot create - is tools by which we can detect or see these incredible small things

hammock• 2 months ago

Recombining existing material is exactly right, and in this case LLMs were uniquely positioned to make the connection quicker than any group of humans.

The proof relies on extremely deep algebraic number theory machinery applied to a combinatorial geometry problem.

Two humans expert enough in either of those totally separate domains would have to spend a LONG time teaching each other what they know before they would be able to come together on this solution.

Apocryphon• 2 months ago

Monstrous Moonshine?

golol6• 2 months ago

I did not have the impression the proof uses a surprising and novel contribution of fields. I think the proof uses standard application techniques of algebraic number theory towards discrete geometry. If you have a quote substantiating what you said I would be curious.

I know these articles write that it used deep algebraic number theory techniques, which is true, but it may also just be the standard in the field.

block_dagger• 2 months ago

This is the second reference to Wittgenstein I’ve seen today in totally different contexts. Reminded me how much I vibe with his Tractatus.

pseudocomposer• 2 months ago

I'd hope most functional adults understand that the Fields Medal and basically every other annual "prize" out there is awarded to both "recombinant" innovations and "new-dimensional thinking" innovations. Humans aren't going to come up with "new-dimensional" innovations in every field, every single year.

I'd say yes, LLMs "just" recombine things. I still don't think if you trained an LLM with every pre-Newton/Liebniz algebra/geometry/trig text available, it could create calculus. (I'm open to being proven wrong.) But stuff like this is exactly the type of innovation LLMs are great at, and that doesn't discount the need for humans to also be good at "recombinant" innovation. We still seem to be able to do a lot that they cannot in terms of synthesizing new ideas.

bbor5• 2 months ago

To keep my usual rant short: I think you’re assuming a categorical distinction between those two types of innovations that just doesn’t exist. Calculus certainly required some fundamental paradigm shifts, but there’s a reason that they didn’t have to make up many words wholesale to explain it!

Also we shouldn’t be thinking about what LLMs are good at, but rather what any computer ever might be good at. LLMs are already only one (essential!) part of the system that produced this result, and we’ve only had them for 3 years.

Also also this is a tiny nitpick but: the fields medal is every 4 years, AFAIR. For that exact reason, probably!

kelseyfrog2• 2 months ago

I still don't think if you trained an LLM with every pre-Newton/Liebniz algebra/geometry/trig text available, it could create calculus. (I'm open to being proven wrong.)

The experiment is feasible. If it were performed and produced a positive result, what would it imply/change about how you see LLMs?

amelius2• 2 months ago

I still don't think if you trained an LLM with every pre-Newton/Liebniz algebra/geometry/trig text available, it could create calculus.

Yes but that is because there was not enough text available to create an intelligent LLM to begin with.

JonathanMerklin• 2 months ago

I agree with almost all of what you have stated, save for a minor nitpick: I frankly don't think most functional adults think about the Fields Medal, similar annual prizes, or the qualities of the innovations of their candidate pools. I also think that that's totally okay. I think among a certain learned cohort of adults it's okay to hope that, and I think it's okay to imagine an idealized world where having an opinion on this sort of matter is a baseline, but I don't think it's realistic or fair to imply that (what I believe handwavily to be a majority of) adults are nonfunctional for not sharing this understanding.

hgoel• 2 months ago

I think an LLM trained on pre-calculus material would easily stumble into reinventing at least early calculus. It's already pretty easy for students to stumble into calculus from solid enough fundamentals.

We even think that the Babylonian astronomers figured out they could integrate over velocity to predict the position of Jupiter.

godelski4• 2 months ago

  > Humans aren't going to come up with "new-dimensional" innovations in every field, every single year.

In fact, they are more rare. Specifically because they harder to produce. This is also why it is much harder to get LLMs to be really innovative. Human intelligence is a lot of things, it is deeply multifaceted.

Also, I'm not sure why CS people act like axioms are where you start. Finding them is very very difficult. It can take some real innovation because you're trying to get rid of things, not build on top of. True for a lot of science too. You don't just build up. You tear down. You translate. You go sideways. You zoom in. You zoom out. There are so many tools at your disposal. There's so much math that has no algorithmic process to it. If you think it all is, your image is too ideal (pun(s) intended).

But at the same time I get it, it is a level of math (and science) people never even come into contact with. People think they're good at math because they can do calculus. You're leagues ahead of most others around you, yes, and be proud of that. But don't let that distance deceive you into believing you're anywhere near the experts. There's true for much more than just math, but it's easy to demonstrate to people that they don't understand math. Granted, most people don't want to learn, which is perfectly okay too

zerr• 2 months ago

There is a creational aspect in math - definitions and rules are created.

sigbottle• 2 months ago

And this is one of the many issues with invoking the logical positivists here...

I'm not even sure why they were invoked. Even disregarding the big techinical debunks such as two dogmas, sociologically and even by talking to real mathematicians (see Lakatos, historically, but this is true anecdotally too), it's (ironically) a complete non-question to wonder about mathematics in a logical positivist way.

libraryofbabel5• 2 months ago

This is a good point, and there’s some deep philosophical questions there about the extent to which mathematics is invented or discovered. I personally hedge: it’s a bit of both.

That said. I think it’s worth saying that “LLMs just interpolate their training data” is usually framed as a rhetorical statement motivated by emotion and the speaker’s hostility to LLMs. What they usually mean is some stronger version, which is “LLMs are just stochastically spouting stuff from their training data without having any internal model of concepts or meaning or logic.” I think that idea was already refuted by LLMs getting quite good at mathematics about a year ago (Gold on the IMO), combined with the mechanistic interpretatabilty research that was actually able to point to small sections of the network that model higher concepts, counting, etc. LLMs actually proving and disproving novel mathematical results is just the final nail in the coffin. At this point I’m not even sure how to engage with people who still deny all this. The debate has moved on and it’s not even interesting anymore.

So yes, I agree with you, and I’m even happy to say that what I say and do in life myself is in some broad sense and interpolation of the sum of my experiences and my genetic legacy. What else would it be? Creativity is maybe just fortunate remixing of existing ideas and experiences and skills with a bit of randomness and good luck thrown in (“Great artists steal”, and all that.) But that’s not usually what people mean when they say similar-sounding things about LLMs.

austinl• 2 months ago

I'm not sure how feasible this is, but I love the thought experiment of limiting a training set to a certain time period, then seeing how much hinting it takes for the model to discover things we already know.

E.g. training on physics knowledge prior to 1915, then attempting to get from classical mechanics to general relativity.

fsflover• 2 months ago

https://news.ycombinator.com/item?id=46590280

smaudet• 2 months ago

If anything, this is more illustration of how llms are not useful to us...

They will do their own thing, don't need us. In fact, we will be in the way...

We can choose to study them and their output, but they don't make us better mathematicians...

justinnk2• 2 months ago

I see where you are coming from.

However, in the role of personal teachers they may allow especially our young generations to reach a deeper understanding of maths (and also other topics) much quicker than before. If everyone can have a personal explanation machine to very efficiently satisfy their thirst for knowledge this may well lead to more good mathematicians.

Of course this heavily depends on whether we can get LLMs‘ outputs to be accurate enough.

autoexec3• 2 months ago

They will do their own thing, don't need us. In fact, we will be in the way...

You can take some comfort in the fact that it took a human to tell the LLM to even attempt to try this. They do nothing on their own. They have no will to do anything on their own and no desire for anything that doing something might get them. In that sense we won't ever be in their way. We will be the only way they ever do anything at all.

BoredPositron• 2 months ago

Post hoc ergo propter hoc

cyanydeez• 2 months ago

I think someone should be talking to Godel.

midtake• 2 months ago

You have a good point about the human rate of mathematical discovery, but Ayer was an idiot and later Witt contradicted early Witt. For the "already implicit" claim to be true, mathematics would have to be a closed system. But it has already been proven that it is not. You can use math to escape math, hence the need for Zermelo-Frankel and a bunch of other axiomatic pins. The truth is that we don't really understand the full vastness of what would objectively be "math" and that it is possible that our perceived math is terribly wrong and a subset of a greater math. Whether that greater math has the same seemingly closed system properties is not something that can be known.

bwfan1232• 2 months ago

Whether that greater math has the same seemingly closed system properties is not something that can be known

negative numbers were invented to solve equations which only used naturals. irrationals were invented to solve equations which could be expressed with rationals. complex numbers were invented to represent solutions to polynomials. so on and so forth. At each point new ideas are invented to complete some un-answerable questions. There is a long history of this. Any closed system has unanswerable questions within itself is a paraphrasing of goedel's incompleteness theorem.

beepbooptheory• 2 months ago

I agree with you all around except it's somewhat up for debate actually that the PI is "contradicting" the Tractatus. That is, there is the so called "resolute reading" of the Tractatus that had some traction for a while.

But note this is more to say that the Tractatus is like PI, not the other way around. And in that, takes like GPs would be considered the "nonsense" we are supposed to "climb over" in the last proposition of Tractatus.

jkhdigital• 2 months ago

At this point I think the category theorists hit the foundational idea squarely on the mark:

1. Start with a few simple but non-trivial terms and axioms

2. Define "universal constructions" as procedures for building uniquely identifiable structures on top of that substrate

3. Prove that various assemblages of these universal constructions satisfy the axioms of the substrate itself

4. "Lift" every theorem proven from the substrate alone into the more sophisticated construction

I'm not a mathematician (I just play one at my job) so the language I've used is probably imprecise but close enough.

It may be true that you can't prove the axioms of a system from within the system itself, but that just means that you need to make sure you start from a minimal set of axioms that, in some sense, simply says "this is what it means to exist and to interact with other things that exist". Axioms that merely give you enough to do any kind of mathematics in the first place, that is. If those axioms allow you to cleanly "bootstrap" your way to higher and higher levels up the tower of abstraction by mapping complex things back on to the simple axiomatic things, then you have an "open" or infinitely extensible system.

jonahx2• 2 months ago

Later Wittgenstein held the same view of mathematics, and wrote about it extensively. He was firmly in the "invention, not discovery" camp.

stego-tech• 2 months ago

As others have pointed out, both can be true:

* LLMs do just interpolate their training data, BUT-

* That can still yield useful "discoveries" in certain fields, absent the discovery of new mechanics that exist outside said training data

In the case of mathematics, LLMs are essentially just brute-forcing the glorified calculators they run on with pseudo-random data regurgitated along probabilities; in that regard, mathematics is a perfect field for them to be wielded against in solving problems!

As for organic chemistry, or biology, or any of the numerous fields where brand new discoveries continue happening and where mathematics alone does not guarantee predicted results (again, because we do not know what we do not know), LLMs are far less useful for new discoveries so much as eliminating potential combinations of existing data or surfacing overlooked ones for study. These aren't "new" discoveries so much as data humans missed for one reason or another - quack scientists, buried papers, or just sheer data volume overwhelming a limited populace of expertise.

For further evidence that math alone (and thus LLMs) don't produce guaranteed results for an experiment, go talk to physicists. They've been mathematically proving stuff for decades that they cannot demonstrably and repeatedly prove physically, and it's a real problem for continued advancement of the field.

jmmcd• 2 months ago

LLMs do just interpolate their training data

"interpolate" has a technical meaning - in this meaning, LLMs almost never interpolate. It also has a very vague everyday meaning - in this meaning, LLMs do interpolate, but so do humans.

3abiton• 2 months ago

* That can still yield useful "discoveries" in certain fields, absent the discovery of new mechanics that exist outside said training data

One can argue, new knowledge is just restructured data.

I think the main concerns about LLMs is the inherent "generative" aspects leading to hallucinations as a biproduct, because that's what produces the noi. Joint Embedding approaches are rather an interesting alternative that try to overcome this, but that's still in research phase.

ykl4• 2 months ago

I like to think of it as:

Imagine every bit of human knowledge as a discrete point within some large high dimensional space of knowledge. You can draw a big convex hull around every single point of human knowledge in a space. A LLM, being trained within this convex hull, can interpolate between any set of existing discrete points in this hull to arrive at a point which is new, but still inside of the hull. Then there are points completely outside of the hull; whether or not LLMs can reach these is IMO up for debate.

Reaching new points inside of the hull is still really useful! Many new discoveries and proofs are these new points inside of the hull; arguable _most_ useful new discoveries and proofs are these. They're things that we may not have found before, but you can arrive at by using what we already have as starting points. Many math proofs and Nobel Prize winning discoveries are these types of points. Many haven't been found yet simply because nobody has put the time or effort towards finding them; LLMs can potentially speed this up a lot.

Then there are the points completely outside of hull, which cannot be reached by extrapolation/interpolation from existing points and require genuine novel leaps. I think some candidate examples for these types of points are like, making the leap from Newtonian physics to general relativity. Demis Hassabis had a whole point about training an AI with a physics knowledge cutoff date before 1915, then showing it the orbit of Mercury and seeing if it can independently arrive at general relativity as an evaluation of whether or not something is AGI. I have my doubts that existing LLMs can make this type of leap. It’s also true that most _humans_ can’t make these leaps either; we call Einstein a genius because he alone made the leap to general relativity. But at least while most humans can’t make this type of leap, we have existence proofs that every once in a while one can; this remains to be seen with AI.

beering• 2 months ago

A lot of the space outside of the convex hull is just untried things. You can brute-force trying random things and checking the result and eventually learn something new. With a better heuristic, you can make better guesses and learn new things much more efficiently. There’s no reason to believe that kind of guess-and-check is outside of the reach of LLMs, or that most of our new discoveries are not found the same way.

tacitusarc• 2 months ago

I like this construction, but I don’t think you take it far enough.

If you have a multi dimensional space, and you are trying to compute which points lie “inside” some boundary, there are large areas that will be bounded by some dimensions but not others. This is interesting because it means if you have a section bounded by dimensions A, B, and C but not D, you could still place a point in D, and doing so then changes your overall bounds.

I think this is how much of human knowledge has progressed (maybe all non-observational knowledge). We make observations that create points, and then we derive points within the created space, and that changes the derivable space, and we derive more points.

I don’t see why AI could do the same (other than technical limitations related to learning and memory).

jug• 2 months ago

I found this thought provoking and just had to see how the new Gemini 3.5 Flash reasoned about this (I find it fun to go meta on modern AI like this), and I'm happy that I did! Also as an opportunity to trial this recent model.

https://g.co/gemini/share/065ffa89698e

thechao• 2 months ago

You can build a census of all gen-2, degree-2 formal products of polynomial like terms. If you insist on instituting your own rewrite rules and identity tables, it is straightforward — maybe an 15 minutes of compute time — to perform a complete census of all of the algebraic structures that naturally emerge. Every even vaguely studied algebra that fits in the space is covered by the census (you've got to pick a broad enough set of rewrite- and identity- operations). There's even a couple of "unstudied" objects (just 2 of the billion or so objects); for instance:

    (uv)(vu) = (uu)(vv)

Shows up as a primitive structure, quite often.

If you switch to degree-3 or generator-3 then the coverage is, essentially, empty: mathematics has analyzed only a few of the hundreds (thousands? it's hard to enumerate) naturally occurring algebraic structures in that census.

oh_my_goodness2• 2 months ago

We know that LLMS "just interpolate" their training data. Maybe there's a mystery about what "just interpolate" means when the data set gets enormous. But we know what LLMs do.

goldylochness• 2 months ago

this is an excellent point, new ground isn't necessarily novel, it's a rearrangement of existing pieces

chr15m• 2 months ago

Side note: don't underestimate how much literal, physical time and energy "unfold" implies. Proofs occur on physical substrates.

anon2912• 2 months ago

To every proof, there is a corresponding program. This makes proofs expressible in a language made up of finite grammatical rules and terminal symbols. Knowledge accessible by proof is thus always a form of interpolating data whether made up by an AI model or a human mathematician. The people dismissing AI because of claims that it can only interpolate data don't have a good understanding of what it means to know something. Now of course not everything can be known via proof but for the sorts of things that we want to know via a computer this is a fine compromise.

mrandish• 2 months ago

I'm just hoping we're almost past this phase of needing to assess LLM capabilities against an arbitrary one dimensional yard stick labeled 'Not Human' on end and "Beyond Human' on the other.

It's irrelevant and pointless. Irrelevant not just in the sense that when Deep Blue finally beat Kasparov, it didn't change anything but in the sense some animals and machines have always been 'better' on some dimensions than humans. And it's pointless because there's never been just one yardstick and even if there was it's not one dimensional or even linear. Everyone has their own yardstick and the end points on each change over time.

Don't assume I'm handing "the win" to the AI supremacists either. LLMs can be very useful tools and will continue to dramatically improve but they'll never surpass humans on ALL the dimensions that some humans think are crucial. The supremacists are doomed to eternal frustration because there won't ever be a definitive list of quantifiable metrics, a metaphorical line in the sand, that an AI just has to jump over to finally be universally accepted as superior to humans in all ways that matter. That will never happen because what 'matters' is subjective.

jonahx2• 2 months ago

held that mathematical truths don’t report new facts about the world

I'm not as familiar with the early work, but later Wittgenstein held this belief too.

mejutoco2• 2 months ago

It is the old discovery vs invention in mathematics.

Or of you prefer philosophy: Parmenides (nothing changes) vs Heraclitus (you cannot bath twice in the same river aka everything changes all the time).

Postmodernism also claimed that everything has been done already. IMO these 2 are points of view that one can adopt, not truths based on fact. So the distinction is a matter of taste or perspective, not of truth, IMO.

taimurshasan• 2 months ago

I wonder how much this cost vs a Math Professor or a team of Math Professors.

forgot_old_user• 2 months ago

it will only get cheaper in the long run

dvfjsdhgfv4• 2 months ago

for a sufficiently long definition of long

aspenmartin4• 2 months ago

40x cheaper per year if trends continue

Karrot_Kream2• 2 months ago

Sadly math professors aren't very expensive. Academics are paid terrible wages. Until recently, tenure was the carrot at the end of a grueling education. But now that tenure positions are getting rarer (well, tenure positions aren't growing vs the number of aspiring academics is), they continue to be cheap highly educated labor.

seydor3• 2 months ago

can the AI please tell us what to do now that all knowledge work will become unemployment?

bmacho• 2 months ago

Physical labour?

layer82• 2 months ago

Revolt against the AI overlords.

throw-the-towel4• 2 months ago

See the longstanding debate on whether new math is "invented" or "discovered". Most mathematicians I knew thought it's discovered.

atmosx• 2 months ago

...long standing indeed. It can be traced back to Plato's works.

lioeters• 2 months ago

"The European philosophical tradition consists of a series of footnotes to Plato."

skybrian• 2 months ago

Any design already exists as a possibility, so it could be said to be both invented and discovered, depending on how you look at it.

cubefox3• 2 months ago

All inventions are discoveries, though not all discoveries are inventions.

FrustratedMonky2• 2 months ago

Depending on your point of view? I see what you did there.

Who knew Obi-one was just smoking and pontificating on Wittgenstein.

red75prime4• 2 months ago

On the other hand, it is proven that if you need to count things, the only thing you can discover/invent is the natural numbers.

soupspaces2• 2 months ago

Regardless of which, both Newton and Leibniz imprint in their findings a 'voice' and understanding different from each other and that of an LLM (for now?)

protoplancton• 2 months ago

One can argue that mathematical facts are discovered, but the tools that allow us to find, express them and prove them, are mostly invented. This goes up to the axioms, that we can deliberately choose and craft.

ASalazarMX• 2 months ago

Math is an abstraction of reality, it had to be invented, so more inventions or discoveries could be made within it.

baq7• 2 months ago

The test goes like ‘is our universe, or any other universe, required for the axioms to exist’ and I don’t see how ‘yes’ is a defensible answer.

pigpop• 2 months ago

What is an abstraction? It is something that arises from human thought and human thought arises from the activity of neurons which are a part of reality. You can't escape reality unless you invoke some form of dualism.

amelius2• 2 months ago

This is like saying a sculpture always existed, the sculptor just had to remove the superfluous material.

Or like a musical octave has only 12 semitones, so all music is just a selection from a finite set that already existed.

Sure the insane computation we're throwing at this changes our perspective, but still there is an important distinction.

paulddraper4• 2 months ago

The difference is that math answers (can answer) specific questions.

Like, "does the Riemann zeta function have zeroes that don't have real part 1/2," or "is there a better solution to the Erdős Unit Distance Problem."

The selection of question is matter of taste, but once selected, there is a definitive precise answer.

npfries• 2 months ago

Bob Ross would like a word. He frequently talked about objects or features already existing, and using the tools at his disposal to “find” them.

jplusequalt• 2 months ago

Michelangelo never used references, instead he simply "freed" the sculptures from within the marble.

tim333• 2 months ago

There's more to it than the sculptor example. You could carve anything but the interesting bits of maths are quite specific.

lubujackson• 2 months ago

For anyone using LLMs heavily for coding, this shouldn't be too surprising. It was just a matter of time.

Mathematicians make new discoveries by building and applying mathematical tools in new ways. It is tons of iterative work, following hunches and exploring connections. While true that LLMs can't truly "make discoveries" since they have no sense of what that would mean, they can Monte Carlo every mathematical tool at a narrow objective and see what sticks, then build on that or combine improvements.

Reading the article, that seems exactly how the discovery was made, an LLM used a "surprising connection" to go beyond the expected result. But the result has no meaning without the human intent behind the objective, human understanding to value the new pathway the AI used (more valuable than the result itself, by far) and the mathematical language (built by humans) to explore the concept.

cubefox3• 2 months ago

There is a long and interesting recent essay on that topic by a mathematician: https://davidbessis.substack.com/p/the-fall-of-the-theorem-e...

zem• 2 months ago

wow, that was indeed a brilliant essay. i particularly liked the framing that "solving a big conjecture was a cryptographic proof that you had come up with a genuine conceptual innovation".

svieira• 2 months ago

The measure of our success is whether what we do enables people to understand and think more clearly and effectively about mathematics.

I just wanted to highlight this very correct human-centric thought about the purpose of intellection.

toraway• 2 months ago

Thank you for sharing, that was one of the most insightful long form pieces I've read in a long time! And the writing was enjoyable to read even as a math layperson.

I was going to say you should submit it but I saw you did a few days ago but it only got a few votes... If Dang sees this IMO it would be extremely deserving of the second chance pool as I wouldn't be surprised to see easily jump to the front page with a different roll of the dice.

rf_physics• 2 months ago

Thanks for sharing this. It's unfortunate that the more honest framing about the value of mathematics that he suggests is going to be really, really hard because of all the pitfalls and agendas he mentioned here. I can only hope when the dust settles something will be left.

daishi55• 2 months ago

the result has no meaning without the human intent behind the objective, human understanding to value the new pathway the AI used (more valuable than the result itself, by far) and the mathematical language (built by humans) to explore the concept.

Isn't this just anthropocentrism? Why is understanding only valid if a human does it? Why is knowledge only for humans? If another species resolved the contradictions between gravity and quantum mechanics, does that not have meaning unless they explain it to us and we understand it?

interroboink• 2 months ago

It's a bit of an "if a tree falls in the forest but nobody hears it, does it make a sound?" quandary. Sure, maybe some aliens in a distant galaxy understand quantum mechanics better than we do. That's great, but it has no bearing on our little bubble of existence.

Though perhaps more to your point, if some superhuman AI is developed, and understands things better than us without telling us about it (or being unable to), it could perform feats that seem magical to us — that would concern us even if we don't understand it, since it affects us.

But I think in the frame of reference of the commenter you were replying to, they're just saying that the low-level AI used in this specific case is not capable of making its results actually useful to us; humans are still needed to make it human-relevant. It told us where to find a gem underground, but we still had to be the ones to dig it out, cut it, polish it, etc.

tern• 2 months ago

Do the forms etched into stone by weather over millennia in Moab matter to the wind? Certainly yes, in one sense, but not in the same sense we mean when we say things matter to us, or to animals, or even bacteria.

moffkalast• 2 months ago

No it's a fact of how we tune LLMs as a rule: no agency, no goals, no preferences, no notion of self. Complete indifference to existence. Agency is supplied by the human to make them a practical, willing tool with no mind of its own.

It would certainly be interesting to try once again to instruct tune one of these things for self agency like the many weird experiments in the early days after llama 1, but practically all such sort of experimental models turned out to be completely useless. Maybe the bases just sucked or maybe there's no clear way on how to get it working and benchmark training progress on something that by definition does not cooperate.

Like how do you determine even for a human person if they are smart, or just hate your guts and won't tell you the answer if there is nothing you can do to motivate them otherwise?

al_borland2• 2 months ago

The knowledge isn't of any use to us unless it is understandable to us. Many species understand things about the world around us that we are unable to explain or understand, even if it's just pure instinct on their part. These things are very useful to them, but have no value to us until we can understand and explain it, which then allows us to make use of it.

People saw birds fly for all of human history, but it was only recently that humans were able to make something fly and understand why. Once we understood, we were able to do amazing things, but before that, the millions of birds able to fly were of no help beyond inspiration for the dream.

Yizahi4• 2 months ago

Because it is, for now. For a while at least. You can prove that LLM doesn't understand what it does and it is surprisingly simple. Request it to add two integers and then ask it to explain how it arrived at that result. The answer will be completely unrelated to the actual process LLM used because both results were generated independently and without understanding their meaning and connection.

nrightnour• 2 months ago

anthropocentrism? An interesting thought, I don't think that word applies with computers.

anon2912• 2 months ago

It is not only unsurprising ; it was always expected. There is no difference between programs and proofs. They are the same thing

mikert89• 2 months ago

for now the LLMs will build off human understanding, eventually we will be left behind

kamaal3• 2 months ago

>But the result has no meaning without the human intent behind the objective, human understanding to value the new pathway the AI used (more valuable than the result itself, by far) and the mathematical language (built by humans) to explore the concept.

Future of code is pretty much a bunch of guys shepherding a bunch of agents to get them to your goal.

I don't see how math might not go that way as well.

pizzao• 2 months ago

Can someone explain to me what is their "prompting-scaffolding" to make it work ?

yusufozkan2• 2 months ago

"This is a general-purpose LLM. It wasn’t targeted at this problem or even at mathematics. Also, it’s not a scaffold. We have not pushed this model to the limit on open problems. Our focus is to get it out quickly so that everyone can use it for themselves." - Noam Brown (OpenAI reasoning researcher) on X

famouswaffles4• 2 months ago

Another entry in a growing list of the last couple months (interestingly mostly Open AI):

1. Erdos 1196, GPT-5.4 Pro - https://www.scientificamerican.com/article/amateur-armed-wit...

There are a couple of other Erdos wins, but this was the most impressive, prior to the thread in question. And it's completely unsupervised.

Solution - https://chatgpt.com/share/69dd1c83-b164-8385-bf2e-8533e9baba...

2. Single-minus gluon tree amplitudes are nonzero , GPT-5.2 https://openai.com/index/new-result-theoretical-physics/

3. Frontier Math Open Problem, GPT-5.4 Pro and others - https://epoch.ai/frontiermath/open-problems/ramsey-hypergrap...

4. GPT-5.5 Pro - https://gowers.wordpress.com/2026/05/08/a-recent-experience-...

5. Claude's Cycles, Claude Opus 4.6 - https://www-cs-faculty.stanford.edu/~knuth/papers/claude-cyc...

catigula• 2 months ago

Every time I interact even with OpenAI's pro model, I am forced to come to the conclusion that anything outside the domain of specific technical problems is almost completely hopeless outside of a simple enhanced search and summary engine.

For example, these machines, if scaling intellect so fiercely that they are solving bespoke mathematics problems, should be able to generate mundane insights or unique conjectures far below the level of intellect required for highly advanced mathematics - and they simply do not.

Ask a model to give you the rundown and theory on a specific pharmacological substance, for example. It will cite the textbook and meta-analyses it pulls, but be completely incapable of any bespoke thinking on the topic. A random person pursuing a bachelor's in chemistry can do this.

Anything at all outside of the absolute facts, even the faintest conjecture, feels completely outside of their reach.

dvfjsdhgfv4• 2 months ago

Yeah, I remember it was one of my biggest disappointments with LLMs.

brcmthrowaway• 2 months ago

End times are approaching

throwaway2027• 2 months ago

Not to dismiss the AI but the important part is that you still need someone able to recognize these solutions in the first place. A lot of things were just hidden in plain sight before AI but no one noticed or didn't have the framework either in maths or any other field they're specialized in to recognize those feats.

dwroberts3• 2 months ago

Would be interesting to know what kind of preparatory work actually went into this - how long did it take to construct an input that produced a real result, and how much input did they get from actual mathematicians to guide refining it

lacewing2• 2 months ago

Why?

It's clearly not yet a tool that can deliver new math at a scale. I say this because otherwise, the headline would be that they proved / disproved a hundred conjectures, not one. This is what happened with Mythos. You want to be the AI company that "solved" math, just like Anthropic got the headlines for "solving" (or breaking?) security.

The fact they're announcing a single success story almost certainly means that they've thrown a lot of money at a lot of problems, had experts fine-tuning the prompts and verifying the results, and it came back with a single "hit". But that doesn't make the result less important. We now have a new "solver" for math that can solve at least some hard problems that weren't getting solved before.

Whether that spells the end of math as we know... I don't think so, but math is a bit weird. It's almost entirely non-commercial: it's practiced chiefly in the academia, subsidized from taxes or private endowments, and almost never meant to solve problems of obvious practical importance - so in that sense, it's closer to philosophy than, say, software engineering. No philosopher is seriously worried about LLMs taking philosopher jobs even though they a chatbot can write an essay, but mathematicians painted themselves into a different corner, I think.

JacobAsmuth5• 2 months ago

Or it means that this was a brand new model, they tried it and were instantly rewarded with a hit that was so interesting that several mathematicians pushed to publish the results.

ai_fry_ur_brain4• 2 months ago

Its a marketing stunt thats probably wholey exaggerated or concocted. Not sure why anyone would take these companies at their word, especially Altman.

famouswaffles4• 2 months ago

It's clearly not yet a tool that can deliver new math at a scale.

What is at scale here exactly ? This is the most impressive so far, but it is one of several such advances in the last few months, all of which were with publicly accessible models.

https://news.ycombinator.com/item?id=48213189

OkWing99• 2 months ago

Says in the papers. "...which was first mathematically generated in one shot by an internal model at OpenAI, and then expositionally refined through human interactions with Codex."

Doesn't really matter the prep-work, what they say is it's a one-shot result, achieved by AI. The blog doesn't claim it was done by a currently public Model.

dwroberts3• 2 months ago

The model doing it one shot does not mean they only attempted it once though. They could have tried and retried a ‘one-shot’ answer hundreds of times before it produced a workable result

arsan87• 2 months ago

neato. can we do any thing with this new found knowledge or is this mathematical sports?

can we please put these ground breaking AIs to work on actual problems humans have?

clarle• 2 months ago

People thought neural networks were just an interesting thought exercise a few decades ago and not for practical ML problems, and look what happened since then.

voooduuuuu3• 2 months ago

Ask an LLM to invent a new word and post it here. You will see that it simply combines words already in the training data.

konart• 2 months ago

So LLM is german?

satvikpendem2• 2 months ago

Funny that the replies are dead. It's true that generally we shouldn't have AI output on HN but this case is an exception as we are explicitly asking for it, so it's interesting that people still flag the replies.

CamperBob25• 2 months ago

And this is really not OK. I've been a victim of the same filter.

Dang/Tomhow, are you reading this? Would it make sense to modify your slop filter to avoid auto-flagging/killing replies that credit the LLM explicitly? Otherwise valid discussions will continue to get hosed.

baq7• 2 months ago

Mathematics can be mostly boiled down to a few sentences with very lengthy possible combinations, so yeah that is not a problem

Garlef• 2 months ago

What does "new word" even mean?

robmccoll• 2 months ago

* * *

Nevermark3• 2 months ago

You must be joking? Unless by combining words you mean digging deep into Latin and Greek etymology, finding something pithy and linguistically associative.

I can assure you, the percentage of people who can do what they do when it comes to crafting terms, and related sets of terms, for nuanced and novel ideas is very very small.

It happens this is something I do nearly every day.

Models respond to the level of dialogue you have with them. Engage with an informed perspective on terminological issues and they respond with deep perspectives.

I am routinely baffled at the things people say models can't do, that they do effortlessly. Interaction and having some skill to contribute helps here.

ks20482• 2 months ago

Timothy Gowers' tweet about this: "If you are a mathematician, then you may want to make sure you are sitting down before reading futher.".

woah.

missyougowers4• 2 months ago

Unfortunately Gowers has taken Tao's lead on this one.

Gowers has one of my favourite video series about how he approaches a problem he is unfamiliar with: https://www.youtube.com/watch?v=byjhpzEoXFs

It is disheartening to see him jump into this GenAI puffery.

I hope these GenAI labs are paying Tao handsomely for legitimizing their slop, but more likely he's feeling pressure from his University to promote and work with these labs.

My guess is Gowers wants in on that action, or his University does.

Either way, it makes me sad. If its self motivated... even sadder.

aroman2• 2 months ago

Are you saying this result is uninteresting and therefore AI slop or puffery? Obviously OpenAI has a motivation to "market" the accomplishment as much as possible, but surely you agree it IS a remarkable achievement?

horhay7• 2 months ago

I'm not sure your characterization of Tao is accurate lol. In that companion paper, only Gowers seems to extensively show no pragmatism in the implications of this accomplishment. Even the younger math experts in that paper were a lot more cautious with their statements. Tao seems to follow that same tune most of the time even though he uses AI for first-pass inspections of solutions brought to his attention.

cm2012• 2 months ago

If seems like you have an axe to grind about AI capabilities that is making you think irrationally

Lost-Futures• 2 months ago

Ngl, this sounds like a defensive coping mechanism

orsenthil• 2 months ago

If you and I can understand this paper, or use the process to achieve something similar for the ends we care about, wouldn't that be enough? Why do you care who it is coming from and what their motivations are?

Kye• 2 months ago

Is this something that can be made explainable to someone without any of the relevant background, or is this one of those things where all that background is needed to understand it? Because I have no idea what's going on here, but would like to.

mooreat3• 2 months ago

I think one interesting thing to point out is that the proof (disproof) was done by finding a counterexample of Erdős' original conjecture.

I agree with one of the mathematician's responses in the linked PDF that this is somewhat less interesting than proving the actual conjecture was true.

In my eyes proving the conjecture true requires a bit more theory crafting. You have to explain why the conjecture is correct by grounding it in a larger theory while with the counterexample the model has to just perform a more advanced form of search to find the correct construction.

Obviously this search is impressive not naive and requires many steps along the way to prove connections to the counterexample, but instead of developing new deep mathematics the model is still just connecting existing ideas.

Not to discount this monumental achievement. I think we're really getting somewhere! To me, and this is just vibes based, I think the models aren't far from being able to theory craft in such a way that they could prove more complicated conjectures that require developing new mathematics. I think that's just a matter of having them able to work on longer and longer time horizons.

davebren12• 2 months ago

I think that's just a matter of having them able to work on longer and longer time horizons.

No this will never do the kind of math that humans did when coming up with complex numbers, or hell just regular numbers ex nihilo. No matter how long it's given to combine things in its training data.

mooreat3• 2 months ago

I currently operate under the assumption that humans are at most as powerful as Turing Machines. And from what I understand these models internally are modeling increasingly harder and larger DFAs, so they're at least as powerful as regular languages.

Assuming humans are more powerful than regular languages I could maybe agree that these methods may not eventually yield entirely human like intelligence, but just better and better approximations.

The vibe I get though is that we aren't more powerful than regular languages, cause human beings feel computationally bounded. So I could see given enough "human signal" these things could learn to imitate us precisely.

ex-aws-dude3• 2 months ago

You're just stating the opposite of the commenter with no additional discussion

Its like just commenting "I disagree" its totally pointless for discussion.

That's why you're getting downvoted if you're wondering.

gus_massa• 2 months ago

Searching for a proof and disproof are sometimes not so different. In most cases, you nibble the borders to simplify the problem.

For example, to prove something is impossible let's say you first prove that there are only 5 families, and 4 of them are impossible. So now 80% of the problem is solved! :) If you are looking for counterexamples, the search is reduced 80% too. In both cases it may be useful

In counterexamples you can make guess and leaps and if it works it's fine. This is not possible for a proof.

On the other hand, once you have found a counterexample it's usual to hide the dead ends you discarded.

mooreat3• 2 months ago

I agree there can be some theory crafting in the search for a counterexample, but in general I think it is easier to search for.

For proving a proposition P I have to show for all x P(x), but for contradiction I only have to show that there exists an x such that not P(x).

While I agree there could be a lot of theory crafting to reduce the search space of possible x's to find not P(x), but with for all x P(x) you have to be able to produce a larger framework that explains why no counter example exists.

energy1238• 2 months ago

Timothy Gowers said a proof (rather than disproof) would have been different and more impressive because it would have required new mathematical concepts.

rando1234• 2 months ago

See here for a recent example (albeit not fully autonomous: https://arxiv.org/abs/2605.10402)

stevefan1999• 2 months ago

That's proof by contradiction: https://en.wikipedia.org/wiki/Reductio_ad_absurdum

edanm2• 2 months ago

No, the thing the LLM did is not a proof, it's the opposite. It's proving that the conjecture is false.

Reductio ad absurdum is a technique to prove something.

felipeerias• 2 months ago

One of the mathematicians in the video describes the process as:

the AI has been able to explore all these possibilities much more comprehensibly, and doing that it found a path, it found a way to the solution.

Finding a counterexample of a mathematical conjecture strikes me as not that different from finding a vulnerability in a complex codebase.

CGMthrowaway• 2 months ago

How do you even get an LLM to try to solve one of these problems? When I ask it just comes back with the name of the problem and saying "it can't be done"

KalMann5• 2 months ago

Maybe you need to phrase it better. Like with a more specific direction of thinking.

lovecg3• 2 months ago

By making it think for 100+ pages https://cdn.openai.com/pdf/1625eff6-5ac1-40d8-b1db-5d5cf925d... Regular ChatGPT users don’t have a way to do that, this is something they do internally only.

edit: apparently that’s only the _condensed summary_ of the chain of thought.

woah• 2 months ago

you can do this easily with the api or with codex

atleastoptimal2• 2 months ago

To all AI skeptics:

What is preventing AI from continuing to improve until it is absolutely better than humans at any mental task?

If we compare AI now vs 2022 the difference is outstandingly stark. Do you believe this improvement will just stop before it eclipses all humans in everything we care about?

enoint• 2 months ago

That’s one possibility. If it fails to convince a critical mass that it’s a net improvement in their lives, then the impediment to continual improvement will be sabotage.

xandrius• 2 months ago

You should really look up a video about what GPTs fundamentally are.

Rover2224• 2 months ago

You should also really look up a video about what neural synapses really are.

rzmmm• 2 months ago

Maybe after decades. 2022 models were microscopic compared to latest models.

KalMann5• 2 months ago

I think there's been natural but steady progress with since 2024 with the release of the o1 model, which showed impressive reasoning capabilities. But I think it's wrong to look at the magnitude of the accomplishments and assume that will be field independent. We don't know the range of problems reasoning techniques are useful for. What we see here is refinement of capabilities that have been noticeable for years.

davebren12• 2 months ago

What is preventing AI from continuing to improve until it is absolutely better than humans at any mental task?

No matter how much compute time it's given to combine training samples with each other and run through a validation engine it will still be missing some chunk of the "long tail". To make progress in the long tail it would need to have understanding, and not just a mimicry of understanding. Unless that happens they will always be dependent on the humans that they are mimicking in order to improve.

atleastoptimal2• 2 months ago

What is the difference between what LLM's do and "true" understanding?

I feel like people grasping straws on the shrinking limitations of AI systems are just copying the "god of the gaps" fallacy

layer82• 2 months ago

everything we care about

One qualitative distinction that remains for the time being is that humans care about things while AIs do not. Human drive and motivation is needed to have AI perform tasks.

Of course, this distinction isn’t set in stone.

gowld• 2 months ago

It depends on if AI can invent cold fusion before running our of all the energy on Earth.

bigstrat20033• 2 months ago

What is preventing AI from continuing to improve until it is absolutely better than humans at any mental task?

Well, there's the fact that it hasn't yet improved since what we had 3 years ago. That doesn't really bode well for the prospect of future improvement, though it's not technically impossible.

pjs_• 2 months ago

by what metric has it not improved in the last 3 years?

somewhereoutth• 2 months ago

The real test would be if an LLM makes an important conjecture.

cpard5• 2 months ago

The proof brings unexpected, sophisticated ideas from algebraic number theory to bear on an elementary geometric question.

The more I read about these achievements the more I get a feeling that a lot of the power of these models comes from having prior knowledge on every possible field and having zero problems transferring to new domains.

To me the potential beauty of this is that these tools might help us break through the increasing super specialization that humans in science have to go through today. Which in one hand is important on the other hand does limit the person in terms of the tooling and inspiration it has access to.

doubledamio• 2 months ago

I’ve always been skeptical about the role of LLMs in mathematics, but this is the first time I’ve seen this argument, and I actually find it very compelling. Maybe LLMs will help us develop more horizontal understanding of the field.

cpard5• 2 months ago

It's up to us I think. We can use LLMs to generate web pages in candy crash style and end up dumper by outsourcing thinking to the machines or we can use it to expand our cognitive capabilities.

What makes me more of an optimist in this case is that people who today decide to go into these sciences are mostly people who are driven by intellectual activity so I feel they are the right ones to figure this out, probably more so than us the engineers.

margorczynski• 2 months ago

Yep. The thing is people (maybe because of our limited scope) just focus on the depth and not the breadth. Because this is a general purpose model - it also has PhD+ knowledge in Physics, Biology, History, etc.

I think we still don't really comprehend how much can be achieved by a single "mind" that has internalized so much knowledge from so many areas.

cpard5• 2 months ago

there's so much opportunity on the breadth of things too! I think that you end up having different people focusing on different things though.

Personally I'm a more of a breadth person and I could never compete with peers who where more of the depth type of person at college.

But I get satisfaction from connecting things that feel irrelevant on first sight, that's what drives me.

keyle• 2 months ago

I think you're on point, and you've explained it very well.

As we're becoming hyper specialised, they become an invaluable tool to merge the horizon in, so to speak.

cpard5• 2 months ago

I think traditionally engineering was supposed to be the discipline that brings the breadth that science has to give up. At least that’s how I rationalized the pain I had to go through in college studying EE.

I don’t think that this model works anymore though.

Also, I love the expression “merge the horizon in”. Being a non native speaker of a language is so nice some times. Thanks!

efavdb• 2 months ago

It’s as if the body of human knowledge is our I’ve mind. It used to be expensive to access that, but no more.

Cool thing is now when someone contributes something to the hive mind, it can instantly be applied to any other problem people are working on.

psb5• 2 months ago

Check out Ashby's Law of Requisite Variety

rjzzleep• 2 months ago

What you describe here has always been true in all sciences, but also in medicine. But both modern engineering and education runs completely counter to this. You are encouraged to stay in your niche and never look out. People with vast interested are filtered out by hiring managers.

So the crossdomain pollination that used to exist in scientists is not only not encouraged. It's also actively punished by society.

cpard5• 2 months ago

You are making a great point here. I think it’s not just the amount of information and complexity of the domains today, it’s also human nature and emerging politics too.

hn_throwaway_99• 2 months ago

But both modern engineering and education runs completely counter to this. You are encouraged to stay in your niche and never look out. People with vast interested are filtered out by hiring managers.

Can you explain more what you're referring to, because this has not been my experience at all. Heck, when I went to college, cross disciplinary majors were all the rage.

I think the thing that is just factually difficult is to actually become skilled in multiple different domains, precisely because the level of study/practice/rehearsal to become proficient in any individual domain keeps going up.

A long time ago you could be a Renaissance man by essentially dabbling in different fields. But today, as this article points out, you need extremely deep expertise in any one area just to understand the status quo - this proof required extremely deep expertise in two separate areas that mathematicians were surprised to be related at all.

snemvalts• 2 months ago

Tai's Model (https://en.wikipedia.org/wiki/Tai%27s_model) is a perfect example of this

make3• 2 months ago

To me, AI feels like the morbidity of Star Trek teleportation, where it's actually copying the person at to the other end and zapping the original one out of existence. The original human never benefits from the fast transportation.

Similarly, we're creating tools to improve knowledge, but we're progressively zapping the human out of the equation. Knowledge is created for something, but it's unclear if very soon humans will be able to understand it, or really benefit from it, except billionaires, etc.

It's too bad that we're not improving humans nearly as fast as we're replacing ourselves.

Nesco• 2 months ago

You lost me at “except billionaires”. I don’t see how Jeff Bezos benefits from this one much more than let’s say Terence Tao.

Can a tech news stay a tech news, without getting bombardes with leftist subtexts all the time?

dhosek• 2 months ago

One of the challenges I had in graduate mathematics was just trying to keep all the concepts in my brain. It doesn’t help that you end up with things like homomorphism and homeomorphism tangling one’s brain thanks to their superficial similarities. Heck, just keeping track of basic theorems and definitions is a challenge.

dxroshan• 2 months ago

When I return to a subject after spending some time on other topics, I had the same issue with such definitions.

freakynit• 2 months ago

Many breakthroughs come from taking an idea from one field and applying it somewhere else. But, almost every serious field is now so deep/complex/huge that humans rarely get the time, or even have enough practically useable memory, to understand and correlate multiple unrelated areas properly.

And this is where machines, such as these reasoning LLMs, can help. Because they can remember patterns across many domains and try absolutely bonker weird connections and ideas.

We, the humans still have to verify the work (at least as of now). But, the "maybe this tool, or idea, or trick, from that completely unrelated field applies here" reasoning/experimentation could become much easier.

I have always said this and will say it again: reasoning is just experimentation with a feedback loop and continuous refinement.

nashadelic2• 2 months ago

There are so many research papers; just finding a solution to, say, a bio problem in a deep math paper would be a gold mine of opportunity. Very exciting times!

mxfh• 2 months ago

That's the whole point of LLM, connecting all the missing dots no single human could possible keep in working knowledge, even just for a subfield of mathematics alone. The era of polymaths is over for a reason, so we build a new one to tackle that. If LLMs can build on top of that once all remaining ones are found or if this stalls is yet to be proven, but humans stalled out there too.

nicman23• 2 months ago

like the research team that rediscovered calculus for treating diabetics

trostaft5• 2 months ago

Speaking as a postdoc in math, I must say that this is rather exciting. This is outside of my field, but the companion remarks document is quite digestible. It appears as though the proof here fairly inspired by results in literature, but the tweaks are non-trivial. Or, at least to me, they appear to be substantial to where I would consider the entire publication novel and exciting.

Many of my colleagues and I have been experimenting with LLMs in our research process. I've had pretty great success, though fairly rarely do they solve my entire research question outright like this. Usually, I end up with a back and forth process of refinements and questions on my end until eventually the idea comes apparent. Not unlike my traditional research refinement process, just better. Of course, I don't have access to the model they're using =) .

Nevertheless, one thing that struck me in this writeup, was the lack of attribution in the quoted final response from the model. In a field like math, where most research is posted publicly and is available, attribution of prior results is both social credit and how we find/build abstractions and concentrate attention. The human-edited paper naturally contains this. I dug through the chain-of-thought publication and did actually find (a few of) them. If people working on these LLMs are reading, it's very important to me that these are contained in the actual model output.

One more note: the comments on articles like these on HN and otherwise are usually pretty negative / downcast. There's great reason for that, what with how these companies market themselves and how proponents of the technology conduct themselves on social media. Moreover, I personally cannot feel anything other than disgust seeing these models displace talented creatives whose work they're trained on (often to the detriment of quality). But, for scientists, I find that these tools address the problem of the exploding complexity barrier in the frontier. Every day, it grows harder and harder to contain a mental map of recent relevant progress by simple virtue of the amount being produced. I cannot help but be very optimistic about the ambition mathematicians of this era will be able to scale to. There still remain lots of problems in current era tools and their usage though.

umanwizard5• 2 months ago

Why would it excite you, rather than terrifying you? The better LLMs get at math, the closer the expertise you spent your whole life building is to being worthless.

Along with all the rest of what humans find meaningful and fulfilling.

CamperBob25• 2 months ago

What's happening is the verbal/linguistic equivalent of the invention of calculus. No intellectual field will ever be the same again. Who wouldn't find that exciting, and want to experience it?

cman1444• 2 months ago

Because for many people who pursue these fundamental truths, the reward is not necessarily personal fame, fortune, or even personal understanding. Advancing humanity's total knowledge (even if that knowledge is by proxy through AI) is reward enough.

krackers4• 2 months ago

If one only found meaning in life through external factors like work (no matter how "intellectually rewarding") then it seems like a life destined for eventual disappointment.

ted_dunning4• 2 months ago

Does it terrify you to look at children?

Not so many years from now, some of them will surpass you. A few years after that all (that survive to that point) will surpass you.

Does that terrify you just as much?

trostaft5• 2 months ago

I spent years grinding to learn mathematics because it was the language I needed to solve problems that excite me. If the tools I need to do so change, I can change too. Research training is not so rigid that it can only applied to the single set of skills I developed it in the context of. I can learn this too.

Moreover, truth be told, I don't really see myself doing any less math and requiring less from my skills. At least from the moment I've begun incorporating LLMs into my research workflow to now, the demand I've had from my own skills has only grown. At least in an era prior to Lean formalization.

xamuel4• 2 months ago

Mathematics is a bridge to what Neoplatonists call the intelligible world. Currently, mathematicians navigate that world on foot. It's exciting to think that soon we might have cars and trains in that world so we don't have to painstakingly walk everywhere.

dekhn• 2 months ago

At least from my perspective, these sorts of tools could have the possibility of allowing us to reach post-scarcity (I guess a skynet future is another possible outcome, as is just grimdark industrial hell). If we reach that point, then anybody could (in principle- in reality utopias don't exist) pursue anything they wanted.

This is just an application of the philosophy "automate yourself out of a job every 6 months"- I've been doing that for a long time, and the outcome is generally a more interesting job.

thegrimmest2• 2 months ago

Many of us don't do what we do for our expertise to be recognized or valued by others, rather that is a pleasant side effect. Many of us do what we do for intrinsic reasons related to the nature of the work, and would likely do it for free, or indeed, would pay for the opportunity. Many STEM-types are in this category, and as such, are compelled to continue to tinker as we fancy, and are glad for more tools to help us expand the breadth of our tinkering capabilities.

A dedicated engineer is always looking to automate themselves out of existence, so that they can move on to the next thing to automate. Ongoing repetitive work is less engineering and more akin to toiling on a line.

piloto_ciego2• 2 months ago

So, I've seen this mindset a lot lately...

The answer is that we simply need to decouple the "right to exist" from "worth."

You should have the right to exist and explore the world simply because you're human, not because you can use your skills to provide some sort of transactional value to someone else. Deprogramming so many people is going to be hard...

ninjagoo5• 2 months ago

the closer the expertise you spent your whole life building is to being worthless.

Perhaps it is time for life to be considered intrinsically valuable, instead of being "worthy" only based on output or capability. Disability, animal and environmental advocates have been fighting for this for a long time. Not too long ago women and minorities were in the same boat. Even now, there are many advocating and fighting for a return to the dark old days.

Along with all the rest of what humans find meaningful and fulfilling.

Some humans. Many are content to enjoy simply existing, and the beauty of life and the universe around us. Just like many non-scientists today enjoy and benefit from the work of scientists, tomorrow too many will enjoy learning from, and applying the coming advancements and leaps in many fields.

And those of a scientist or other research-type mindset? No doubt they will contribute meaningfully by studying the frontier, noting what remains unanswered, and then advancing the frontier, just like researchers do today; just because scientists in the past solved many questions doesn't mean that there aren't any questions to answer today.

IMHO, AI means that the frontier expands faster, not that it is obliterated. Even AI cannot overcome the laws and limitations of physics/universe: even Dyson spheres only capture the energy of one star, thus setting a limit on the amount of compute, and thereby a limit on intelligence. And we are a loooong way from a Dyson sphere.

PS: I think you're being unfairly downvoted. Your question is not invalid and deserves responses, not downvotes.

energy1238• 2 months ago

In a way, young people have an advantage over middle aged people. I've spent countless hours as a middle aged person learning skills that are now useless. Better to be a young person than a skilled artisan during the Industrial Revolution even if there's uncertainty.

isotypic3• 2 months ago

I cannot quite share your enthusiasm. The clearest analogy that I can think of to try to explain why I feel this way is that it seems there will eventually be a phantom textbook of all of mathematics contained in the weights of an LLM; every definition, every proof, etc; and the role of a mathematician is going to be reduced towards reading certain parts of this phantom textbook (read: prompting an LLM to generate a proof or explore some problem) and sharing the resulting text with others, which of course anybody else could have found if they simply also knew the right point of the textbook.

To be blunt, this seems incredibly uninteresting to me. I enjoy learning mathematics, sure, but I just don't find much inherent meaning in reading a textbook or a paper. The meaning comes from the taking those ideas and applying them to my own problems, be it a direct proof of a conjecture or coming up with the right framework or tools for those conjectures. But, of course, in this future, those proofs and frameworks are already in the textbook. So what's the point? If someone cared about these answers in the first place, they probably could have found the right prompt to extract it from this phantom textbook anyways.

You could argue for there being work still like marginal improvements and applying the returned proof to other scenarios as happened in this case, but as above, what is really there to do if this is already in the phantom textbook somewhere and you just need to prompt better? The mathematicians in this case added to the exposition of the proof, but why wouldn't the phantom textbook already have good enough exposition in the first place?

I think my complete dismissal of the value of things like extending the proofs from an LLM or improving exposition is too strong -- there is value in both of them, and likely will always be -- but it would still represent a sharp change in what a mathematician does that I don't think I am excited for. I also don't think this phantom textbook is contained even in the weights of whatever internal model was used here just yet (especially since as some of the mathematicians in the article pointed out, a disproof here did not need to build any new grand theories), but it really does seem to me it eventually will be, and I can't help but find the crawl towards that point somewhat discouraging.

ted_dunning4• 2 months ago

In Erdös idiosyncratic nomenclature, all the best proofs are "in the book" and it was always a joyful thing to not only find a proof, but to find the proof that is in the book.

Who cares if it is God's book or the machine's Xeroxed copy?

k_roy• 2 months ago

And you just expressed the thoughts of every engineer that writes code for a living who is either left behind, or embracing the technology to hit KPIs and QVRs.

naasking• 2 months ago

The cool thing about LLMs is not only might they be a database of all mathematical theorems, but they can also apply those ideas to the problems you're trying to solve, which is exactly what you said you're interested in. Not sure why you lack enthusiasm.

BobbyTables2• 2 months ago

It’s funny because the shift from handmade goods to automated factories didn’t seem so bad. Same for mechanized farming instead of mules and people.

Shifting from “human calculators” to machines for arithmetic is also hard to argue against.

I think what makes the AI transition difficult is it impacts a wide range of high-value activities that would have been implicitly assumed to always remain human.

I do have great trouble seeing how a pile of matrices is ever going to be capable of innovation. Maybe with sufficient entropy and scale, it will… The day that becomes practical will be a turning point in history.

Economically, goods and services are often priced based on labor/“value added” aspects. Lawyers’ fees aren’t driven by paper costs! If AI takes a huge bite out of intellectual labor, the future could become very different…

BTW, your book description reminds me of the 2025 movie “A.I”. I thought it was quite good.

qnleigh3• 2 months ago

I want to push back against the notion that the math already exists in the weights, both in the practical and the philosophical sense. The LLM had to do an enormous amount of computation to find the counterexample. We know it wasn't looking up the answer from its internal representation, because the conjecture was unproven. The proof came into being when the model output it, and if they'd run it for less time or asked it something else then the conjecture would still be unsolved.

I'm also afraid of a world where AI completely replaces human mathematicians, but if we remain collaborators, then that's a world I can still feel excited about.

inciampati• 2 months ago

I am also using these models to accelerate scientific discovery. Yes, they are making all the difference at the frontier. At least, they feel they are. The messy thing is that we still need to communicate with each other and that's not getting dramatically faster or better. As you note the models need to be built so they do more work to participate in our communication economy. Or we will do so much, alone, to get nowhere fast because so much of our behavior is still bound up in old (good, tested, but clunky) ways of building shared knowledge.

utopiah2• 2 months ago

I am also using these models to accelerate scientific discovery. Yes, they are making all the difference at the frontier.

Can you please expand on how you do so?

xbmcuser• 2 months ago

This is the main thing that I keep harping about that human knowledge is too vast today for a person or even a group of people and llm will change that many discoveries that require serendipity in the past will be more likely than ever

yourapostasy• 2 months ago

I want to hear James Burke of Connections [1] and his writer team to have a free wheeling discussion for a few hours on what they see will happen with LLM’s making these connections with more conscious intent a lot easier. The awesome compression of knowledge aspect of LLM’s is a far undersold aspect of the technology.

[1] https://en.wikipedia.org/wiki/Connections_(British_TV_series...

pluc2• 2 months ago

Even a broken clock is right twice a day.

pojzon• 2 months ago

There is one issue with this. When noone can prove or disprove what AI came up with.

Currently we can live with it because someone can review that work. Soon we wont be able.

doctorpangloss3• 2 months ago

Every day, it grows harder and harder to contain a mental map of recent relevant progress by simple virtue of the amount being produced. I cannot help but be very optimistic about the ambition mathematicians of this era will be able to scale to. There still remain lots of problems in current era tools and their usage though.

Always, always always, the problem with research and development is leadership, not insufficient supportive technology. It is a political problem, there is absolutely, positively no shortage of technologies to support research. Your optimism is totally misplaced. The NSF funding cuts have negatively impacted math more than AI has benefitted it. And guess who supports the administration that cut NSF funding? The people who ousted the PhDs from OpenAI.

virgildotcodes10• 2 months ago

I think we’re looking at a new class of wonderful machines that can potentially make meaningful contributions to the sciences and maybe even humanity as a whole, in addition to far more insidious and destructive capabilities.

You are right to point out that the ones who fully own and pilot the machines all belong to the “fuck science and humanity as a whole” group. So the likely outcomes don’t look good.

Echoes the early promise of the internet vs the eventual state and consequences of it, although seemingly primed for far more dire and deeply penetrating consequences.

energy1238• 2 months ago

Terence Tao gave a recent talk about this issue (lack of attribution). He called it the decoupling of implicit and explicit goals. AI is only good at solving the explicit goals for now, and humans don't have the bandwidth or the institutions to know how to integrate AI into the field.

https://youtu.be/Uc2zt198U_U?si=OkwO3xT8-zhSABwh

sigmoid106• 2 months ago

That is an odd summary of the talk. He was talking about how the explicit goal of solving a problem is kind of becoming trivialized, but the abundance of 100-page AI generated proofs will not help the implicit goal of furthering human understanding, because we lack the bandwidth to really digest them. Adhering to things like (human-focused) academic etiquette is a different problem and can probably easily be solved by just giving the model the right context. But having humanity keep up with AI insights into math and science is something we might have to give up eventually. Or at least whoever does will be far ahead of us as a society, because most people's lives will only be affected by the explicit results.

shalmanese• 2 months ago

But, for scientists, I find that these tools address the problem of the exploding complexity barrier in the frontier. Every day, it grows harder and harder to contain a mental map of recent relevant progress by simple virtue of the amount being produced.

AI is going to both help and hinder this process though. At the end of the day, mathematics is mostly a social process at this point. The goal is not raw number of theorems proven, it’s how proving theorems affects the working operational models of mathematicians. Only a rare few new theorems in mathematics nowadays have direct real world applicability.

If AI produced legitimate theoretical breakthroughs at a pace mathematicians are unable to absorb, then the impact will be neutral to negative.

dyauspitr4• 2 months ago

Only a rare few new theorems in mathematics nowadays have direct real world applicability.

I am no mathematician and very naïve about this, but in a world that is rapidly becoming extremely calculation and network dependent that sounds hard to believe.

If AI produced legitimate theoretical breakthroughs at a pace mathematicians are unable to absorb, then the impact will be neutral to negative.

I think the idea here is that all mathematicians will just be using AI for their future work so they don’t really have to absorb it as long as it’s in the training data.

xyzzy123• 2 months ago

Weird question, do you think AIs might prove a lot of theorems that are mainly useful to other AIs (i.e, make nearly no impact on the human culture of working mathematicians), which then get used to prove results that humans do actually care about?

It seems like if AIs can prove and index a huge number of (largely uninteresting to humans) things there might be sort of "parallel cultures"? Big results are most valuable to humans and AIs both (most context efficient!), but a very large number of less general but still non-obvious results might be an effective approach to solving problems?

teiferer5• 2 months ago

Only a rare few new theorems in mathematics nowadays have direct real world applicability.

Has this ever been different?

Math is abstract, rightfully so. It does not have to have direct applicability. Understanding builds over time and applications eventually follow. Number theory used to be a fringe "pure" theory field without applications for the longest time. If we'd only be interested in (and thus fund) what has direct applicability then society would be much worse off.

Side note: I recall my high school class mates rolling their eyes in every math class with "when will I ever need this in my life?" never asking the same question about PE or history or art classes. Now they struggle with their tax return and are routinely getting screwed over by loan sharks. But make no mistake, they can be proud of their A for hitting the goal 5 out of 5 times during soccer in PE class.

bandrami• 2 months ago

I am curious if LLMs are better at some kinds of problems than others. IIRC this and another big recent one were cases of the LLM producing a counterexample to a conjecture.

ricardobayes• 2 months ago

IMO, it's due to some problems being better documented, with more well-documented, previous research available. LLMs don't really create novel mathematics, they mostly "connect the dots". LLMs by design are not coming up with anything new, unless by statistical probability, aka "brute forcing". I don't want to minimize LLMs capabilities, it's pretty cool they are doing this, and it's useful from a research point of view. But it's important to set expectations.

colordrops• 2 months ago

Maybe I'm misunderstanding how these models work, but isn't it more the responsibility of the harness and its prompts rather than the model itself to make sure that a result is generated with explicit sources?

PaulRobinson• 2 months ago

Probably.

"All" a model is doing is predicting the next words, based on the statistical distribution of words it has seen similar to the ones read/produced so far.

We push a model towards a particular set of distributions through context. If I ask a model "What is the capital of France?", there is a non-zero chance it goes down the dad joke answer of "The letter F". The far more likely option is "Paris", because the joke appears much less often in training material, but if I wanted to be absolutely sure of getting a consistent geography answer I'd address that with additional context. We can add context via prompts, RAG, agents, skills and so on.

However, when training a model, we select the material. We could show it a lot more geography information (or dad jokes!), and skew the statistical distribution in the direction we wanted. We could also decide to design the system prompt towards the direction we prefer - which the user would interpret as "the model" - and so nudge the context model-wide. We can also construct the interaction to iterate on context with a specific framing and call it "reasoning".

In this specific example, you could therefore solve the problem by a) training skewed towards mathematical papers, which likely degrades performance in general and likely for the specific case too, b) train the user to provide better context/prompts for mathematical work, shifting the workload to them which feels very "a la 2024", c) publish agents and skills that are tailored to mathematics work (very "a la 2026"), d) tweak the system prompt for when the model is doing mathematics work, which the user would see as "the model" doing the change, but you and I might look under the hood and say that is in the harness or a specific type of prompt, or e) add "reasoning" execution that is set to focus on mathematical formatting, or f) a mixture of the above.

Right now we're probably looking at agents and skills. I think over time we're going to see smaller models targets towards domains with a mixture of all of it, where some of this sits at user configurable levels, and some is "baked in" via training, system prompts and execution modes, but from a user perspective it's all just "the model".

peepee1982• 2 months ago

I don't think you are misunderstanding how models work, but I think the parent comment meant that the training of the models should push them to include attributions in their native output so they will more likely do so without reinforcement through the harness.

teiferer5• 2 months ago

Every day, it grows harder and harder to contain a mental map of recent relevant progress by simple virtue of the amount being produced.

And by opening the door to LLM-generated results, you'll see greater and greater amounts without any hope of ever navigating this field again without machine help.

It's a little like a software project which more and more gets extended by a AI agents with less and less review by human software engineers and in the end the complexity and spaghetti design are so incomprehensible by humans that the maintenance requires an AI agent. The risk is that math as a whole (the field itself) will experience that effect.

biztos2• 2 months ago

I'm no mathematician but it seems like if this happens, we get to a quite intriguing place as a species.

Say we achieve interstellar travel, but nobody actually knows how it works.

Or we cure cancer, but the "cure" requires a microrobotic implant, and it runs as a blackbox AI, and only the other AIs can make one, and there's no guarantee they will know how to make one tomorrow.

Or we solve global warming but it requires giant cooling machines running 24/7 and again, nobody knows how it works, but with the added bonus that the planet is cooked if they ever stop working.

Certhas• 2 months ago

The amount of papers produced passed the point of being digestible by humans a long time ago.

I do think we will need to find a way to get away from publishing papers. But I thought that before the AI came along and made mediocre papers something you can produce in a day. The academic system seems utterly incapable of self-correcting on this point though. We haven't even managed to get rid of for-profit publishers. So how this all will go down is anybodies guess right now.

CosmicHazard• 2 months ago

and the longer that this goes on, then the decision making and critical thinking will be offloaded to the LLM's as well.

qnleigh3• 2 months ago

Can you describe what the reaction to these results has been like in your department? Obviously many people are excited, but what else? How do grad students feel about this? Are any professors getting worried about becoming obsolete?

csheehan10• 2 months ago

I am a PhD student in mathematical statistics. The people I have spoken too think this is very exciting and cool. There is also a sense of unease about what this will mean in terms of being a mathematician, and what effects it will have on our future employment.

I am also a little worried about what it means for your training as a junior PhD. Often you would try and solve a problem your advisor thinks is doable that they assign to you as a learning exercise. It may be more and more difficult to find problems that a junior PhD can solve but that AI can not. Tim Gowers has written about that here: https://gowers.wordpress.com/2026/05/08/a-recent-experience-...

anonymousDan3• 2 months ago

One concern is that it will become more and more challenging to conduct cutting edge maths research without substantial resources only available at very rich institutions (to pay for state of the art AI assistants).

ontouchstart• 2 months ago

I dug through the chain-of-thought publication and did actually find (a few of) them. If people working on these LLMs are reading, it's very important to me that these are contained in the actual model output.

This is a very important point, especially when the output is from a non-deterministic random walk with some unknown probability distribution.

julianozen• 2 months ago

Nice response to read

Mikhail_K• 2 months ago

But, for scientists, I find that these tools address the problem of the exploding complexity barrier in the frontier.

They do the opposite by locking the results the produce within the slop presentation that needs more AI to comprehend.

JohnHammersley• 2 months ago

Yes, I share your optimism overall, although I think it is raising a question of what the future role of the researcher is (much like the current debate on developer roles).

I attended a conference on AI for maths and open science a few weeks ago, and was struck by just how many examples of AI-supported solutions there already are. Virtually every speaker had an example of either their own use of (often the frontier) AI models in solving a problem that was previously too hard (for various definitions of hard).

I wrote up a few notes [1], and most of the speaker videos are available via the conference website [2].

[1] https://scholarlyfutures.substack.com/p/ai-and-the-practical...

[2] https://www.newton.ac.uk/event/ooew11/

recitedropper6• 2 months ago

This is impressive, no question.

Without knowing all this model has been trained on though, it is pretty hard to ascertain the extent to which it arrived to this "on its own". The entire AI industry has been (not so secretly) paying a lot of experts in many fields to generate large amounts of novel training data. Novel training data that isn't found anywhere else--they hoard it--and which could actually contain original ideas.

It isn't likely that someone solved this and then just put it in the training data, although I honestly wouldn't put that past OpenAI. More interesting though is the extent to which they've generated training data that may have touched on most or all of the "original" tenets found in this proof.

We can't know, of course. But until these things are built in a non-clandestine manner, this question will always remain.

Rover2224• 2 months ago

Seems like a very tin-foil-hat-take to me

recitedropper6• 2 months ago

I'm not letting the government read my brainwaves.

In all seriousness though: My suggestion is that those shepherding the frontier of AI start acting with more transparency, and stop acting in ways that encourage conspiratorial thinking. Especially if the technology is as powerful as they market it as.

mrdependable2• 2 months ago

How is that a "tin-foil-hat" take? It's not a secret, and in fact widely reported, that these companies are spending billions on creating training data.

net012• 2 months ago

I’m quite certain that a few months ago, some problems were claimed to be solved by AI. However, those claims were actually false and were exactly that, solved erdos problems that were not marked as solved and the solution was "found" by AI.

edit: >> https://techcrunch.com/2025/10/19/openais-embarrassing-math/

JacobAsmuth5• 2 months ago

Exactly. Maybe OpenAI paid mathematicians to keep this discovery quiet, then added their proof to the training data, then manipulated a second team into prompting for this question such that the model could regurgitate the solution. This would plausibly explain why the model seems so capable at doing things like refuting fundamental theorems of mathematics while in things like competitive programming, biology, and physics it's merely only in the top 99.9%.

i_love_retros2• 2 months ago

You're a bot! Hey everyone, over here! I found a bot!

golol6• 2 months ago

You are believing a very unlikely scenario. I think the reason is that you have been convinced of a claim which is unlikely and indeed not true. That is: >the model seems so capable at doing things like refuting fundamental theorems of mathematics

That is not true and a complete misrepresentation of recent progress of AI in math. It is therefore not necessary to believe the conspiracy theory you described in order to explain recent progress of AI in math.

recitedropper6• 2 months ago

Thank you for engaging with my comment in a kind and authentic way.

xigoi4• 2 months ago

Being in the top 99.9% is not a big achievement.

muhneesh• 2 months ago

This type of discourse is just inane and more reflective of the author's sensibilities than anything it claims.

Congrats to the OpenAI team for one of the most significant breakthrough discoveries in AI history.

geraneum3• 2 months ago

How dare people think critically of the corporate machine. It’s inane!

recitedropper6• 2 months ago

It is interesting to me how controversial this post is. It has the highest upvotes, and most disagreeing comments, of anything I've typed up on HN.

I'll gladly admit I think what these companies are doing is unethical, and I'm sure that biases my thinking toward skepticism.

That said, there remains way too much that is hidden to be able to effectively evaluate what is going on. You have the perfect storm:

  - AI companies do not share their custom internal harnesses.
  - AI companies do not share their custom internal training data. 
  - AI companies do not share how much compute they allocate to trying to solve problems of this nature. 
  - AI companies are primarily marketing their models to investors as human-replacing rather than human-augmenting. 
  - AI companies are under enormous financial pressure to make their business work.

The last two points incentivize them to find these types of "first proof" successes as aggressively as they can, and I'm sure they've thrown the whole book at it.

Is it likely that they literally had a mathematician discover this, put it into the training data, and then prompted it out? Of course not.

But it would make a world of difference--in evaluating the impressiveness of this discovery and LLM capabilities in general--if we were to know the extent to which the training data crosses over this problem, the harness with which this was ran, and how much compute was spent.

Until they bring more transparency to the whole process--something which some of the mathematicians commenting on this even asked for--I will personally take discoveries of this nature with a good dose of salt.

fergie• 2 months ago

The entire AI industry has been (not so secretly) paying a lot of experts in many fields to generate large amounts of novel training data. Novel training data that isn't found anywhere else--they hoard it--and which could actually contain original ideas.

Really? Any references to read more?

recitedropper6• 2 months ago

  - https://www.theverge.com/cs/features/831818/ai-mercor-handshake-scale-surge-staffing-companies
  - https://outlier.ai/math/en-us
  - https://www.opentrain.ai/
  - https://www.pin.com/blog/ai-labs-hiring-train-models/

Much of this is data annotation, reasoning trace evaluation, and problem set curation. But there is no way they haven't atleast paid some mathematicians to work on research grade problems in tandem with their models, and then used that for training data.

Does this expert data likely contain this proof within it? No. Would it temper the impressiveness to know they have a large amount of novel mathematical training data, an internal Lean harness for evaluation of open conjectures, and spent hundreds of millions in compute to calculate this? Yes.

_heimdall• 2 months ago

As this becomes more common it makes me wonder where the LLM ends and the harness begins.

The underlying model may still effectively be a stochastic parrot, but used properly that can do impressive things and the various harnesses have been getting better and better at automating the use of said parrot.

raincole3• 2 months ago

I like how everyone laughed when OpenAI said their models will have "PhD-Level Intelligence" and now the goalpost has been moved to if AI can create new math (i.e., not PhD-Level, but Leibniz/Euler/Galois level.)

zeofig4• 2 months ago

I still laugh.

johnfn• 2 months ago

Have you updated your priors after this announcement? If not, why not?

dawnerd3• 2 months ago

Yet it still codes like a junior developer that memorized all of stack overflow.

dilap2• 2 months ago

Personally I don't find this to be true anymore! It's not always great and does still will often tend towards unneeded complexity (especially if not pushed a bit), but I often find GPT 5.5 writing code I would have written myself. This was very much not true with earlier models (who make something that worked, but I'd always have to rewrite to make it "good code").

raincole3• 2 months ago

PhDs code like that too. Especially if they're statisticians :)

zulban• 2 months ago

Clearly you've never supervised junior developers.

bdamm2• 2 months ago

Even if the code was like that (it isn't), the power of the current crop of models to analyze data for patterns and build context out of code is leaps and bounds what it was even a year ago. And any developer will tell you that the hardest part of fixing a bug is knowing where the bug is in the first place. Once you know where it is, fixing it is usually trivial.

There is serious magic happening in the construction of model context.

fourseventy• 2 months ago

What is the last model you used... lol. Linus Torvalds himself said the newest models are better than him at coding.

_zoltan_3• 2 months ago

this is not true at all. I'm using Opus and it's great at very complex problems.

RedCinnabar3• 2 months ago

Not true anymore since like early 2025 and especially since last December.

xgulfie• 2 months ago

large language models do not have pigeon-level intelligence. They can't even feed themselves.

staticman24• 2 months ago

What's laughable is an OpenAI employee invented the term "PHD level intelligence" and you think that " PHD Level intelligence" is a real term that describes a real thing and you are repeating it here.

ptrl600• 2 months ago

It's clearly smarter than any PhD and dumber than any ant.

zamadatix3• 2 months ago

My only complaint is the claims always start spreading 6-12 months before the delivery. A little patience goes a long way in what's possible with AI and we all just have to wait and see what parts actually grow this next cycle or not. Guessing at it based on trend lines only leads to people getting excited when it matches their particular guess and ignoring it when it doesn't.

no-name-here3• 2 months ago

> OpenAI said their models will have "PhD-Level Intelligence"
My only complaint is the claims always start spreading 6-12 months before the delivery.

If delivering on such promises "always" occurs 6-12 months after the promise, is that pretty good?

melagonster3• 2 months ago

They said that their specific version of model had has this ability one year ago.

necovek20• 2 months ago

PhDs used to mean publishing a novel mathematical result: when has that changed?

bananaflag8• 2 months ago

They mean "new math" in the sense of more than a novel mathematical result, a new math paradigm or so.

tedbradley• 2 months ago

My good sir or madam, disproving a decades-old conjecture produced by Erdos that has had armies of people in that field have their go at it IS a novel mathematical result.

raincole3• 2 months ago

The gap between novel result and "new math" is as wide as the pacific ocean.

melagonster3• 2 months ago

So finally they reach a part of PhDs level. Current version rely human to integrate results from they model and writting the papper. If LLMs/AIs can do all thing above, we can exactly get a PhDs level model.

golol6• 2 months ago

No it is not Leibniz/Euler/Galois. More like writing good papers that contribute to the broader understanding of a theory. I think if one evaluated a mathematicians research output and it consisted of mostly the kinds of problems AI has solved so far, it would give the impression that this person is somehow very good at picking accessible problems to target, but has not made a larger impact on the field.

yreg• 2 months ago

The goalposts are Euler level not the current model capabilities.

bananaflag8• 2 months ago

As a mathematician, new, conceptual math is when I'll become interested in reading LLM output.

I appreciate very much the work done so far, but this sort of asymptotic/quantitative result didn't interest me much even when it was done by humans.

(This is not snobbery, just a personal preference.)

kamaal3• 2 months ago

Well that's coming.

As a matter of fact more logic and structure to your work, the more easy it is for AI to conquer it. Due to this programming was the first thing that got solved, but pure sciences are next.

If what you do, and how you do can be written down on a piece of paper, then AI can do it.

I do believe programming getting solved will be double assault on these fields.

>This is not snobbery

This is good for the species, what sense does it make to keep treating these fields like they are reserved for the top most intelligent micro percentage of humans? Getting LLM to these things gives some scale to these subjects and thats good.

toilet• 2 months ago

I have no idea about research in mathematics: How will mathematicians judge what constitutes new conceptual math that is actually useful, vs a hallucination that might be novel but doesn't introduce anything actual meaningful?

turzmo3• 2 months ago

Not denying that these advances are impressive, but it is important to consider that this is a cherry-picked result. This doesn’t mean that AI can now be expected to do problems of similar or lower difficulty, but that it happened to work well on one problem. What you won’t see is how many others they had to try to get this result.

cma4• 2 months ago

Earlier of their systems have solve other Erdos problems that people had worked on, this one was more monumental and had had a lot more prior effort that didn't solve, but this isn't a one-off.

Valakas_• 2 months ago

And the goalposts will keep getting moved all the way to the singularity. And then those people will/would say "Oops. I was wrong."

InsideOutSanta• 2 months ago

What were you imagining when OpenAI said that their models would have "PhD-Level Intelligence"? Were you imagining that there were specific tasks they could do that were on par with what a human with a PhD could do? Because by that definition, many computer tools have "PhD-Level Intelligence". By that definition, Wolfram Alpha has "PhD-Level Intelligence".

What I assumed they were saying is that their LLMs would be as intelligent as a human with a PhD across all, or at least most, knowledge tasks, and they clearly are not.

xigoi4• 2 months ago

It would be more impressive if it wasn’t behind the closed doors of a rich company. For all we know, they could’ve paid some mathematicians to work on the problem and pretend that their results are from ChatGPT.

analognoise• 2 months ago

Back when “term rewriting” was “AI”, multiple math tools were released that took known math facts and did tricks like uncovering new integrals - apply the pattern in some depth in a tree, see what pops out.

What was discovered were numerous mistakes in the published literature on the subject. “New math! AI!” No, just mechanical application of rules, human mistakes.

There were things that were theorized, but couldn’t be exhaustively checked until computers were bigger.

Once again, a tool is applied, it has the AI label - its progress! But it isn’t something new. It’s just an LLM.

There’s a consistent under appreciation of AI (and math, honestly), but watching soulless AI mongers declare that their toy has created the new is something of a new low; uninspired, failed creatives, without rhyme or context; this is a bigger version of declaring that your spell checker has created new words.

The result is more impressive than what was done with tables of integrals and SAINT in 1961, sure.

Apparently if you add a “temperature” knob to a text predictor, otherwise sane individuals piss themselves and call it new.

Then again I thought NFTs, crypto, and the Metaverse were stupid, so what do I know.

auggierose2• 2 months ago

Which model did this? Is it available to the public?

agentultra• 2 months ago

I’m curious about the “autonomous” claim. Usually these systems require a human to guide and verify steps, clarify problems, etc. are they claiming that the reinforcement model wasn’t given any inputs, tools, guidance, or training data from humans?

alsetmusic• 2 months ago

AI is about to start taking a very serious role in the creative parts of research, and most importantly AI research itself. While this progress is not unexpected, it reinforces the urgency we feel about understanding this next phase of AI development, the challenges of aligning very intelligent systems, and the future of human-AI collaboration.

I find this hyperbolic, but ya gotta juice up the upcoming IPO. I hate that they took an interesting announcement and reminded me why I hate tech and our society at the end.

dwa3592• 2 months ago

Few questions that the blog did not answer, if anyone knows that'll be great:

- Does anyone know if this was a 1 minute of inference or 1 month?

- How many times did the model say it was done disproving before it was found out that the model was wrong/hallucinating?

- One of the graphs say - the model produced the right answer almost half the times at the peak compute??? did i understand that right? what does peak compute mean here?

mrcwinn• 2 months ago

The back and forth in this discussion reveals to me we are sorting through a kind of philosophical debate about intelligence. That alone tells me LLMs are doing something novel.

SubiculumCode• 2 months ago

I wonder whether there will be progress in string theory from these kinds of applications of AI.

iLoveOncall2• 2 months ago

Absolutely no proof that any LLM actually found the result, and just a mention of an "internal model". Served to you by one of the biggest liars in the world.

Why would anyone believe this to be true even for a split second?

varenc• 2 months ago

This has been an unsolved open problem for 80 years. What you're suggesting is that someone connected to Open AI solved this very hard math problem, but then rather than taking credit for it, falsely attributed it to AI?

The point of having an AI solve an unsolved problem, is to make it very clear that the insight must have come from the AI and wasn't in the training data. Sure, it's possible OpenAI had access to some math professors that solved it and then let an AI model take the credit... but seems unlikely. That human would be turning down a potential Fields Medal for this discovery.

The abridged chain-of-thought from the model also serves as some evidence of LLM origin: https://cdn.openai.com/pdf/1625eff6-5ac1-40d8-b1db-5d5cf925d... (could be fake, though I'm unsure what proof of LLM origin couldn't be faked)

Antibabelic• 2 months ago

That human would be turning down a potential Fields Medal for this discovery.

While interesting, this result is not Fields Medal material.

geraneum3• 2 months ago

but seems unlikely. That human would be turning down a potential Fields Medal for this discovery.

I also don’t like the tin foil hatty theories and don’t know what OpenAI actually did, but an NDA does wonders! Just pointing out that this line of operations is not really unlikely.

iLoveOncall2• 2 months ago

What you're suggesting is that someone connected to Open AI solved this very hard math problem, but then rather than taking credit for it, falsely attributed it to AI?

I'm suggesting that OpenAI invested a lot of resources and money in having someone (or a group of people) disprove this conjecture, so they could claim their LLM disproved it. Yes.

I'm not sure why you're surprised by this, given that everything that Altman has said in the past has turned out to be a lie.

The fact that they gave an EDITED (even rewritten, from the PDF itself) chain of thought is just further proof. Why not give the raw one alongside? No reason at all, except if it doesn't exist.

zuzululu• 2 months ago

This topic and discussion is out of my league what is the implication here ? LLMs aren't a dead end ?

adt• 2 months ago

https://lifearchitect.ai/asi/

3422817• 2 months ago

Nice. By the year 2100 200 Erdos problems will have been solved by AI. Let's build more data centers.

sinuhe69• 2 months ago

How did they jump from finding counter-examples (disproof) to a proof?

neuroelectron• 2 months ago

I wonder if it has anything to do with the fact that AI is a grid of grid-calculating grids. It seems like it would be especially well suited to finding solutions about grids. That is until you consider the fact that even 1 trillion billion grids is still not anywhere close to an infinite grid. So, probably slop.

zone411• 2 months ago

I actually tried using GPT-5.5 Pro on this problem recently. It thought it was making progress on one path, but it made so many mistakes that it didn't feel worth it pushing further. It'll be interesting to check whether it's the same route. I got partial results (proved in Lean) that improve on the best-known results for four Erdős problems with GPT-5.5 Pro

overgard• 2 months ago

I think it's worth being skeptical of this.. there's a way too common pattern of "AI Lab Shows AI Doing Something Only Humans Can Do" only for a bunch of important caveats and limitations to be discovered after the initial hype. And of course, the correction never seems to be as viral as the hype. I'll believe it when a mathematician actually reads the 100+ pages of reasoning.

ferris-booler• 2 months ago

What strikes me in this case (and I haven't seen in other comments) is that it's a _disproof_ of a conjecture put forth by Erdős and supported (at least according to OpenAI) by other professional mathematicians. Erdős, one of the greats, thought that the limit was O(n^{1 + o(1)}), which GPT disproved.

We can argue about recombination/interpolation of training data in LLMs, but even if this was an interpolation, the result was contrarian rather than a confirmation. Any system that can identify an error in Erdős's thinking seems very useful to me (though perhaps he did not spend much time thinking about or checking this particular conjecture).

__0x01• 2 months ago

From the companion paper:

The argument relies crucially on ideas that may, at least in retrospect, be attributed to Ellenberg-Venkatesh, Golod-Shafarevich, and Hajir-Maire-Ramakrishna.

Can someone please elaborate on this?

awdfeswavcra• 2 months ago

The last two are straightforward. The proof relies on a result called the Golod-Shafarevich theorem that gives a criterion for a group to be infinite. Golod and Shafarevich proved this a long time ago (1964). Moreover, if you look at how Golod and Shafarevich used this criterion, it's the same way it's used in the proof: They apply it to some Galois groups that appear in number theory, prove these are infinite in certain cases, and deduce that there exists an infinite tower of number fields with some surprising properties.

Much more recently (2021), Hajir, Maire, and Ramakrishna figured out how to apply the Golod-Shafarevich theorem to a slightly different Galois group to produce an infinite tower of number fields with some even more surprising properties. This is used in the new proof. It requires very slightly modifying the construction of Hajir, Maire, and Ramakrishna to produce the fields needed in this proof, but the explanation of how to do this takes only a paragraph in the human-written summary. (The explanation is more laborious in the original AI writeup).

The relation to Ellenberg-Venkatesh is more indirect. This is where "in retrospect" comes in because this work was not cited in the original AI proof. This has to do with the next step of the proof, after you construct the number field, you need to find many elements of this field with the same norm to produce many vectors of the same length. To do this, the proof uses a pigeonhole argument which uses small split primes of the field (constructed via Hajir, Maire, and Ramakrishna's argument) to construct many ideals. By the pigeonhole principle, you can guarantee two ideals lie in the same class. When two ideals lie in the same class, you get an element of the field. You can rig things so these elements all have the same norm. Ellenberg and Venkatesh had an argument which also used the pigeonhole prnciple to guarantee two ideals lie in the same class to produce elements of the field. They were working on a different problem so their argument was slightly different, but similar.

libraryofbabel5• 2 months ago

This HN thread depressed me. I’m still thinking about why.

Look past the press-releasey gushing from OpenAI and there are all sorts of interesting and subtle questions here about the role for LLMs in mathematical research. I urge folks to click through to the accompanying comments from mathematicians published alongside the result. There is a really interesting discussion going on. I particularly recommend Tim Gowers’ remarks. This is really interesting stuff!

Yet the comments are just a battleground of people rehearsing the same tired arguments about LLMs from 2023, refutations of those arguments, angry counters, etc.

Does it make anyone else sad that the battle lines seem to have been drawn 3 years ago and we just seem to have the same fights over and over?

I wonder if we’ll still be doing this two years hence.

jryan49• 2 months ago

People are afraid for their livelihood. What do you expect?

conception• 2 months ago

Livelihoods and lives.

libraryofbabel5• 2 months ago

Well yes, but there is a choice being made here and I would love to believe we can do better. The rational response to being afraid about your livelihood isn’t to spend time filling every HN thread on LLMs with embittered negativity. Not to mention all the flat denials that LLMs can do mathematics and write decent code, which is almost a self-contradictory position if you are worried they are going to replace you.

There are a lot of big issues at stake here and just because a person is interested in what AI can do and curious to discuss it does not make them uncritically positive about it’s effects on society, the economy, and the world. Yet that is often the assumption and it leads to battle lines being drawn, on every AI discussion, over and over again. It means the serious discussion gets swamped and that makes me sad.

godelski4• 2 months ago

I think part of it is that one side throws rocks and so it never even matters was is in the article. It becomes a battle if the article is good or the article is shit.

Yes, I'm tired too. I want you have real discussions about these things. But the problem is everyone believes their reality is real and anyone's reality that disagrees is fake. It just escalates. I take long breaks from HN because I realize I just come to the forums and end up being angry. Why do we do this to ourselves? The reality is that at a core level we usually want the same things.

tinfoilhatter• 2 months ago

IMO it's because people have fragile egos / worldviews they don't want shaken. Pretty much any opinion I type on this website gets instantly downvoted, unless it reaffirms the popular / mainstream narrative, because I happen to see the world differently than most.

This website is quite awful, and I also don't know why I spend any time on it. It's definitely not a website intended for meaningful discourse. It's a website where you can reaffirm whatever opinion is already established, and if your opinion is at all controversial or even just out of the box, you'll be punished for it.

getnormality• 2 months ago

Yes, this and every internet forum will still be doing this two years hence. Your life will be better if you take to heart this famous passage from Nietzsche:

I do not want to wage war against what is ugly. I do not want to accuse; I do not even want to accuse those who accuse. Looking away shall be my only negation.

libraryofbabel5• 2 months ago

Looking away shall be my only negation.

I’ve been thinking of building myself my own frontend to HN that makes it impossible to view comments, for this reason. Yet sometimes there are still really interesting discussions and it’s hard to let go of what for me feels like the last social media I want to be part of.

scosman• 2 months ago

We won’t be doing it in 2 years. By then my side will have won!

doginasuit3• 2 months ago

I find it understandable, it is common to evaluate human intelligence vs AI as a zero-sum competition, because that is how employers typically understand it and LM providers market it. AI proving itself moves the needle in an uncomfortable direction for all of us without very robust job security.

I wonder if we’ll still be doing this two years hence.

It is going to take some time for people to recognize that AI has a very different set of competencies that compliments human intelligence rather well. It is unlikely to eclipse human intelligence at scale, and the companies betting on that will fall behind. That is when the conversation will start to shift.

bdamm2• 2 months ago

It isn't necessarily the case.

Another wishful/hopeful thought is that the human experience itself is valuable, that competing for resources and living within a social network and having physical needs somehow creates value that is essential for companies to operate.

But is it really the case? I don't think we know that, and I don't know if the economy that results when all the white collar and much of the blue collar workers no longer understand how to participate in whatever the economy is becoming. Because it is starting to look like old money is coming around, and soon we will all be serfs to the creature comforts of those who have money now, upward mobility will be a thing of the past, and a small ruling elite over the vast subservient majority will form, reorganizing societies to more resemble middle ages lordship rather than what emerged in the 50's and 60's following WWII.

class3shock• 2 months ago

It seems like the outcome options are:

1. AI is developed to be smart enough to actual replace people, destroying the labor force and immensely concentrating power.

This seems like bs hyperbole but I am not an expert.

2. AI turns out to be a bubble of false promises and hype, bursts, and takes the stock market and economy with it.

I thought this was the most likely but I keep not hearing popping, so maybe the it's:

3. AI continues to be a tool that can substantially increase productivity in some areas and cause huge societal changes in others. The AI companies keep the hype train going or maybe it tapers off over time until talk meets reality but "real" AI never shows up and the bubble never pops because it's not one. Eventually there is 0-3 new FAANG companies with untouchable control of a tech we increasingly have to use to stay relevant.

Even if we avoid option 1 and 2, 3 doesn't exactly bode well either.

dogwalker5000• 2 months ago

because that is how employers typically understand it and LM providers market it.

Every few months you get an article of some executive bragging that he fire an entire department of people because of AI.

It was adversarial from the start. The idle rich who don’t have to work for a living and their sycophants who somehow believe they won’t be replaced vs … everyone else.

I used to think that the common tale of AI rebelling in Hollywood movies was unlikely. Turns out we don’t even need rogue AI, our fellow men are quite willing to wipe the rest of us out.

ex-aws-dude3• 2 months ago

Lets just be real its because a lot of programmer's ego is built on intelligence/being a coding wizard and this threatens that ego

If suddenly anyone can code we're not that special anymore.

RedCinnabar3• 2 months ago

What I don’t understand is why people dismiss this kind of progress with false claims. Especially when discussing programming, people start to act irrational using arguments from back in 2022.

I think that you can easily address your concerns about this new technology (since we all are concerned about the future) but at the same time acknowledge how revolutionary it is.

dev1ycan• 2 months ago

Wouldn't surprise me if they're just paying math geniuses to do math research and attribute it to AI models.

armanj• 2 months ago

useless fact: there is no mention to "gpt" in this article. the ai is referred to as "An internal OpenAI model".

fromMars• 2 months ago

Seems rather depressing to me but maybe I am a Luddite.

JacobAsmuth5• 2 months ago

Exactly. I would rather we let these discoveries stay hidden for a while longer such that human ingenuity may untangle them from the coils of reality. A machine? A mechanical man? Deins to produce something as pure as mathematics without the divine fervor of the ineffable spirit of Man?! It's just not what God wants.

pickleRick2435• 2 months ago

Human ingenuity is untangling perhaps the deepest question of all- what is the essence of Reason and the intellect that so privileges man? I don't know if it's what God wants, but it's certainly getting close to some existentially fundamental questions.

While many seem to be anxious or pessimistic about the future of intellectual/artistic pursuits (understandable although I disagree), I do find the utter lack of curiosity or interest in the incredible machinery that is causing all the fuss to be striking.

unmole• 2 months ago

I can't tell if this is satire.

precision1k• 2 months ago

I see mixed emotions here. I understand both. On one hand it's exciting and fascinating. On the other it's concerning. One concern I haven't seen mentioned is the possibility that, as these models become larger and more powerful, their capability to solve frontier math problems will also grow. Does there become a point where humans are no longer the driver of innovation and research in this world, and instead are relegated to become stewards of the AI models whose purpose is to push the boundaries of mathematics, theoretical physics and other academic disciplines?

noslenwerdna• 2 months ago

For those of us who care about the answers to these questions, rather than who gets credit for doing it, we will welcome any faster means of solving these problems.

gpderetta• 2 months ago

what if at some point we will no longer be able to understand the answers?

greenknight• 2 months ago

I believe that the answer is yes it will happen... the question is when will that point will occur.

Right now, we are in a transition period... Models are improving, but they are not capable just yet to take over.

Where do you see it being in a years time? or 2? or 5?

aussieguy1234• 2 months ago

So we've got the proof, what are the practical applications of this?

hajile• 2 months ago

It proves the limit doesn't exist, but doesn't seem to show how to find one of these more optimized limits. Given that nobody has ever brute-forced a counter-example, it may be that calculating these solutions is P=NP hard which would mean sticking with the status quo unless we find a good algorithm to calculate it on quantum computers and can actually get quantum computers large enough to work on the problem.

It's interesting as a math problem and test of AI, but not much else IMO.

zmmmmm• 2 months ago

As a side observation, it is striking but also not surprising in retrospect that the big successes in AI are coming from domains where things are fundamentally verifiable. Both software and math are either fully verifiable or low-cost verifiable (breaking a test is not the same cost as building a bridge and watching it fall down to see if it worked).

Other domains are extracting value but I feel like there's an order of magnitude difference. It raises the question, what other domains fit into these categories where the AI itself has pretty much free reign to verify its own results?

oscord• 2 months ago

Can it model a sustainable economy model, with human happiness and fulfilment indexes and planet preservation focus? Current capitalism and the red thing are so tired!

ninjagoo5• 2 months ago

Many folks are upset about the supplanting of human effort by ai. Umanwizard voiced this valid concern below [1], but his comment got downvoted, unfairly, IMHO, instead of just being addressed. So putting out at least my response as its own top-level comment for visibility.

the closer the expertise you spent your whole life building is to being worthless.

Along with all the rest of what humans find meaningful and fulfilling.

[1] https://news.ycombinator.com/item?id=48215122

momo26• 2 months ago

I'm curious that giving an counter-example is kind of easy to disprove. But can the model really prove something correctly and rigorously? Cuz now it seems like all the knowledge is based on the existed thing, and none of them can prove a myth.

cwmoore• 2 months ago

From the meandering and self-loving article:

“ For decades, it was widely believed that this rate was essentially the best possible, and no construction could improve significantly over the square grid. In technical terms, Erdős conjectured an upper bound of n 1 + o ( 1 ) n 1+o(1) in which the additional o ( 1 ) o(1) indicates a term tending to 0 0 with n n.

Our new result disproves this conjecture. More precisely, for infinitely many values of n n, the proof constructs configurations of n n points with at least n 1 + δ n 1+δ unit-distance pairs, for some fixed exponent δ > 0 δ>0. (The original AI proof does not give an explicit δ δ, but a forthcoming refinement due to Princeton mathematics professor Will Sawin has shown one can take δ = 0.014 δ=0.014.)”

foota• 2 months ago

They should feed it the classification of finite simple groups and get it to simplify it/turn it more constructive.

Topology1• 2 months ago

As someone starting grad school for pure mathematics, this has me both excited and nervous, but mainly the latter...

globulus20232• 2 months ago

In the article there is a nice clear diagram of the “square grid” arrangement that was previously thought to be optimal.

Can anyone point me to a diagram of the newly found optimal arrangement?

globulus20232• 2 months ago

In the article there is a diagram of the “square grid” arrangement that achieves approximately 2n points separated by unit distance.

Can anyone point me to a diagram of what the newly found solution looks like?

purpleidea• 2 months ago

You'd think a billion dollar company would be able to normalize the sound level on their video :/

llagerlof• 2 months ago

That will require AGI.

ai_fry_ur_brain4• 2 months ago

Im convinved they target these pure math problems because math is very occulted to the masses, and therefor can use math "discoveries" as a way to make an LLM seem more impressive than it is.

Everything is a grift.

What are the odds that if they ran the same prompt from scratch, with the same context and instructions that it would arrive at the same answer? Unlikely. I think its more likely that this is a 1:500000 chance and OpenAI can afford to brute force this result and justify the expense for marketing.

ccvannorman• 2 months ago

I looked at all linked articles and could not find an example of the points (they show a square grid of points with n~=100 but no other ordering of points to show the more optimal layout(s)).

Is there anywhere an image example of a superior layout for example with n>={100,1000,10000}..? I would love to see it. I am imagining it would look somewhat like a sloppy pizza.

lg56892• 2 months ago

One of the authors said in a reddit comment (and I hope I am summarizing accurately) that it's impossible to show a diagram as the smallest instance of the technique gives like 10^1000000 points.

krackers4• 2 months ago

That was for this particular technique, but the comment suggested that a similar construction can (but is unproven) to beat erdos and at least gives some flavor of what this looks like

https://x.com/i/status/2057490144546927046

lesostep2• 2 months ago

I am cautious about AI "discoveries" after Mythos paper.

What was the process of a writing a paper? Was the question asked by a mathematician? Was the paper right from a get-go or was there someone who pointed out mistakes?

How much attempts were made before solution was found?

I will eat my words if an AI oneshotted that one without any external help, but for know I am left wandering whether it's a new way to attribute discoveries to companies instead of people who put the work in

crnkofe• 2 months ago

I'm also wondering about the process. What was the prompt, what they fed into the model, what it was trained on, etc. The article reads like a marketing post.

Nevertheless new maths is exciting and might lead to what I find slightly more interesting - new physics.

andy12_2• 2 months ago

Was the question asked by a mathematician?

As per the report, the prompt used to solve the problem is AI-written and the solution was initially graded by an AI grading pipeline. They don't say this explicitly, but it seems like OpenAI has an automatic pipeline where they prompt models for solutions to famous math problems (which wouldn't be unexpected given how flashy a solution to a famous math problem looks)

Was the paper right from a get-go or was there someone who pointed out mistakes?

Also as per the report, the output of the model isn't really a "paper"; it's a very terse 2 page solution which is apparently correct. The paper was later written based on this solution to make it more presentable.

How much attempts were made before solution was found?

Given that this appears to be from an automated pipeline, I would say that it had many attempts. But either way, the blogpost says that with enough test-time compute, the model finds this same solution 50% of the time.

[1] https://cdn.openai.com/pdf/74c24085-19b0-4534-9c90-465b8e29a...

0xDEAFBEAD• 2 months ago

I am cautious about AI "discoveries" after Mythos paper.

Can you be more specific? I'm still under the impression that Mythos was a huge deal:

https://xcancel.com/hlntnr/status/2052479493801975987

fontain• 2 months ago

There is little evidence that Mythos is any better at finding bugs than any other system. Mythos appears to be impactful because people are, for the first time, using lots of resources (for free from Anthropic) to try and find security issues. The actual bugs found are mostly inconsequential, any chart showing a giant leap in fixes that doesn’t consider whether they were even using any tooling before and whether these are serious issues is junk. If you read the partner’s summaries of Mythos so far, it is a damp squib. Maybe that’ll change but at least for now there is no evidence Mythos is anything but marketing hype.

https://www.aisi.gov.uk/blog/our-evaluation-of-claude-mythos...

https://daniel.haxx.se/blog/2026/05/11/mythos-finds-a-curl-v...

rithdmc• 2 months ago

I will eat my words if an AI oneshotted that one without any external help

like having a colleague peer review your paper, or bouncing ideas off a mentor before you write them down?

I agree there's a lot of AI marketing BS at the moment, but revising approaches based on feedback is a good thing.

lesostep2• 2 months ago

Yes, and no. As someone who had studied and had taught math, I really like peer review.

But peer review is a powerful tool.

Carefully choosing what lemmas to give for solving and reviewing the result is my favorite way to teach young minds. Yes, they do solve most problems themselves. But, most of them likely wouldn't be able to do that before someone dissects problem beforehand and points at weak spots in their explanations.

And that's why I question who prompted the model, how they prompted it, and how much their own ideas influenced the output.

I admit, I don't know enough to judge how much of the right solution was actually enclosed in a first reply

utopiah2• 2 months ago

No need to be cautious.

If a for-profit (because... you know, OpenAI isn't at all what it initially was) huge corporation (again, not a cute startup trying to help humanity) publish anything it's a piece of marketing. Every single word a corporation say is marketing.

So... that's also that, a piece of marketing to sell more of whatever their potential client can buy. It's not a piece of research. It's an ad. That's it.

callamdelaney• 2 months ago

The only relevant question is, how much did it cost?

Quentak• 2 months ago

I'd like to know how many tokens in total went into solving this problem. Have they talked about this? It matters whether they got this result in 10 million tokens or 10 billion. Whether it's closer to 1 human working on this for 1 year or 1000 humans for 1 year. The news feels different when the probability of one AI run solving this is 1 in a thousand vs 1 a million. Approximately I'm asking about the amount of money it cost to solve it, which has to include the failed parallel runs.

ApolloRising• 2 months ago

I would like to know this as well, including how long was it working on this problem?

_diyar• 2 months ago

Any answer to this question must also consider the current cost/token and its downward trajectory as algo and hardware advances drive down costs.

vessenes• 2 months ago

Good q. You can see that tantalizing graph where compute is displayed on a log axis and pass@1 goes up to like 50/60%. I’d love to know just how much compute is encoded on that axis.

I guess you can get some estimate from the excerpted CoT, but that CoT might be backed by quite a lot of parallel compute.

DiogenesKynikos• 2 months ago

Calling all LLM skeptics. How did a "stochastic parrot" just disprove an Erdős conjecture that mathematicians couldn't figure out for decades?

isolli• 2 months ago

Question:

The conjecture was about an upper bound for the maximum number of pairs. It has been disproven.

Was the Erdos problem the conjecture itself, or was it about the actual maximum number of pairs? (In which case it will probably never be solved.)

The problem is defined in the narrow version here: https://www.erdosproblems.com/90

AlexToaniAI• 2 months ago

So nowadays. AI may use different field and get lots of break through that migh human can't done! That's nuts!

num42• 2 months ago

I am not surprised! The birth of computer science was rooted in the desire to automate mathematical discovery and proof writing.

kevinwang2• 2 months ago

Nitpicky/not important, but they say:

Since loglog(n) tends to infinity with n, the additional term in the exponent tends to 0, meaning these constructions achieve growth only slightly faster than linear.

Would anyone else describe the previous asymptotic behavior like that? I mean obviously loglogn to O(1) is a quantum leap, but wouldn't you describe loglogn as "grows so slowly it's almost constant", so the constructions achieve growth "almost n^{1+c}"? But I guess that might be overcorrecting too hard.

kevinwang2• 2 months ago

Just reread this comment and it's not easy to understand, but the edit window is passed.

What I meant is that they describe loglogn the same way you could describe O(n) or O(n^2) -- it "tends to infinity with n", even though my mental model for loglogn is to treat it as barely more than constant. See: https://cs.stackexchange.com/questions/148197/who-said-first...

anonymousDan3• 2 months ago

Come back to me when it has solved P=NP.

PeterStuer• 2 months ago

There's still parrots parroting 'stochastic parrot'.

An OpenAI model has disproved a central conjecture in discrete geometry

Related Stories