Terence Tao on O1

https://mathstodon.xyz/@tao/113132502735585408

by dselsam • 2 years ago

664 482 2 years ago

jarbus• 2 years ago

I wonder how long it took for each of the responses it gave

diggan• 2 years ago

It varies a lot. If it's a simple question, it just does 3-4 sections of "thinking & reflection" but for more complicated ones I think I've seen something like 10 or more. Maybe 3-4 seconds per section on average I'd guess.

zamadatix3• 2 years ago

It's unclear if Terence is referring to "GPT-o1... a prototype version of the model that I was granted access to" as in "he was given access to GPT-o1 by the research team" or as in "he is using o1-preview". The differences in scale and quality between his shared output and the answer I get trying the same prompt from o1-preview suggest perhaps the former (otherwise luck). I haven't actually seen any examples of how long o1 "full" will think about this kind of question, though I expect it's somewhere in the same ballpark given the thought expansion still only has one real concept in it.

d0mine2• 2 years ago

"with even the latest tools the effort put in to get the model to produce useful output is still some multiple (but not an enormous multiple now, say 2x to 5x) of the effort needed to properly prompt and verify the output. However, I see no reason to prevent this ratio from falling below 1x in a few years, which I think could be a tipping point for broader adoption of these tools in my field"

Given the log scale on compute to improve performance, it is not a guarantee that the ratio can be improved so much in a few years

aoeusnth1• 2 years ago

The y axis is also log scale (log likelihood). It’s a power law, not an exponential law.

d0mine2• 2 years ago

I was referring to the o1 AIME accuracy figure (x log scale compute, y is % (not log)) and similar https://openai.com/index/learning-to-reason-with-llms/

bitexploder• 2 years ago

The novelty to me is that the “The experience seemed roughly on par with trying to advise a mediocre, but not completely incompetent, graduate student.” in so many subject areas! I have found great value in using LLMs to sort things out. In areas where I am very experienced it can be really helpful at tons of small chores. Like Terrence was pointing out in his third experiment — if you break the problem down it does solid work filling in smaller blanks. You need the conceptual understanding. Part of this is prompting skill. If you go into an area you don’t know you have to try and build the prompts up. Dive into something small and specific and work outward if the answer is known. Start specific and focused if starting from the outside in. I have used this to cut through conceptual layers of very complex topics I have zero knowledge in and then verify my concepts via experts on YT/research papers/trusted sources. It is an amazing tool.

wenc11• 2 years ago

This has been my experience as well. I treat LLMs like an intern or junior who can do the legwork that I have no bandwidth to do myself. I have to supervise it and help it along, checking for mistakes, but I do get useful results in the end.

Attitudinally, I suspect people who have had experience supervising interns or mentoring juniors are probably those who are able to get value out of LLMs (paid ones - free ones are no good) rather than grizzled lone individual contributors -- I myself have been in this camp for most of my early career -- who don't know how to coax value out of people.

wslh• 2 years ago

... that I have no bandwidth to do myself.

One of the most interesting aspects of this thread is how it brings us back to the fundamentals of attention in machine learning [1]. This is a key point: while humans have intelligence, our attention is inherently limited. This is why the concept behind Attention Is All You Need [2] is so relevant to what we're discussing.

My 2 cents: our human intelligence is the glue that binds everything together.

[1] https://en.wikipedia.org/wiki/Attention_(machine_learning)

[2] https://en.wikipedia.org/wiki/Attention_Is_All_You_Need

wenc11• 2 years ago

Once GPT is tuned more heavily on Lean (proof assistant) -- the way it is on Python -- I expect its usefulness for research level math to increase.

I work in a field related to operations research (OR), and ChatGPT 4o has ingested enough of the OR literature that it's able to spit out very useful Mixed Integer Programming (MIP) formulations for many "problem shapes". For instance, I can give it a logic problem like "i need to put i items in n buckets based on a score, but I want to fill each bucket sequentially" and it actually spits out a very usable math formulation. I usually just need to tweak it a bit. It also warns against weak formulations where the logic might fail, which is tremendously useful for avoiding pitfalls. Compare this to the old way, which is to rack my brain over a weekend to figure out a water-tight formulation of MIP optimization problem (which is often not straightforward for non-intuitive problems). GPT has saved me so much time in this corner of my world.

Yes, you probably wouldn't be able to use ChatGPT well for this purpose unless you understood MIP optimization in the first place -- and you do need to break down the problem into smaller chunks so GPT can reason in steps -- but for someone who can and does, the $20/month I pay for ChatGPT more than pays for itself.

side: a lot of people who complain on HN that (paid/good - only Sonnet 3.5 and GPT4o are in this category) LLMs are useless to them probably (1) do not know how to use LLMs in way that maximizes their strengths; (2) have expectations that are too high based on the hype, expecting one-shot magic bullets. (3) LLMs are really not good for their domain. But many of the low-effort comments seem to mostly fall into (1) and (2) -- cynicism rather than cautious optimism.

Many of us who have discovered how to exploit LLMs in their areas of strength -- and know how to check for their mistakes -- often find them providing significant leverage in our work.

threeseed3• 2 years ago

side

Or (4) LLMs simply do not work properly for many use cases in particular where large volumes of trained data doesn't exist in its corpus.

And in these scenarios rather than say "I don't know" it will over and over again gaslight you with incoherent answers.

But sure condescendingly blame on the user for their ignorance and inability to understand or use the tool properly. Or call their criticism low-effort.

wenc11• 2 years ago

That's category (3).

zamadatix3• 2 years ago

What's the difference between (3) and (4), shouldn't the former contain the latter?

WhatIsDukkha• 2 years ago

I entirely agree about their utility.

HN, and the internet in general, have become just an ocean of reactionary sandbagging and blather about how "useless" LLMs are.

Meanwhile, in the real world, I've found that I haven't written a line of code in weeks. Just paragraphs of text that specify what I want and then guidance through and around pitfalls in a simple iterative loop of useful working code.

It's entirely a learned skill, the models (and very importantly the tooling around them) have arrived at the base line they needed.

Much Much more productive world by just knuckling down and learning how to do the work.

edit: https://aider.chat/ + paid 3.5 sonnet

benterix3• 2 years ago

I've found that I haven't written a line of code in weeks

Which is great until your next job interview. Really, it's tempting in the short run but I made a conscious decision to do certain tasks manually only so that I don't lose my basic skills.

apsurd2• 2 years ago

LLMs are certainly not useless.

But "lines of code written" is a hollow metric to prove utility. Code literacy is more effective than code illiteracy.

Lines of natural language vs discrete code is a kind of preference. Code is exact which makes it harder to recall and master. But it provides information density.

by just knuckling down and learning how to do the work?

This is the key for me. What work? If it's the years of learning and practice toward proficiency to "know it when you see it" then I agree.

riku_iki• 2 years ago

Meanwhile, in the real world, I've found that I haven't written a line of code in weeks. Just paragraphs of text that specify what I want and then guidance through and around pitfalls in a simple iterative loop of useful working code.

could it be that you are mostly engaged in "boilerplate coding", where LLMs are indeed good?

threeseed3• 2 years ago

HN, and the internet in general, have become just an ocean of reactionary sandbagging and blather about how "useless" LLMs are.

This is cult like behaviour that reminds me so much of the crypto space.

I don't understand why people are not allowed to be critical of a technology or not find it useful.

And if they are they are somehow ignorant, over-reacting or deficient in some way.

delusional4• 2 years ago

What sort of problems do you solve? I tried to use it. I really did. I've been working on a tree edit distance implementation base on a paper from 95. Not novel stuff. I just can't get it to output anything coherent. The code rarely runs, it's written in absolutely terrible style, it doesn't follow any good practices for performant code. I've struggled with getting it to even implement the algorithm correctly, even though it's in the literature I'm sure it was trained on.

Even test cases have brought me no luck. The code was poorly written, being too complicated and dynamic for test code in the best case and just wrong on average. It constantly generated test cases that would be fine for other definitions of "tree edit distance" but were nonsense for my version of a "tree edit distance".

What are you doing where any of this actually works? I'm not some jaded angry internet person, but I'm honestly so flabbergasted about why I just can't get anything good out of this machine.

anujsjpatel2• 2 years ago

For someone who didn't study a STEM subject or CS in school, I've gone from 0 to publishing a production modern looking app in a matter of a few weeks (link to it on my profile).

Sure, it's not the best (most maintainable, non-redundant styling) code that's powering the app but it's more than enough to put an MVP out to the world and see if there's value/interest in the product.

holoduke• 2 years ago

People in general don't like change and are naturally defending against it. And the older people get the greater the percentage of people fighting against it. A very useful and powerful skill is to be flexible and adaptable. You positioned yourself in the happy few.

skydhash2• 2 years ago

Much Much more productive world by just knuckling down and learning how to do the work.

The fact everyone that say they've become more productive with LLMs won't say how exactly. I can talk about how VIM have make it more enjoyable to edit code (keybinding and motions), how Emacs is a good environment around text tooling (lisp machine), how I use technical books to further my learning (so many great books out here). But no one really show how they're actually solving problems with LLMs and how the alternatives were worse for them. It's all claims that it's great with no further elaboration on the workflows.

I haven't written a line of code in weeks. Just paragraphs of text that specify what I want and then guidance through and around pitfalls in a simple iterative loop of useful working code.

Code is intent described in terms of machinery actions. Those actions can be masked by abstracting them in more understandable units, so we don't have to write opcodes, but we can use python instead. Programming is basically make the intent clear enough so that we know what units we can use. Software engineering is mostly selecting the units in a way to do minimal work once the intent changes or the foundational actions do.

Chatting with a LLM look to me like your intent is either vague or you don't know the units to use. If it's the former, then I guess you're assuming it is the expert and will guide you to the solution you seek, which means you believe it understands the problem more than you do. The second is more strange as it looks like playing around with car parts, while ignoring the manuals it comes with.

What about boilerplate and common scenarios? I agree that LLMs helps a great deal with that, but the fact is that there are perfectly good tools that helped with that like snippets, templates, and code generators.

ijustlovemath• 2 years ago

How much do you typically pay in a month of tokens?

evilfred3• 2 years ago

writing code is the easy part, designing is hard and not LLMable

cjbgkagh2• 2 years ago

In my view these models produce above average code which is good enough for most jobs. But the hacker news sampling could be biased towards the top tier of coders - so their personal account of it not being good enough can also be true. For me the quality isn't anywhere close to good enough for my purposes, all of my easy code is already done so I'm only left working on gnarly niche stuff which the LLMs are not yet helpful with.

For the effect on the industry, I generally make the point that even if AI only replaces the below average coder it will cause a downward pressure on above average coders compensation expectation.

Personally, humans appear to be getting dumber at the same time that AI is getting smarter and while, for now, the crossover point is at a low threshold that threshold will of course increase over time. I used to try to teach ontologies, stats, SMT solvers to humans before giving up and switching to AI technologies where success is not predicated on human understanding. I used to think that the inability for most humans to understand these topics was a matter of motivation, but have rather recently come to understand that these limitations are generally innate.

minkles2• 2 years ago

That’s fine until your code makes its way to production, an unconsidered side effect occurs and then you have to face me.

You are still responsible for what you do regardless of the means you used to do it. And a lot of people use this not because it’s more productive but because it requires less effort and less thought because those are the hard bits.

I’m collecting stats at the moment but the general trend in quality as in producing functional defects is declining when an LLM is involved in the process.

So far it’s not a magic bullet but a push for mediocrity in an industry with a rather bad reputation. Never a good story.

amrrs• 2 years ago

Curious why Aider? Why not Cursor ?

rafaelmn• 2 years ago

I use sonet 3.5 and while it's actually usable for codegen (compared to gpt/copilot) it's still really not that great. It does well at tasks like "here's a stinky collection of tests that accrued over time - clean this up in style of x" but actually writing code still shows fundamental lack of understanding of underlying API and problem (the most banal example being constantly generating `x || Array.isArray(x)` test)

wokwokwok2• 2 years ago

I've found that I haven't written a line of code in weeks

Please post a video of your workflow.

It’s incredibly valuable for people to see this in action, otherwise they, quite legitimately, will simply think this is not true.

skybrian• 2 years ago

What sort of code do you write this way?

acedTrex5• 2 years ago

I've found that I haven't written a line of code in weeks

How are people doing this, none of the code that gpt4o/copilot/sonnet spit out i ever use because it never meets my standards. How are other people accepting the shit it spits out.

perching_aix4• 2 years ago

HN, and the internet in general, have become just an ocean of reactionary sandbagging and blather about how "useless" LLMs are.

Now imagine how profoundly depressing it is to visit a HN post like this one, and be immediately met with blatant tribalism like this at the very top.

Do you genuinely think that going on a performative tirade like this is what's going to spark a more nuanced conversation? Or would you rather just the common sentiment be the same as yours? How many rounds of intellectual dishonesty do we need to figure this out?

_wire_• 2 years ago

Meanwhile, in the real world, I've found that I haven't written a line of code in weeks. Just paragraphs of text that specify what I want and then guidance through and around pitfalls in a simple iterative loop of useful working code.

Comment on first principles:

Following the dictum that you can't prove the absence of bugs, only their presence, the idea of what constitutes "working code" deserves much more respect.

From an engineering perspective, either you understand the implementation or you don't. There's no meaning to iteratively loop of producing working code.

Stepwise refinement is a design process under the assumption that each step is understood in a process of exploration of the matching of a solution to a problem. The steps are the refinement of definition of a problem, to which is applied an understanding of how to compute a solution. The meaning of working code is in the appropriateness of the solution to the definition of the problem. Adjust either or both to unify and make sense of the matter.

The discipline of programming is rotting when the definition of working is copying code from an oracle you run it to see if it goes wrong.

The measure of works must be an engineering claim of understanding the chosen problem domain and solution. Understanding belongs to the engineer.

LLMs do not understand and cannot be relied upon to produce correct code.

If use of an LLM puts the engineer in contact with proven principles, materials and methods which he adapts to the job at hand, while the engineer maintains understanding of correctness, maybe that's a gain.

But if the engineer relies on the LLM transformer as an oracle, how does the engineer locate the needed understanding? He can't get it from the transformer: he's responsible for checking the output of the transformer!

OTOH if the engineer draws on understanding from elsewhere, what is the value of the transformer but as a catalog? As such, who has accountability for the contents of the catalog? It can't be the transformer because it can't understand. It can't be the developer of the transformer because he can't explain why the LLM produces any particular result! It has to be the user of the transformer.

So a system of production is being created whereby the engineer's going-in position is that he lacks the understanding needed to code a solution and he sees his work as integrating the output of an oracle that can't be relied upon.

The oracle is a peculiar kind of calculator with a unknown probability of generating relevant output that works at superhuman speeds, while the engineer is reduced to an operator in the position of verifying that output at human speeds.

This looks like a feedback system for risky results and slippery slope towards heretofore unknown degrees of incorrectness and margins for error.

At the same time, the only common vernacular for tracking oracle veracity is in arcane version numbers, which are believed, based on rough experimentation, to broadly categorize the hallucinatory tendencies of the oracle.

The broad trend of adoption of this sketchy tech is in the context of industry which brags about seeking disruption and distortion, regards its engineers as cost centers to be exploited as "human resources", and is managed by a specialized class of idiot savants called MBAs.

Get this incredible technology into infrastructure and in control of life sustaining systems immediately!

po76• 2 years ago

Give it a few months. ChatGPT will be recommending GPTs to use or do it automatically.

Nothing is static in the way things are moving.

airstrike• 2 years ago

It also doesn't help that Lean has had so many breaking changes in such little time. When I tried using GPT-4 for it, it mostly rendered old code that would fail to run unless you already knew the answer and how to fix it, which basically made it entirely unhelpful.

benterix3• 2 years ago

people who complain on HN that (paid/good - only Sonnet 3.5 and GPT4o are in this category)

Correction: I complain that the only decent model in "Open"AI's arsenal, that is GPT-4, has been replaced by a cheaper GPT-4o, which gives subpar answers to most of my question (I don't care it does it faster). As they moved it to "old, legacy" models, I expect they will phase it out, at which point I'll cancel my OpenAI subscriptions and Sonnet 3.5 will become the clear leader for my daily tasks.

Kudos to Anthropic for their great work, you guys are going in the right direction.

bongodongobob4• 2 years ago

Nah, o1 is fucking impressive. It's really fucking good. I'm guessing you haven't used it yet.

eab-• 2 years ago

Why do you expect GPT being tuned on Lean will help it for research-level math?

CamperBob22• 2 years ago

But many of the low-effort comments seem to mostly fall into (1) and (2) -- cynicism rather than cautious optimism.

One good riposte to reflexive LLM-bashing is, "Isn't that just what a stochastic parrot would say?" Some HN'ers would dismiss a talking dog because the C code it wrote has a buffer overflow error.

jazzyjackson3• 2 years ago

Id rather live in the world without talking dogs if their main utility is authoring buggy code

Workaccount23• 2 years ago

It's understandable that people whose career and lifelong skill set that are seemingly on the precipice of obsolescence are going to be extremely hostile to that threat.

How many more years is senior swe work going to be a $175k/yr gig instead of an $75k check-what-the-robot-does gig?

riffraff• 2 years ago

_can_ GPT be tuned more heavily on Lean? It looks like the amount of python code in the corpus would outnumber Lean something like 1000:1. Although I guess OpenAI could generate more and train on that.

EvgeniyZh• 2 years ago

There is ~3 order of magnitude more Python code in the internet than Lean code (200GB vs 200MB in the stack v2). You can't tune it "the same way"

agumonkey3• 2 years ago

Fair point but a lot of python code is redundant and low quality.

agumonkey3• 2 years ago

side question, are there good OR websites / platforms (reddit, mastodon) to get involved in the field ?

rabf• 2 years ago

Most people are on X.

dtquad• 2 years ago

Most OR researchers and practitioners are on Mastodon.

l33t73322732• 2 years ago

I an also working in OR and I have had the complete opposite experience with respect to MILP optimization(and the research actually agrees; there was a big survey paper published earlier this year showing LLMs were mostly correct on textbook problems but got more and more useless as complexity and novelty increased.)

The results are boiler plate at best, but misleading and insidious at worst, especially when you get into detailed tasks. Ever try to ask a LLM what a specific constraint does or worse ask it to explain the mathematical model of some proprietary CPLEX syntactic sugar? It hallucinates the math, the syntax, the explanation, everything.

wenc11• 2 years ago

Can you point me to that paper? What version of the model were they using?

Have you tried again with the latest LLMs? ChatGPT4 actually (correctly) explains what each constraint does in English -- it doesn't just provide the constraint when you ask it for the formulation. Also, not sure if CPLEX should be involved at all -- I usually just ask it for mathematical formulations, not CPLEX calling code (I don't use CPLEX). The OR literature primarily contains math formulations and that's where LLMs can best do pattern matching to problem shape.

Many of the standard formulations are in here:

https://msi-jp.com/xpress/learning/square/10-mipformref.pdf

All the LLM is doing is fitting the problem description to a combination of these formulations (and others).

marmakoide• 2 years ago

I had the same experience with computational geometry.

Very good at giving a textbook answer ("give a Python/ Numpy function that returns the Voronoi diagram of set of 2d points").

Now, I ask for the Laguerre diagram, a variation that is not mentioned in textbooks, but very useful in practice. I can spend a lot of time spoon-feeding the answer, I just have the bullshiting student answers.

I tried other problems like numerical approximation, physics simulation, same experience.

I don't get the hype. Maybe it's good at giving variations of glue code ie. Stack Overflow meet autocomplete ? As a search tool it's bad because it's so confidently incorrect, you may be fooled by bad answers.

Davidzheng5• 2 years ago

I'm not sure the lean coverage of pure math research is that much (maybe like 1% is represented on mathlib). But I think a system like alpha proof could even today be useful for mathematicians--I mostly dislike systems like o1 where they confidently say nonsense with such high frequency. But i think value is already there.

lanstin4• 2 years ago

The point about using lean is you don't have to trust you can verify.

thelastparadise2• 2 years ago

but for someone who can and does, the $20/month I pay for ChatGPT more than pays for itself.

Would you be willing to pay even more, if it meant you were getting proportionally more valuable answers?

E.g. $200/month or $2,000/month (assuming the $2,000/month gets into employee/intern/contractor level of results.)

This might drive a positive feedback loop.

sterlind• 2 years ago

I also do OR-adjacent work, but I've had much less luck using 4o for formulating MIPs. It tends to deliver correct-looking answers with handwavy explanations of the math, but the equations don't work and the reasoning doesn't add up.

It's a strange experience, like taking a math class where the proofs are weird and none of the lessons click for you, and you start feeling stupid, only to learn your professor is an escaped dementia patient and it was gobbledygook to begin with.

I had a similar experience yesterday using o1 to see if a simple path exists through s to t through v using max flow. It gave me a very convincing-looking algorithm that was fundamentally broken. My working solution used some techniques from its failed attempt, but even after repeated hints it failed to figure out a working answer (it stubbornly kept finding s->t flows, rather than realizing v->{s,t} was the key.)

It's also extremely mentally fatiguing to check its reasoning. I almost suspect that RLHF has selected for obfuscating its reasoning, since obviously-wrong answers are easier to detect and penalize than subtly-wrong answers.

mjburgess• 2 years ago

Yip. We need research into how long it takes experts to repair faulty answers, vs. generate them on their own.

Benchmarking 10,000 attempts on an IQ test is irrelevant if on most of those attempts the time taken to repair an answer is long than the time to complete the test yourself.

I find its useful to generate examplars in areas you're roughly familiar with, but want to see some elaboration or a refresher. You can stich it all together to get further, but when it comes time to actually build/etc. something -- you need to start from scratch.

The time taken to reporduce what it's provided, now that you understand it, is trivial compared to the time needed to repair its flaws.

lanstin4• 2 years ago

Yeah I have been using them to help with learning graduate maths as a grad student. Claude Sonnet 3.5 was unparalleled and the first quite useful one. GPT4o preview seems about equal (based on cutting and pasting the past six months of prompts into it).

CJefferson3• 2 years ago

I'm currently teaching a course on MIP, and out of interest I tried asking 4o about some questions I ask students. It could give the 'basic building blocks' (How to do x!=y, how to do a knapsack), but as soon as I asked it a vaguely interesting question that wasn't "bookwork", I don't think any of it's models were right.

I'm interested on how you seem to be getting better answers than me (or, maybe I just discard the answer once I can see it's wrong and write it myself, once I see it's wrong?)

In fact, I just asked it to do (and explain) x!=y for x,y integer variables in the range {1..9}, and while the constraints are right, the explanation isn't.

wenc11• 2 years ago

I had to prompt it correctly (tell it to exclude x=y case in the x≠y formulation), but ChatGPT seems to have arrived at the correct answer:

https://chatgpt.com/share/66e652e1-8e2c-800c-abaa-92e29e0550...

andrepd2• 2 years ago

I take cynicism over unbridled optimism. People speak as if we were on the cusp of technological singularity, but I've seen nothing to indicate we're not already past the inflection point of the logistic curve, and well into diminishing returns territory.

RayVR• 2 years ago

I’m amazed you have had any luck with 4o. I found 4 was much better than 4o but still quite bad.

I tried to use 4/4o for a MIP several months ago. Frequently, it would iterate through three or four bad implementations over and over.

Claude 3.5 has been a significant improvement. I don’t really use chatgpt for anything at this point.

ninetyninenine• 2 years ago

A specialized LLM could possibly meet his criteria already.

317070• 2 years ago

Probably. The missing factor is the dataset and the fact that so far OAI seems to be the only one who has figured out how to train this thing for reasoning.

But yeah, given o1 exists, it looks very doable. It's hard to imagine a reason for why something matching his criteria would be more than a decade out.

nyc111• 2 years ago

I checked the links and I think it's amazing and it answers with Latex formatted notation.

But I was curious and I asked something very simple, Euclid's first postulate and I got this answer:

Euclid's Postulate 1: "Through any two points, there is exactly one straight line."

In fact Euclid's Postulate 1 is "To draw a straight line from any point to any point." http://aleph0.clarku.edu/~djoyce/java/elements/bookI/bookI.h...

I think AI answer is not correct, it may be some textbook interpretation but I was expecting Euclid's exact wording.

Edit: Google's Gemini gives the exact wording of the postulate and then comments that this means that you can draw one line bitween two points. I think this is better

supermatt• 2 years ago

I think AI answer is not correct, it may be some textbook interpretation but I was expecting Euclid's exact wording

It was written before English even existed. That said, the original never implied "exactly one", so I agree its a bad translation.

slavboj• 2 years ago

Euclid wrote in ancient Greek, so the "exact wording" in English does not exist.

pama2• 2 years ago

The original text is: Ἠιτήσθω ἀπὸ παντὸς σημείου ἐπὶ πᾶν σημεῖον εὐθεῖαν γραμμὴν ἀγαγεῖν. Roughly: let it be required that from any point to any point it is possible to draw a straight line.

Both gpt4o and o1 roughly know the correct original text, so prompting, the model’s background memory, or random chance may influence your outcomes, though hopefully (in an improved model) you should never get you incorrect info.

https://farside.ph.utexas.edu/Books/Euclid/Elements.pdf

Edit: in case it isnt clear, I could not reproduce this error on my end with o1-mini

roywiggins3• 2 years ago

it's definitely wrong though, "exactly one" straight line between two points is a different postulate and a stronger one.

Euclid has been translated, restated, and re-presented in enough books and textbooks that I'd expect a big-enough LLM to have actually memorized this correctly tbh

wging• 2 years ago

“Exact wording” would be Ancient Greek. Euclid did not even write in English. You’re checking whether the model matches a specific translation, which is not valuable. If you search around you’ll find many sources that choose a more intelligible phrasing.

layer8• 2 years ago

Regardless of the wording being exact or not, ChatGPT’s answer is incorrect in its contents. The statement “exactly one” requires the parallel postulate, since otherwise it’s not necessarily true. Specifically in spherical geometry, which is considered to be consistent with Euclid’s first four postulates (i.e. without the parallel postulate).

The bottom line is, you can’t take any single LLM statement at face value, even in seemingly easy to answer cases like this.

roywiggins3• 2 years ago

it's the point-line postulate, you can use it as part of a set of axioms equivalent to Euclid but it definitely not one of Euclid's

https://en.wikipedia.org/wiki/Point%E2%80%93line%E2%80%93pla...

nyrikki• 2 years ago

Euclid's Elements is less pervasive on the Internet then content produced for Liberal Arts math courses. As those courses tend to emphasize critical thinking and problem-solving math over pure theory and advanced concepts, they tend to be far more common and tend to win compared to more domain specific meanings.

Examples:

https://en.wiktionary.org/wiki/Euclidean_geometry

https://www.cerritos.edu/dford/SitePages/Math_70_F13/Postula...

Problems with polysemy across divergent, more advanced theories has been one of my biggest challenges in probing some of my areas of intrest.

Funny enough, one of my pet areas of obscure interest, riddled basins, is constantly muddied not by math, but LSAT questions, specifically non-math content directed at a reading comprehension test: "September 2006 LSAT Section 1 Question 26"

IMHO a lot of the prompt engineering you have to do with these highly domain specific problems is avoiding the most common responses in the corpus.

LLM responses will tend to reflect common usage, not academic terminology unless someone cares enough to change that for a specific case.

kzz102• 2 years ago

It's interesting that humans would also benefit from the "chain of thought" type reasoning. In fact, I would argue all students studying math will greatly increase their competence if they are required to recall all relevant definition and information before using it. We don't do this in practice (including teachers and mathematicians!) because recall is effortful, and we don't like to spent more effort than necessary to solve a problem. If recall fails, then we have to look up information which takes even more effort. This is why in practice, there is a tremendous incentive to just "wing it".

AI has no emotional barrier to wasted effort, which make them better reasoners than their innate ability would suggest.

Satam• 2 years ago

Wow! I love this take. Somehow with all this evidence of COT helping out LLMs, I never thought about using it more myself. Sure, we kind of do it already but definitely not to the degree of LLMs, at least not usually. Maybe that's why writing is so often admired as a way to do great thinking - it enables longer chains of thoughts with less effort.

schappim• 2 years ago

Showing your work in tests is kind of like “chain of thought” reasoning, but there’s a slight difference. Both force you to break down your process step by step, making sure the logic holds and you aren’t skipping crucial steps. But while showing your work is more about demonstrating the correct procedure, “chain of thought” reasoning pushes you to recall relevant definitions and concepts as you go, ensuring a deeper understanding. In both cases, the goal is to avoid just “winging it,” but “chain of thought” really digs into the recall aspect, which humans tend to avoid because it’s effortful.

jvvw• 2 years ago

I assumed that everybody did this when trying to solve a maths problem they are stuck on (thinking university type level maths rather than school maths) and when I was teaching I would always get people to go back to the definitions.

I wasn't amazing at maths research (did a PhD and post-doc and then gave up) but my experience was that it was partly thinking hard about things and grappling with what was going on and trying to break it down somehow, but also scanning everything you know related to the problem, trying to find other problems that resemble it in some way that you can steal ideas from etc.

perihelions3• 2 years ago

I'm so excited in anticipation of my near-term return to studying math, as an independent curiosity hobby. It's going to be epically fun this time around with LLM's to lean on. Coincidentally like Terence Tao, I've also been asking complex analysis queries* of LLM's, things I was trying to understand better in my working through textbooks. Their ability to interpret open-ended math questions, and quickly find distant conceptual links that are helpful and relevant, astonishes me. Fields laureate Professor Tao (naturally) looks down on the current crop of mathematics LLM—"not completely incompetent graduate student..."—but at my current ability level that just means looking up.

*(I remember a specific impressive example from 6 months ago: I asked if certain definitions could be relaxed to allow complex analysis on a non-orientable manifold, like a Klein bottle, something I spent a lot of time puzzling over, and an LLM instantly figured out it would make the Cauchy-Riemann equations globally inconsistent. (In a sense the arbitrary sign convention in CR defines an orientation on a manifold: reversing manifold orientation is the same as swapping i with -i. I understand this now, solely because an LLM suggested looking at it). Of course, I'm sure this isn't original LLM thinking—the math's certainly written down somewhere in its training material, in some highly specific postgraduate textbook I have no knowledge of. That's not relevant to me. For me, it's absolutely impossible to answer this type of question, where I have very little idea where to start, without either an LLM or a PhD-level domain specialist. There is no other tool that can make this kind of semantic-level search accessible to me. I'm very carefully thinking how best to make use of such an, incredibly powerful but alien, tool...)

WanderPanda• 2 years ago

How will we even measure this? Benchmarks are gamed/trained on and there is no way that there is much signal in the chatbot arena for these types of queries?

I think in just a few month the average user will not be able to tell the difference in performance between the major models

nybsjytm9• 2 years ago

How will you know if its answers are correct or not?

perihelions3• 2 years ago

Because I'm verifying everything by hand, as is the whole point of studying pure mathematics.

rossant• 2 years ago

I agree. Having access to a kind of semantic full search engine on basically all textbooks on Earth feels like a superpower. Even better would be if it could pinpoint the exact textbook references it found the answer in.

artninja1988• 2 years ago

could not generate conceptual ideas of their own

Is the most important part imo. A big goal should be some ai system coming up with its own discovery and ideas. Really unclear how we can get from the current paradigm to it coming up with something like general relativity, like Einstein. Does it require embodiment?

sfink• 2 years ago

Why should that be a big goal? It's difficult, it's not what they are good at, and they can get a lot better at assisting in other ways through incremental improvements. I'm happy to leave this part to the humans, at least for now, especially when there's so much more improvement still possible in other directions.

It also seems like one of those things where we ought to ask whether we should, before asking whether we could. Why not focus on areas that are easier, more beneficial, and less problematic from a "should" perspective?

roywiggins3• 2 years ago

we don't know how to reliably produce humans who produce GR-level ideas, this might be biting off a lot more than we can chew

sgt1012• 2 years ago

Is there a list of discoveries or siginficant works/constructions made by people collaborating with LLM's? I mean as opposed to specific deep networks like Alphafold or Graphcast?

adt• 2 years ago

I'd like to see that, too.

I have a related list of GPT accomplishments here: https://docs.google.com/spreadsheets/d/1kc262HZSMAWI6FVsh0zJ...

sgt1012• 2 years ago

Super spreadsheet - very useful. Thank you for doing the work and sharing.

rvnx3• 2 years ago

It may cause a reputation or legal issue, so it is not in their interest to admit it. In the real world, is there PhD students or researchers using ChatGPT to move forward and help them think their ideas ?

Obviously yes, but admitting it may not be the right move.

abstractbill• 2 years ago

My experience with O1 has been very different. I wouldn't even say it's performing at a "good undergrad" level for me.

For example, I asked a pretty simple question here and it got completely confused:

https://moorier.com/math-chat-1.png https://moorier.com/math-chat-2.png https://moorier.com/math-chat-3.png

(Full chat should be here: https://chatgpt.com/share/66e5d2dd-0b08-8011-89c8-f6895f3217...)

jghn2• 2 years ago

Anecdata, but I've been finding O1 to be worse than 4o & Claude 3.5 Sonnet. To add insult to injury, it's slower & chattier.

anujsjpatel2• 2 years ago

And sometimes it just bugs out and doesn't give any response? Faced that twice now, it "thought" for like 10-30s then no answer and I had to click regenerate and wait for it again.

abdullahkhalids• 2 years ago

Thinking about training LLMs on geometry. A lot of information in the sources would be contained in the diagrams accompanying the text. This model is not multi-modal, so maybe it wasn't trained on the accompanying diagrams at all.

I would really like if people check on a set of geometry and a set of analysis questions and compare the difference.

jazzyjackson3• 2 years ago

It will be trash. I'll have to dig up a chat I had the weekend GPT4 was released, I was musing about dodecahedron packing problems and GPT4 started with an assertion that a line through a sphere intersects the surface 3 times.

Maybe if you fine tuned it on Euclid's Elements and then allowed it to run experiments with Mathematica snippets it could check its assumptions before spouting nonsense

almostgotcaught• 2 years ago

Why would they do this - make it speak like a customer service agent. The ideal experience here is short and succinct, not verbose and obsequious.

ljlolel2• 2 years ago

Performs better on chat bot arena head to head

svdr• 2 years ago

Did you find out what the error was in computing the volume of the truncated icosidodecahedron?

gary_0• 2 years ago

Tao mentions grad students; I wonder how they feel reading this?

As LLMs continue to improve I feel like anyone making a living doing the "99% perspiration" part of intellectual labor is about to enter a world of hurt.

fragmede6• 2 years ago

The experience seemed roughly on par with trying to advise a mediocre, but not completely incompetent, graduate student.

And you thought you had imposter syndrome before!

teaearlgraycold3• 2 years ago

Or can everyone now lead research projects and build businesses?

asdasjhG2• 2 years ago

No, almost everyone who gets funding for a business already belongs to the monied royalty and gets it either directly from his family, via friends of the family or laundered through a VC.

There are exceptions of course, but that's how the bulk of businesses, especially those with stupid ideas are funded. In the latter category success does not even matter, the trust fund baby just has to have the appearance of a leader position.

lupire4• 2 years ago

He clarified that he is referring to a very specific task of assisting an expert, not PhD research in general.

eigenvalue4• 2 years ago

The o1 model is really remarkable. I was able to get very significant speedups to my already highly optimized Rust code in my fast vector similarity project, all verified with careful benchmarking and validation of correctness.

Not only that, it also helped me reimagine and conceptualize a new measure of statistical dependency based on Jensen-Shannon divergence that works very well. And it came up with a super fast implementation of normalized mutual information, something I tried to include in the library originally but struggled to find something fast enough when dealing with large vectors (say, 15,000 dimensions and up).

While it wasn’t able to give perfect Rust code that compiled on the very first try, it was able to fix all the bugs in one more try after pasting in all the compiler warning problems from VScode. In contrast, gpt-4o usually would take dozens of tries to fix all the many rust type errors, lifetime/borrowing errors, and so on that it would inevitably introduce. And Claude3.5 sonnet is just plain stupid when it comes to Rust for some reason.

I really have to say, this feels like a true game changer, especially when you have really challenging tasks that you would be hard pressed to find many humans capable of helping with (at least without shelling out $500k+/year in compensation for).

And it’s not just the performance optimization and relatively bug free code— it’s the creative problem solving and synthesis of huge amounts of core mathematical and algorithmic knowledge plus contemporary research results, combined with a strong ability to understand what you’re trying to accomplish and making it happen.

Here is the diff to the code file showing the changes:

https://github.com/Dicklesworthstone/fast_vector_similarity/...

aprilthird20212• 2 years ago

But a lot of what you pay humans $500k a year for is to work with enormous existing systems that an LLM cannot understand just yet. Optimizing small libraries and implementing fast functions though is a huge improvement in any programmer's toolbox.

eigenvalue4• 2 years ago

Yes, that’s certainly true, and that’s why I selected that library in particular to try with it. The fact that it’s mathematical— so not many lines of code, but each line packs a lot of punch and requires careful thought to optimize— makes it a perfect test bed for this model in particular. For larger projects that are simpler, you’re probably better off with Claude3.5 sonnet, since it has double the context window.

dyauspitr• 2 years ago

Can’t Gemini work with a million+ input tokens?

jdiez17• 2 years ago

1,337 additions

cough

Ylpertnodi• 2 years ago

you would be hard pressed to find many humans capable of helping with (at least without shelling out $500k+/year in compensation for).

And now we have a $number we can relate, and refer, to.

kldnav• 2 years ago

Tao and Aaronson are optimistic about LLMs. What are they telling their students? That math and science degrees will soon have the same value as a degree in medieval dance theory?

If they are overly optimistic, perhaps it would be good to hear the opinions of Wiles and Perelman.

ljlolel2• 2 years ago

If you look at a lot of people’s PHDs, we now teach these things to 1st years. PhDs today do incredible deep work and the edge of science will just go further.

raincole2• 2 years ago

Tao isn't that optimistic. His opinion on LLMs is rather conservative.

https://www.scientificamerican.com/article/ai-will-become-ma...

If you want to prove an unsolved conjecture, one of the first things you need to do is to break it up into smaller pieces, each of which has a better chance of being proven. But you will often break up a problem into harder problems. It’s very easy to transform a problem into one that’s harder than into one that’s simpler. And AI has not demonstrated any ability to be any better than humans in this regard.

Not sure if O1 changed his mind tho.

Davidzheng5• 2 years ago

What does this mean? Of course math AI will take over top research in next ten years but usefulness to society has never been a goal of pure mathematics. I don't know if you understand the motivation for studying pure math. Personally I think it will be mostly good for research math

asdasjhG2• 2 years ago

The "value of a degree" means the employment prospects for the degree holder.

Which is going to zero if the optimistic predictions are correct, so the optimistic professors should warn their students.

I understand the motivation for pure math quite well. It is about beauty, understanding things and discovering things for oneself. If computers do the work, the discovery part is gone and pure math is ruined.

For the non-research part, the AI zealots will want to replace all human labor with software.

ein0p• 2 years ago

Idk I think the fact that it needs “hints” and “prodding” is a good thing, myself. Otherwise we wouldn’t need humans to get those answers, would we. I want it to augment humans, not replace them.

MrFots• 2 years ago

Incompetent Graduate Students is the name of my new sketch group.

itissid• 2 years ago

One thing it's certainly doing better is exploring the search space better e.g.: https://x.com/sg3487/status/1835040593703010714

If you know the contours of the answer and can describe what you are looking for it can quickly find it for you.

nmca• 2 years ago

Note the selection effect in “a mediocre graduate student” (that got to work with Terry Tao)

reverseblade2• 2 years ago

Here's a little test I try on LLMs. So far only O1 and Microsoft Copilot (bing chat) was able to solve it:

Find a, b, c distinct positive integers satisfying a^3 + b^3 = c^4. Hint: try dividing all sides by c^3, then giving values to (a/c) and (b/c).

zeroonetwothree5• 2 years ago

Any integer that is a sum of 2 cubes produces a solution. Since if x^3 + y^3 = z then we have (xz)^3 + (yz)^3 = z^4. So this doesn't seem super interesting?

nybsjytm9• 2 years ago

Daniel Litt, an algebraic geometer on twitter, said "Pretty impressed by o1-preview! Still not having much luck asking it to do any interesting math but it seems much more reliable with simple things; I can actually imagine it being a net time-saver at this point with some non-mathematical tasks."

Any other takes by mathematicians out there?

knotthebest• 2 years ago

Do note that Terry has access to the full o1. o1-preview is, well, a preview.

fsndz• 2 years ago

Completely agree with Terence Tao. this is a real advancement. I've always believed that with the right data allowing the LLM to be trained to imitate reasoning, it's possible to improve its performance. However, this is still pattern matching, and I suspect that this approach may not be very effective for creating true generalization. As a result, once o1 becomes generally available, we will likely notice the persistent hallucinations and faulty reasoning, especially when the problem is sufficiently new or complex, beyond the "reasoning programs" or "reasoning patterns" the model learned during the reinforcement learning phase. https://www.lycee.ai/blog/openai-o1-release-agi-reasoning

gcanyon2• 2 years ago

The experience seemed roughly on par with trying to advise a mediocre, but not completely incompetent, graduate student.

Coming from Terence Tao that seems pretty remarkable to me?

fnordpiglet4• 2 years ago

Rewind your mind to 2019 and imagine reading a post that said

“The experience seemed roughly on par with trying to advise a mediocre, but not completely incompetent, graduate student.”

With regard to interacting with the equivalent of Alexa. That’s a remarkable difference in 5 years.

talldayo4• 2 years ago

To be honest, I have gotten 100x more useful answers out of Siri's WolframAlpha integration than I ever have out of ChatGPT. People don't want a "not completely incompetent graduate student" responding to their prompts, they want NLP that reliably processes information. Last-generation voice assistants could at least do their job consistently, ChatGPT couldn't be trusted to flick a light switch on a regular basis.

meowface• 2 years ago

I use both for different things. WolframAlpha is great for well-defined questions with well-defined answers. LLMs are often great for anything that doesn't fall into that.

thelastparadise2• 2 years ago

Wait til you generate WolframAlpha queries from natural language using Claude 3.5 and use it to interpret results as well.

Karrot_Kream• 2 years ago

How does this square up with literally what Terence Tao (TFA) writes about O1? Is this meant to say there's a class of problems that O1 is still really bad at (or worse than intuition says it should be, at least)? Or is this "he says, she says" time for hot topics again on HN?

segmondy• 2 years ago

Then you have a skill issue. 10 million paying are for GPT monthly because a large of them are getting useful value out of it. WolframAlpha has been out for a while and didn't take off for a reason. "GPT couldn't be trusted to flick a light switch on a regular basis" pretty much implies you are not serious or your knowledge about the capabilities of LLM is pretty much dated or derived from things you have read.

fnordpiglet4• 2 years ago

I use home assistant with the extended open ai integration from HACS. Let me tell you, it’s orders of magnitude better than generic voice assistants. It can understand fairly flexibly my requests without me having a literal memory of every device in the house. I can ask for complex tasks like turning every light in the basement on without there being a zone basement by inferring from the names. I have air quality sensors throughout and I can ask it to turn on the fan in areas with low air quality and if literally does it without programming an automation.

Usually Alexa will order 10,000 rolls of toilet paper and ship them to my boss when I ask it to turn on the bathroom fan.

Personally tho the utility of this level of skill (beginner grad in many areas) for me personally is in areas I have undergraduate questions in. While I literally never ask it questions in my field, I do for many other fields I don’t know well to help me learn. over the summer my family traveled and I was home alone so I fixed and renovated tons of stuff I didn’t know how to do. I work a headset and had the voice mode of ChatGPT on. I just asked it questions as I went and it answered. This enabled me to complete dozens of projects I didn’t know how to even start otherwise. If I had had to stop and search the web and sift through forums and SEO hell scapes, and read instructions loosely related and try to synthesize my answers, I would have gotten two rather than thirty projects done.

noch2• 2 years ago

The important point is, I feel, that most people are not even at the level of intelligence of a "a mediocre, but not completely incompetent, graduate student." A mediocre graduate science student, especially of the sort who graduates and doesn't quit, is a very impressive individual compared to the rest of us.

For "us", having such a level of intelligence available as an assistant throughout the day is a massive life upgrade, if we can just afford more tokens.

thewanderer19832• 2 years ago

A mediocre graduate science student, especially of the sort who graduates and doesn't quit, is a very impressive individual compared to the rest of us.

Incorrect. University graduates shows a good work ethic, a certain character and a ability to manage time. It's not a measure of being better than the rest of humanity. Also, it's not a good measure of intelligence. If you only want to view the world through credentials. Academics don't consider your intelligence until you have a Ph.D and X years of work in your field. Industry only uses graduates as a entry requirement for junior roles and then favors and cares only about your years of experience after that. Given that statement I can only assume you haven't been to University. You are mistaken to think, especially in time we are in now that the elite class are any more knowledgeable then you are.

kiba• 2 years ago

We are more limited by our emotions, and then our skills in learning and acquiring knowledge.

Intelligence is probably a distant third.

a_wild_dandan2• 2 years ago

My sheer productivity boost from these models is miraculous. It's like upgrading from a text editor to a powerful IDE. I've saved a mountain of hours just by removing tedious time sinks -- one-off language syntax, remembering patterns for some framework, migrating code, etc. And this boost applies to nearly all of my knowledge work.

Then I see contrarians claiming that LLMs are literally never useful for anyone, and I get "don't believe your lying eyes" vibes. At this point, such sentiments feel either willfully ignorant, or said in bad faith. It's wild.

sn92• 2 years ago

Anyone intelligent enough to make a living programming likely has more than enough IQ to become a mediocre somewhat competent graduate student in math.

They just don't have the background, and probably lack the interest to dedicate studying for a few years to get to that level.

meroes2• 2 years ago

I mean paying several hundred to thousands of grad students to RLHF for several years and you get a corpus of grad-student text. I'm not surprised at all. AI companies hire grad students to RLHF in every subject matter (chemistry, physics, math, etc).

The grad-students write the prompts, correct the model, and all of that is fed into a "more advanced" model. It's corpi of text. Repeat this for every grade level and subject.

Ask the model that's being trained on chemistry grad level work a simple math question and it will probably get it wrong. They aren't "smart". It's aggregations of text and ways to sample and then predict.

fnordpiglet4• 2 years ago

Except you’re talking about a general purpose foundation model that’s doing all these subjects at once. It’s not like you choose the subject specific model with Claude or gpt-01.

The key isn’t whether these things are smart or not. The key is that they put something that can answer basic grad level questions on almost any subject. For people that don’t have a graduate level education in any subject this is a remarkable tool.

I don’t know why the statement that “wow this is useful and a remarkable step forward” is always met with “yeah but it’s not actually smart.” So? Half of all humans have an IQ less than 100. They’re not smart either. Is this their value? For a machine, being able to produce accurate answers to most basic graduate level questions is -science fiction- regardless of whether it’s “smart.”

The NLP feat alone is stunning, and going from basically one step above gibberish to “basic grad school” in two years is a mouth dropping rate of change. I suspect folks who quibble over whether it’s “real intelligence” or simply a stochastic parrot have lost the ability to dream.

JumpCrisscross4• 2 years ago

The first profession AI seems on track to decimate is programming. In particular, the brilliant but remote and individual contributor. There is an obvious conflict of interest in this forum.

dartos8• 2 years ago

No, the first profession AI was on track to decimate was artists, but that didn’t really happen.

AI just destroyed shutterstock.

__loam8• 2 years ago

It's going to be incredible watching you people write way more code than you can feasibly maintain.

talldayo4• 2 years ago

I would better believe that if any superior software was being primarily designed by AI.

IncreasePosts2• 2 years ago

Before it can replace the brilliant programmer, it needs to be able to replace the mediocre programmer. There is so much programming and other tech/it related work that businesses or people want, but can't justify paying even low tech salaries in America for.

So far, there is little chance of a non-technical person developing a technical solution to their problems using AI.

ken472• 2 years ago

The conflict of interest might have something to do with the fact that OpenAI's CEO/founder was once a major figure in Y Combinator. But I think you wanted to insinuate that the conflict of interest ran in the other direction.

Once ChatGPT can even come close to replacing a junior engineer, you can retry your claim. The progression of the tech underlying ChatGPT will be sub-linear.

vessenes2• 2 years ago

I see this theory a lot but mostly from people who haven’t tried pair coding with a quality llm. In fact these llms give experienced developers super powers; you can be crazy productive with them.

If you think we are close to the maximum useful software in the world already, then maybe. I do not believe that. Seeing software production and time costs drop one to two orders of magnitude means we will have very different viable software production processes. I don’t believe for a second that it disenfranchises quality thinkers; it empowers them.

langcss• 2 years ago

I believe LLMs decimating the role of a software engineer requires AGI, which the second that happens decimates all jobs.

What it may do is change the job requrements. Web/JS has decimated (reduced by 90% or more) MFC C++ jobs after all.

The programmer doesnt just write Python. That is the how... not the what.

IshKebab3• 2 years ago

I doubt it. It can do some impressive stuff for sure, but I very rarely get a perfectly working answer out of ChatGPT. Don't get me wrong, it's often extremely useful as a starting point and time saver, but it clearly isn't close to replacing anyone vaguely competent.

HarHarVeryFunny4• 2 years ago

The programmers who will find LLMs most useful are going to be those who prior to LLMs were copying and pasting from Stack Overflow, and asking questions online about everything they were doing - tasks that LLMs have precisely replaced (it has now memorized all that boilerplate code, consensus answers, and API usage examples).

The developers who will find LLMs the least useful are the "brilliant" ones who never found any utility in any of that stuff, partly because they are not reinventing the wheel for the 1000th time, but instead addressing more challenging and novel problems.

TrackerFF• 2 years ago

Even more amazing, there plenty - PLENTY - of posters here that routinely either completely shit on LLMs, or casually dismiss them as "hype", "useless", and what have you.

I've been saying this for quite some time now, but some people are in for a very rude awakening when the SOTA models 5-10 years from now are able to completely replace senior devs and engineers.

Better buckle up, and start diversifying your skills.

zeroonetwothree5• 2 years ago

Not sure how this post is evidence of AIs replacing senior devs.

ramraj07• 2 years ago

The way I see it these models especially O1 is an intelligence booster. If you start with zero it gives you back zero. Especially if you’re just genuinely trying to use it and not just trying to do some gotcha stuff.

achierius• 2 years ago

Diversifying to what? When AI can fully replace senior developers the world as we know it is over. Best case capitalism enters terminal decline: buy rifles. Worst case, hope that whatever comes out the either side is either benevolent or implodes quickly.

bamboozled• 2 years ago

Remind your mind to 1850, imagine seeing a lightbulb.

fumeux_fume• 2 years ago

Rewind your mind to 1950 and reading that the future is chatting with bots about solving math homework.

nathanasmith• 2 years ago

They would be wondering why it took so long.

ksec• 2 years ago

Which is why I think the AI era isn't hype but very much real. Jensen said AI has reached the era of iPhone.

We wont have AGI or ASI, whatever definition people have with those terms in the next 5 - 10 years. But I would often like to refer AI as Assisted or Argumented Intelligence. And it will provide enough value that drives current Computer and Smartphone sales for at least another 5 - 10 years. Or 3-4 cycles.

j_timberlake• 2 years ago

Terry is a genius that can get that value out of an LLM.

Average Joe can't do anything like that yet, both because he won't be as good at prompting the model, and because his problems in life aren't text-based anyway.

fnordpiglet4• 2 years ago

I think this is where multi modal LLMs are so powerful. The ability to directly speak to the LLM with your voice is huge.

benreesman• 2 years ago

Reading anything Terrence Tao writes is thought provoking and I doubt I’m seeing anything others haven’t.

There’s at least a “complexity” if not a “problem” in terms of judging models that to a first approximation have been trained on “everything”.

Have people tried putting these things up against serious mathematical problems that are well studied? With or with Lean hinting has anyone gotten like, the Shimura-Taniyama conjecture/proof out?

kevinventullo2• 2 years ago

I believe this is the farthest anyone has gotten: https://deepmind.google/discover/blog/ai-solves-imo-problems...

No FLT yet, but as someone who was initially quite skeptical, I’m starting to be convinced!

lupire4• 2 years ago

Those are not serious mathematical problems. Those are toy math problems, crafted backwards from known facts, designed to be solved in under 1hr, that are hard for most humans because they lack the memorization and recall and search speed that the computer has.

ak_111• 2 years ago

He mentions that he posed to O1 the same challenge he posed to a previous GPT (which he also previously blogged about), so I am wondering how much O1 benefited from potentially "seeing" this discussion in its training set (which probably contains a very well recent snapshot of the world wide web).

lysecret• 2 years ago

In some of the responses o1 actually was telling me it had a cutoff of 2023. not sure if they officially stated it somewhere.

lewhoo• 2 years ago

I wonder if those responses could already be influenced by the fact that the cutoff for some of the models out there was indeed 2023 and people wrote about it all over the internet.

2muchcoffeeman• 2 years ago

What a burn

“The experience seemed roughly on par with trying to advise a mediocre, but not completely incompetent, graduate student.”

busyant• 2 years ago

Well, one thing is clear.

Math grad students everywhere now have a benchmark to determine if Terry Tao considers them to be mediocre or incompetent.

darby_nine• 2 years ago

JUST when you thought the chatbot was dead

__loam8• 2 years ago

This seems like it's just feeding the output back into the model and using more compute to try and get better answers. If that's all, I don't see how it fundamentally solves any of the issues currently present in LLMs. Maybe a marginal improvement in accuracy at the cost of making the computation more expensive. And you don't even get to see the so called reasoning tokens.

lalaithion• 2 years ago

It needs a bigger context, but the moment someone can feed an entire GitHub repo into this thing and ask it to fix bugs... I think O2 may be the beginning of the end.

s1mon• 2 years ago

I'm not a mathematician much beyond AP Calc in high school (almost 40 years ago). I am deeply fascinated by Bézier curves and geometric continuity. I've spent a lot of time digging up research papers and references about this and related Computer Aided Geometric Design mathematics. Mostly I skim them for the illustrations and more geometric relations. For several years I've been trying to understand how to make sure that a Bézier curve is G3 to an adjoining curve, given the tangent direction, and first and second curvature derivatives.

I've tried a variety of ways to ask various LLMs to help solve this. Finally with access to ChatGPT o1-preview I was able to get a good answer. The first answer was wrong, but with a little more prompting and clarification I was able to get the answer I wanted to relate the positions of P0, P1, P2 and P3 so that a Bézier curve could be G3. This isn't something that is unknown because there are many CAD programs which can do this already, but I had not been able to find the answer I was looking for in a form that was useful to me.

I don't really know where that puts o1-preview relative to a math grad student, but after spending tons of time over a couple years on this pet project, getting an answer from a chat bot was one of the more magical moments I've had with technology in a long time.

afro88• 2 years ago

The o1 model is hit and miss for me. On one hand it has solved the NYT Connections game [0] each day I've tried it [1]. Other models, including Claude Sonnet 3.5 cannot.

But on the other hand it misses important detail and hallucinates, just like GPT-4o. And can need a lot of hand holding and correction to get to the right answer, so much so that sometimes you wonder if it would have been easier to just do it yourself. Only this time it's worse because you're waiting 20-60 seconds for an answer.

I wonder if what it excels at is just the stuff that I don't need it for. I'm not in classic STEM, I'm in software engineering, and o1 isn't so much better that it justifies the wait time (yet).

One area I haven't explored is using it to plan implementation or architectural changes. I feel like it might be better for this, but need the right problems to throw at it.

[0] https://www.nytimes.com/games/connections

[1] https://chatgpt.com/share/66e40d64-6f70-8004-9fe5-83dd3653a5...

afian• 2 years ago

As a previously "mediocre, but not completely incompetent, graduate student" at a top research university (who's famous advisor was understandably frustrated with him), I consider this a huge win!

nektro• 2 years ago

lol

maxglute• 2 years ago

The experience seemed roughly on par with trying to advise a mediocre, but not completely incompetent, graduate student. However, this was an improvement over previous models, whose capability was closer to an actually incompetent graduate student.

Appreciate the no fucks given categorization of grad students.

oglop• 2 years ago

Ok lol. I mean just feed it serge lang problems and see what it does is all he’s saying.

It performs way better than undergrads. Funny he didn’t point that out but only made some slight to it about being a bad graduate student. Don’t believe me, open the book and ask away. It’s amazing, even if it is a “mediocre graduate student” which is far better than a good graduate student or professor that gives you no help or time for all that money you forked over.

It’s already worth the money, ignore this shitty write up by someone they doesn’t need its help.

alexnewman• 2 years ago

I tried giving it questions like 9.11 > 9.9 and it got basic stuff like that wrong more often than right

lupire4• 2 years ago

GPT-o1, which performs an initial reasoning step before running the LLM.

Is that an accurate description? I thought it just runs the LLM for longer, and multiple times,and truncates the beginning of the output.

lupire4• 2 years ago

The GPT Share links are 404 for me

jameshart• 2 years ago

‘Able to make the same creative mathematical leaps as Terence Tao’ seems like a pretty high bar to be setting for AI.

This is like when you’re being interviewed for a programming job and the interviewer explains some problem to you that it took their team months to figure out, and then they’re disappointed you can’t whiteboard out the solution they came up with in 40 minutes without access to google.

ColinWright3• 2 years ago

My experience of working with people like Terence Tao, and being nowhere near their standard, is that they are looking for any kind of creativity. Everything is accepted, and it doesn't have to be "at their level".

Having read what he's saying there, and with my experience, I think your characterisation is inaccurate.

And having been at the talk he gave for the IMO earlier this year he is impressed with some of the interactions, it's just that he feels that any kind of "creative spark" is still missing.

baq• 2 years ago

I wonder what the creative spark even is in the context of an autoregressive transformer.

Perhaps it’s an ability to confabulate facts into the context window which are not present in the training data but which are, in the context of maths, viable hypotheses? Every LLM can generate bullshit, but maybe we just need the right bullshit?

drzzhan4• 2 years ago

That's interesting. I am young so I don't know what actually is creativity. Could you explain that part for me?

kmacdough• 2 years ago

Right, Terrance was hoping it would have something new to think about it, some new perspective, right or wrong. GPTs have the ability to process insane amounts of information across all branches of math, science and art. This ability eclipses that of the most motivated intellectuals such as Terrance. It is thus a little disappointing that it was unable to find anything in its vast knowledgebase to apply a new lens to the problem.

nybsjytm9• 2 years ago

‘Able to make the same creative mathematical leaps as Terence Tao’ seems like a pretty high bar to be setting for AI.

There's no need to try to infer this kind of high bar, because what he says is actually very specific and concrete: "Here the result was mildly disappointing ... Essentially the model proposed the same strategy that was already identified in the most recent work on the problem (and which I restated in the blog post), but did not offer any creative variants of that strategy." Crucially the blog post in question was part of his input to ChatGPT.

Otherwise, he's been clear that while he anticipates a future where it is more useful, at present he only uses AI/ChatGPT for bibliography formatting and for writing out simple "Hello World" style code. (He is, after all, a mathematician and not a coder.) I've seen various claims online that he's using ChatGPT all the time to help with his research and, beyond the coding usage, that just seems to not be true.

(However, it's fair to say that "able to help Terence Tao with research" is actually a high bar.)

AndrewKemendo2• 2 years ago

This is precisely the first thing I thought

If arguably the person with the highest IQ currently living, is impressed but still not fully satisfied that a computer doesn’t give Nobel prize winning mathematical reasoning I think that’s a massive metric itself

So what then should the first year maths PhD think? I believe Tao obliquely addresses this with his previous post with effectively “o1 is almost as good as a grad student”

nybsjytm9• 2 years ago

If arguably the person with the highest IQ currently living, is impressed but still not fully satisfied that a computer doesn’t give Nobel prize winning mathematical reasoning

No offense, but every part of this characterization is really unserious! He says "the model proposed the same strategy that was already identified in the most recent work on the problem (and which I restated in the blog post), but did not offer any creative variants of that strategy." That's very different from what you're suggesting.

The way you're talking, it sounds like it'd be actually impossible for him to meaningfully say anything negative about AI. Presumably, if he was directly critical of it, it would only be because his standards as the world's smartest genius must simply be too high!

In reality, he's very optimistic about it in the future but doesn't find it useful now except for basic coding and bibliography formatting. It's fascinating to see how this very concrete and easily understood sentiment is routinely warped by the Tao-as-IQ-genius mythos.

faizshah• 2 years ago

This has been observed by more people than just Terence Tao. Try using chatgpt to program something of higher complexity than tutorial code or write a basic blog post, it lacks creativity and the code is poorly designed.

ilrwbwrkhv2• 2 years ago

Even for like basic Rust programs it ties itself into countless borrow checker issues and cannot get out of it, both the OpenAI as well as Sonnet (Anthropic).

It doesn't really get logic still, but it does small edits well when the code is very clear.

I think this will always remain a problem. Because it can never shut up, it keeps making stuff up and "hallucinate" (works normally, just incorrectly) to dig itself further and further into a hole.

Autocomplete on steroids is what peak AI will look like till the time we can crack consciousness and AGI (which the modern versions are nothing even close to).

vavooom• 2 years ago

Most surprising thing about this article is discovering that 'mathstodon' exists and Terence is active on it!

ColinWright3• 2 years ago

We started Mathstodon, an instance of Mastodon-the-Platform, on April 12, 2017.

So we've been up for over 7 years, and have just over 19K accounts.

j_maffe4• 2 years ago

Awesome work! I love what the server has grown into.

j_maffe4• 2 years ago

Mathstodon is one of the most active Mastodon instances. I have no idea how it reached its current level of activity.

giardini• 2 years ago

Anyone tried using GPT in conjunction with Doug Lenat's tools (AM or Eurisko)?

fizx• 2 years ago

I'm curious how well a o1-like model thinks, given minutes instead of seconds and the temperature set relatively high.

bbor2• 2 years ago

Does anyone here think this will change without a full cognitive apparatus? Aka “agents”, to use the modern term? I have my doubts, but I’m relatively uninformed about the cutting edge of pure ML itself.

Just off the top of my head, maybe a RLHF run performed by academic experts and geared towards “creative applications” could get us farther than we are? Given how much the original RLHF run cost with underpaid workers in developing countries that might be exorbitantly expensive, but it’s worth a dream. Perhaps as a governmental or NGO-driven open source initiative…

Of course, a core problem here is defining “creativity” in stringent — or in Chomsky’s words, “scientific” — terms. RLHF dodged that a bit by leaning on the intuitive capabilities of your human critics. I’m constantly opining about how LLMs solved the frame problem, but perhaps it’s better characterized as a partial solution for a relatively easy/basic environment: stories about the real world. The Abstract/Academic/Scientific Frame Problem might be another breakthrough away, yet…

tambourine_man• 2 years ago

Glad to see Tao using Mastodon instead of Twitter.

idunnoman1222• 2 years ago

How did this guy not know how large language models work? Fancy compression algorithm for all written knowledge, how could it invent that which was not an input ?

cwillu2• 2 years ago

https://arxiv.org/abs/1303.2013

https://en.wikipedia.org/wiki/Hutter_Prize

It's not exactly a new conjecture that intelligence fundamentally is an act of compression.

ocular-rockular3• 2 years ago

I don't understand why this is news? This could have been said by any one particular contributor from HackerNews but just because it's from Terence Tao it hits the front page? I understand that the guy is a great mathematician, but why is his input on this any more valuable than the myriad of discussions about o1 from other professionals on here?

sva_• 2 years ago

Because it is an update on his previous post that got discussed here.

j_maffe4• 2 years ago

If anyone comes across comments from professionals on the same level at Terrance Tao, I'd love for them to share it.

ocular-rockular3• 2 years ago

Most people would likely not engage with that commentary simply because they don't enjoy the same celebrity status as Tao. Yet, I think those voices are equally important to be heard. Terence isn't the only PhD/professional using or discussing these tools.

nybsjytm9• 2 years ago

Tao is famous for being the world's most singular and inspiring genius. It's kind of a meme position* that most people accept because they think everyone else accepts it, but for people of a certain inclination, it makes anything he might say into a Pronouncement to read depths into. If you think someone is possibly the greatest mathematician of all time, you'd be interested in everything they think!

(* I think one could very legitimately view him as the top researcher in harmonic analysis in the world - he is a great mathematician - but it's not clear to me how people go from that to Epochal Genius and his extreme celebrity status across STEM)

cwillu2• 2 years ago

Someone at the top of their field discussing the capability of models in their field is much more interesting than someone mediocre in their field making trite observations about capabilities outside their field.

ocular-rockular3• 2 years ago

There is any number of people discussing o1 from the context of their field. So again, why are we valuing one set of discussions above another? Terence Tao may be great, but he's not the end all, be all of commentary. There's plenty of other PhDs talking about this very same thing.

ghransa• 2 years ago

I suspect, but am not certain - that if it had all of formalized mathematics in its context window it could likely extend the edges slightly further. Would be an interesting experiment irregardless.

Terence Tao on O1

Related Stories