ChatGPT spitballing fundamental physics theories

https://www.ramrajv.com/blog/chatgpt-spitballing-fundamental-physics-theories/

by ramraj07 • 3 years ago

73 67 3 years ago

ramraj078• 3 years ago

I’m not a physicist myself but interested in it, and this was one of the topics I thought I’ll coax the bot into imagining. The theories might not be solid (I wouldn’t know myself), but I’m excited about the bots future as a companion for every day work in science.

I’m a biologist by training and have already made significant progress on multiple difficult questions I’ve had trouble with for decades in the 2 days I’ve had gpt4. Can’t wait to see what all we could accomplish with it!

ChaitanyaSai2• 3 years ago

"made significant progress on multiple difficult questions I’ve had trouble with for decades in the 2 days I’ve had gpt4" -- that is wildly impressive. Can you share some specific questions. Thanks!

ramraj078• 3 years ago

I have had hypotheses regarding Epigenetic modulation of mutation rates playing a role in “epigenetically directed evolution;” this is not my primary field and honestly not a topic anyone has properly focused on. I’ve even corresponded with a couple of experts in related fields but didn’t make progress. Chatting with the bot I discovered scientists who might have actually generated some data on this topic a bit inadvertently that I never discovered merely by googling. I’ve made a list and am going through the literature already. I can’t imagine what else I can do with this tool if I can drag and drop pdfs for it to get more context on such arcane topics.

l33tbro2• 3 years ago

It's certainly handy for summarising the existing corpus that we are unfamiliar with (though take acuracy with a grain of salt).

Proposing new theories I'd be extremely skeptical of. It's just not what this thing is made to do.

ramraj078• 3 years ago

It’s not gonna spew completely solid hypotheses but I can see how these ideas can be amazing seeds for us to pursue further. Importantly I asked it to think like Einstein. In exchanges I didn’t put here, it first just said Einstein would have worked on string theory if he were alive today. I reminded it that einsteins unique ability was to imaginatively think of completely new ideas. It then spewed these two topics. It’s like asking it to write songs in a particular poets style but for science. That sounds amazing to me.

Importantly it clearly has some understanding of these theories. At least as much as any regular person would, in my opinion.

l33tbro2• 3 years ago

It's kind of weird. This technology is designed for the human to give the machine a prompt. What I like about your approach is that you're almost using it to create the inverse: the machine supplying a slightly fuzzy idea that could prompt you to take your research in new and unexpected directions.

nico2• 3 years ago

Love it.

A few days ago I did a similar exercise playing with alternative definitions of gravity.

It was even able to provide (GlowScript 3.1!) code to run a simulation using the concepts we developed (and it worked).

It’s amazing the feeling of going back and forth, iterating quickly on the ideas and then being able to generate code to test them.

ramraj078• 3 years ago

Was it with gpt3x or 4? In these exercises 4 is miles ahead now, I’ll highly recommend ponying the 20 bucks and playing with it to see for yourself!

nico2• 3 years ago

It was with gpt3. Already tried a couple of previous convos with gpt4 and noticed the difference.

Will have to try the gravity one now.

Thanks for the recommendation!

poglet• 3 years ago

Are you referring to ChatGPT Plus?

BiteCode_dev• 3 years ago

Can confirm. This week a physicist friend came by to give it a try. He asked some questions related to his domain. I was fully expecting GPT to bullshit its way through it, from experience with GPT 3. But to my surprise, and to his, it got quite decent answers.

tokamak-teapot• 3 years ago

Any chance of a peek at the demo code and/or transcript please? Sounds fun!

RandomLensman• 3 years ago

Is there a way to get it to babble less, i.e., use more compact and to the point language?

szszrk• 3 years ago

I used to simply ask it to make it brief, provide a summary or contain answer in N words and in my language it complied happily.

ramraj078• 3 years ago

I have done that but it tends to actually omit some facts as well. This convo is after I asked it to stop adding disclaimers as you can see. I take it as just a quirk of a colleague and don’t mind it at this point lol.

marviel2• 3 years ago

had the same issue with the "TenetLang" chat thread -- will have to keep that in mind.

Jack000• 3 years ago

The first theory reminds me of Sarumpaet rules from Schild’s Ladder.

marviel2• 3 years ago

very cool.

supermatt• 3 years ago

I have been using ChatGPT as a Lithuanian language tutor.

It is surprisingly amazing - accurately (verified with a native speaker) conversing in Lithuanian with phrase level translations, explaining pronunciation, explaining clauses and cases, and is able to explain some grammatical concepts (usage of commas, etc) that I haven't found explained in English elsewhere.

ChatGTP• 3 years ago

I asked it to translate my resume recently, saw the word Python and or Linux stopped translating and stared opening a fake terminal and doing weird stuff, so YMMV.

My honest opinion is that very few people are clear on what the thing is actual good at, great at or bad at at this stage. We're still trying to find it's real use case.

IMO it's best for things you already know quite a bit about so at least you don't mess up hard. In other words, it seems like a more advanced ELIZA. Nearly everyone who has had a good time with it has used talk to it until they had a break through.

euroderf• 3 years ago

Works for Interlingua too. A conlang with no native speakers (AFAIK). A great use of its capabilities.

arpowers• 3 years ago

I did this myself a few weeks ago,

while I don’t want to sound crazy…

I managed to get GPT to mathematically derive a formula and calculate actual results to explain some well known anomalies in physics.

I’m aware that gpt might be imagining and contriving false conclusions …

but I have no doubt that ai will be moving physics forward within the decade

_nalply• 3 years ago

Perhaps.

Generative AI needs a way to receive feedback. AI learned to play chess by playing a lot, and the feedback was either winning, losing, or drawing.

If generative AI can repeatedly test physics theories faster than humans, then we may witness progress in physics. AI could generate thousands of theories and conduct experiments successfully, possibly leading to new physics models.

However, I am uncertain whether this will be achievable soon, particularly for theories requiring costly experiments.

A benefit could be that humans crave recognition for success, making papers less valuable. AI may be more willing to take risks and document its failures than humans.

JProthero2• 3 years ago

If generative AI can repeatedly test physics theories faster than humans, then we may witness progress in physics. AI could generate thousands of theories and conduct experiments successfully, possibly leading to new physics models. However, I am uncertain whether this will be achievable soon, particularly for theories requiring costly experiments.

I've long felt that this may be the strongest argument against an AI singularity.

The technical ability to emulate the minds of the world's theoretical physicists and run accelerated simulations of their thought processes may be developed, but the generation of valid new insights in physics might depend strongly on observations and experiments conducted in the physical world, as seems to have been the case historically, and the virtual equivalents of those experiments may prove to be inadequate or impractical to implement.

Steven Pinker made a similar argument in this 2018 discussion with Sam Harris (the remarks begin at 65m03s in this recording [1]; the full conext begins at around 50m36s [2]). Harris is concerned about existential risks posed by advances in artificial intelligence, whereas Pinker is less so, in part for this reason. I agree with Harris that there are risks associated with artificial general intelligence, but I agree with Pinker and the parent comment about the dependence of the scientific process on experiment, and that an inability to conduct accelerated experiments in the physical world may undermine the standard argument about the inevitability of an AI singularity.

An AI capable of interfacing with the physical world might develop the ability to conduct accelerated physical experiments, but it would presumably face the same fundamental and contingent limits as human researchers, and the history of human science suggests those limits may impede exponential progress.

[1] https://www.youtube.com/watch?v=hofkL0RJJtM&t=3903s

[2] https://www.youtube.com/watch?v=hofkL0RJJtM&t=3036s

ShamelessC• 3 years ago

while I don’t want to sound crazy…

Doesn't the author do something similar in the article? Why are you worried people will think you are crazy?

tom-from-july• 3 years ago

I love this so much. Like, most of the concepts and ideas it refers to are not completely new ideas (don’t ask me were I heard about them), but it’s interesting to hear them not proposed as a solution to a specific problem, but just based on the fact that they are thematically close to the works of an existing scientist. A little creepy though, like research from beyond the grave (move rights pending).

Non the less interesting though. I am currently researching into physics-informed neural networks for quantum problems, which are guided with differential equations. Maybe if you’d extend an LLM with an execution engine/numerical model, it’d be able to actually produce differential equations to undermine its hallucinated theories. From my short testing that is something ChatGPT (4) is not that good at. Or it’s just a generally a hard problem to produce novel differential equations

peignoir• 3 years ago

Funny I asked it questions about general relativity as Einstein too :)

raydiatian2• 3 years ago

I don’t know how much people listen to Sabine Hossenfelder’s opinions on things, but the article mentioned the age old desire for unification and symmetry, and frankly, I have to point out how aestheticist, unscientific, biased, and frankly typical it is for an LLM trained on modern (theoretical physics) conversations to go in search of unification. I know AI isn’t doing new science any time soon (or even this decade) but what I do think is that it’s probably dumb to bias your AI to go hunting for grand unified theories when what it should be looking for is unanswered questions grounded in first principles and proposing candidate experiments to discover the shape of data behind those doors, AND ONLY THEN proposing theories.

The modern mechanics revolutions of the 20th century demonstrated that our intuitions were completely mismatched for what the truth was. Planck himself rejected the notion of curve fitting which ultimately birthed energy quanta. It’s OUR macro universe that is the weird one, a strange corner case of quantum reality in the absurdly large.

whatshisface4• 3 years ago

This is really awful, the text is completely content-free but it's got everyone who's not a domain expert hooked.

I didn't know whether ChatGPT's ability to babble convincingly was going to cause trouble or be a funny quirk that we all knew about and worked around, but this thread is really making it look like the pessimists were right about it. The problem is that it gets past many people's filters because its phrasing sends a lot of intelligent/rational/professional signals, as it was engineered to do. Nobody is used to picking smart-sounding text apart word by word to make sure they agree with it, except maybe academics, and that's the vulnerability it reaches us through.

I also think that OpenAI got human nature backwards when they trained it to hedge on everything it said - everybody knows that people who constantly demure are the most reliable! A safe chatbot would sound pushy, like a bad salesman or an ideological agent; like something incapable of self-questioning.

yieldcrv• 3 years ago

Lots of cults and belief systems operate this way

Like that time a science fiction writer started his own religion based on characters from his novel and got a bunch of celebrities financially tied to it

Or when holistic practitioners procedurally generate an unsubstantiated preventative cure

Might as well just turn your brain off now, you won’t need it

Raised by Wolves moment

renewiltord• 3 years ago

Haha that's for now. But humans do this too, attempting to copy the shell around a concept: the way it is expressed.

In American English, this is frequently authority speech mimicking law enforcement, scientists, or lawyers. All while lacking any knowledge of anything but the outer dressings of something.

In this way, the problem is that the AI is just mimicking most of us on most things.

dr_dshiv3• 3 years ago

the text is completely content-free

Since the same could be said about your comment, perhaps you could amend your critique?

dr_dshiv3• 3 years ago

Since I got so many downvotes, let me share the specific claims:

“Dynamic Geometric Interactions in Multi-Layered Spacetime"

This hypothesis proposes that spacetime consists of multiple interconnected layers, each with its unique geometry and properties. The fundamental forces and particles emerge from the interactions between these layers and the geometric transformations that take place within and between them.

Multi-Layered Spacetime: The universe is not a single, continuous spacetime fabric but instead is composed of multiple interconnected layers. Each layer has its own geometric structure, which can be influenced by the other layers.

Interactions and Transformations: Particles and forces emerge from the interactions between the layers and the geometric transformations that occur within and between them. These transformations might involve changes in curvature, connectivity, or other geometric properties, leading to the observed behavior of particles and forces at different scales.

Unification: At high energy scales or specific conditions, the interactions and transformations between the layers might become more unified, leading to a single overarching interaction responsible for all fundamental forces.

[And a rough mathematical explanation] Let's denote the granular spacetime structures as Sij, where i and j represent the indices for the type of spacetime structure and its configuration, respectively. Suppose there are N types of spacetime structures, each corresponding to a specific force or force pattern. The interaction between particles might be described by an interaction matrix I, where each element Iij quantifies the strength of the interaction between spacetime structures Si and Sj. In a high-energy regime, the spacetime structures' patterns might begin to merge, leading to the unification of forces. We can represent this by introducing an energy-dependent matrix U(E), which modifies the interaction matrix I as a function of energy E:

I'(E) = U(E) * I

As the energy E approaches a critical energy level Ec, the matrix U(E) transforms I into a single unified interaction matrix, corresponding to the unified force.

- - -

This comports with my understanding of the Inflaton field, which parametrically resonates (through geometric relationships?) with other fundamental fields during the Big Bang. I’ll pull some references here.

ramraj078• 3 years ago

I (OP) am not a physicist but still a scientist. I’m not gonna say the theories it said are solid. It’s quite clear to me that they’re not. But they are better theories than what I (who knows the fundamentals of physics for the most part) can come up with. So it’s not a subject matter expert but better than the average joe. For what it is, (and given I asked it to imagine theories), its output looks impressive to me. And I can attest to its abilities to do this in biology as well, and I’ll say it does a better job than most biology professors in fact. I just didn’t publish those chats here.

Maybe it did a better job in biology because I was able to correct it at an expert level (which I was not able to here). Given the demonstrably (like right here in these comments) myopic unimaginative nature of physics as a science today, it’s possible not a single physicist would try to entertain this system as a hypothesis generation machine. I mean we have discovered everything already right?

whatshisface4• 3 years ago

I think you asked it some of the right questions, but even then it was able to slip past your skepticism by being really good at sounding like an intelligent human being that was saving you the trouble of the details. There's no hypothesis in there.

You would have to be an insanely skeptical person, one who would drive anybody nuts to talk to, to approach a ChatGPT session in a field you're not an expert in or maybe even one you are, and evaluate it right... The only normal human perspective that fits what ChatGPT is actually like is the one we take on people we think are terrible, so that is why I say it's awful, although it's just a machine.

logifail2• 3 years ago

a hypothesis generation machine

Is anyone claiming there's a market niche for hypothesis generation in the natural sciences?

Full disclosure: I have a science PhD and a couple of published papers as a result. It was a long, slow, frustrating grind, but generating hypotheses wasn't even close to being the hard bit.

simplotek2• 3 years ago

The problem is that it gets past many people's filters because its phrasing sends a lot of intelligent/rational/professional signals, as it was engineered to do. Nobody is used to picking smart-sounding text apart word by word to make sure they agree with it, except maybe academics, and that's the vulnerability it reaches us through.

I think you're making a mountain out of a molehill. To me ChatGPT is basically a clever way to interpolate and extrapolate coherent text based on user input and its training set. If the training set is lacking in some areas, it underfits the output.

I've tested ChatGPT in a couple of engineering fields I'm familiar and I expected the service to respond poorly, but even though it returned nonsense in some areas I thought were low-hanging topics, such as the release year of an international standard, overall its output was very impressive and very entertaining.

Perhaps it's the engineer in me talking, but it's pointless to waste time waxing lirically about human nature. Tools like ChatGPT might one day be superb expert systems and teaching tools, but like any expert system and learning tool you need to corroborate the results by yourself. Human nature has zero to do with this.

whatshisface4• 3 years ago

That description of how it functions is true, but it did fool a lot of people, and now we have to explain why. I also note that a lot of people report positive results in their own fields of expertise, which might be part of the explanation. (It might be building trust before failing inexplicably right when it can no longer be checked.)

i_cannot_hack• 3 years ago

I'm not a domain expert either, but to me the results seem to be entirety what you would expect from a model trained to pattern match very well on existing content.

That is, the ideas, strengths and limitations of the current well-known theories are very well explained and mostly correct. However, the "novel theory" is mostly filler words around some very thin concepts to make it sound like an actual theory with some depth to it, but in reality saying nothing more than that spacetime is granular and there could maybe be some matrices.

00F_• 3 years ago

demur, not demure

raydiatian2• 3 years ago

I think the future will see academia as the last bastion for human ingenuity, before ultimately ML swallows that up too, or climate change swallows the whole thing up.

0xmarcin2• 3 years ago

Not sure about physics but I am really looking forward to see games with NPC characters powered by ChatGPT. That would make some game world really deep and involving.

kfrzcode• 3 years ago

Working on it.

Karrot_Kream• 3 years ago

Ask it to play a MUD with you. You may need to specify the system prompt. It works right now on GPT-3.5 just fine. Try it, I'm playing one currently.

otabdeveloper42• 3 years ago

ChatGPT lowers information complexity by smoothing and averaging the information content, so no, likely you'll see the opposite effect.

We already went through this with Stable Diffusion - the content it produces looks very professional, but also somehow exactly the same no matter the subject.

0xmarcin2• 3 years ago

This assumes that you will have only a single model. I expected at least one model per race (elves, dwarfs...) x adversity (friend, enemy, neutral) x some psychological models (e.g. Jung personality archetypes). So with ~100 GTP models learned on different sets you may have a quite diverse army of characters.

ChatGPT also offers a possibility to e.g. have a real bargaining at the shops: "How much is that sword... ".

manuelmoreale• 3 years ago

I’m not the only one who thought about this! I had this exact thought a couple of weeks ago. It would make open world games a lot more entertaining.

Thank you for this comment, it really made me happy.

lambo4bkfast• 3 years ago

Starfield has been rumored to have this.

mola• 3 years ago

I spent 20minutes being gaslighted by it trying to convince me some very obvious false logical statement in set theory is true. It wouldn't for a second admit any doubts and would just say it's sorry I'm confused and that it'll explain it again more simply.

This thing is not good for developing any useful theories other than a creative crutch.

yread• 3 years ago

It presented me with some weird facts and when I asked for a reference it hallucinated one from 1988. The article name returned 0 results on Google. I went to the journal, volume, issue and page and found out it was in the middle of another article. When I asked GPT what is on that page in that vol. and issue it replied with the article's title again! When asked what is 3 pages earlier (where the actual article starts) it apologized and said it doesn't actually know...

nomercy400• 3 years ago

I know some physics, but not enough in this area to judge if the results here are accurate. With the inaccuracy of AI chatbots, this makes this tool not useful for me for learning new concepts, as they might be blatantly wrong.

zoover2020• 3 years ago

Not only not useful, I'd go as far as extremely dangerous as most people _not_ in niche fields can't distinguish facts from fiction

elashri• 3 years ago

I tried GPT-4 after the launch by a couple of hours and it tried so hard to sell me the idea that terms on the two sides of an equation can be of different dimensions.

danbmil99• 3 years ago

This is actually a much better version of the techno-babble we see all the time in movies and TV. I assume they get it wrong because the story is more important, and tech ppl don't want to be associated with the bs that comes out of the actor's mouths.

A great gig for GPT -- Hollywood bullshitter!

ofrzeta• 3 years ago

So, like Postmodernism text generator but with physics?

https://www.elsewhere.org/pomo/

https://en.wikipedia.org/wiki/Postmodernism_Generator

ChatGPT spitballing fundamental physics theories

Related Stories