‘Machine Scientists’ Distill the Laws of Physics from Raw Data

https://www.quantamagazine.org/machine-scientists-distill-the-laws-of-physics-from-raw-data-20220510/

by hatmatrix • 4 years ago

123 48 4 years ago

quantamagazine.org

hollerith• 4 years ago

What could go wrong?

programmer_dude• 4 years ago

Brilliant! I am inspired to build my own machine scientists.

Maursault2• 4 years ago

It's pretty straight forward. You have to build your own computer scientist, first. Then make them cool. Voila!

ackbar03• 4 years ago

Sounds like 30 year minimum processes, and years 14-18 can be a real pain.

tony_cannistra• 4 years ago

I found it curious that one of the implementations of symbolic regression (the "machine scientist" referenced in the article) is a Python wrapper on Julia: https://github.com/MilesCranmer/PySR

I don't think I've seen a Python wrapper on Julia code before.

amelius4• 4 years ago

Given that Python's ecosystem is enormous, I would be surprised if everything was duplicated in Julia code. It would also be quite wasteful.

krastanov• 4 years ago

But OP is talking about the opposite. Something available only in Julia, that was wrapped for python users.

adgjlsfhk1• 4 years ago

There's also https://github.com/SciML/diffeqpy which wraps Julia diffeq solvers for python.

fennecs2• 4 years ago

Basically approximating a solution, and then finding an equation from some data. Nothing wrong with writing down an approximation to some problem. An approximation can be very useful. As long as you know it’s limitations it’s fine to answer questions where it applies. But this can not replace deeper understanding. It’s all good to write down approximate solutions to some hard n body problem or non linear pde, just don’t look too far ahead in time, or update your model with fresh data if you need to.

riskneutral2• 4 years ago

It’s all good to write down approximate solutions to some hard n body problem or non linear pde, just don’t look too far ahead in time, or update your model with fresh data if you need to.

It sounds like this symbolic regression approach can potentially find correct PDE solutions directly from data:

"In February, they fed their system 30 years’ worth of real positions of the solar system’s planets and moons in the sky. The algorithm skipped Kepler’s laws altogether, directly inferring Newton’s law of gravitation and the masses of the planets and moons to boot."

Note that Newton’s law of gravitation can also be derived from a differential equation. Also note that simply knowing the exact solution to the PDE does not mean that the PDE model itself is an exact representation of reality. For example, we know from Einstein's laws that Newton's laws are not correct at astronomical scales. In the symbolic regression approach, the solution would be limited by the number of different observable variables available to train the model on.

Many PDEs in real world problems also do not have any known exact symbolic solution, so approximate symbolic formulas are used routinely anyway but are just painstakingly discovered and derived by humans with "deeper understanding" (actually, just hoards of PhD students and their advisors throwing everything they can think of at the problem until one of them lands on a unusable formula).

mrow84• 4 years ago

In the symbolic regression approach, the solution would be limited by the number of different observable variables available to train the model on.

What do you mean here? Aren't all approaches limited by the number of observables available (except speculatively)?

potatoman223• 4 years ago

Aren't most physics equations approximate?

fennecs2• 4 years ago

Yes I would say most of physics we currently can make predictions with are just effective models that are only applicable at some certain scale. Certainly there may be some underlying ideas that get shared around but doing calculations and making predictions is effective.

amelius4• 4 years ago

I think the point is that with conventional equations we can make increasingly better approximations by just using pen and paper, while with these ML methods, you need to train your system ab initio if you want to improve your approximations.

mensetmanusman2• 4 years ago

The ones you learn in high school are, but general and special relativity are accurate to trillionths of a percent (their approximations at low speed and without time dilation are the high school equations).

We don’t know if they break down, because we have no means of measuring them more accurately on earth with all the statistical fluctuations happening.

kergonath2• 4 years ago

Yes, they are in absolute terms. To prove that they are exact we would need to have a complete understanding of the universe, which we do not have.

In practical terms, some of them are in the sense that it is impossible to measure their inaccuracy with common equipment and our primitive human senses. In the end, their purpose is to make predictions and help us make sense of what’s around us, they do not need to be exact.

bnewbold• 4 years ago

I'm really curious to see if efforts like this, and from computational augmentation and automation more generally, will yield progress on symbolic modeling of large natural systems.

There is a segment in Adam Curtis's "All Watched Over By Machines of Loving Grace" which describes scientists trying to model the ecology of a prairie. As described by Curtis, the more data they collected and more complex their model, the worse the predictions of the model became. It feels like the lessons of the past couple generations (many decades) have been that symbolic models don't compose together easily; symbolic analysis only works in simple systems and has hit diminishing returns; numeric and ML methods work well enough; etc. I'm curious if better tooling (augmentation and/or automation) can push through some of these challenges and yield real understanding. Or if there are some fundamental scaling problems that can not be overcome.

ThePhysicist• 4 years ago

I worked on genetic programming for my MSc in econometrics, used it to build human-readable risk models for time series data. It worked really well. The article seems to call these methods genetic algorithms, though Koza himself (the "godfather" of genetic algorithms & genetic programming) was pretty keen on distinguishing the two. The main difference is that genetic algorithms represent solutions as linear strings / vectors whereas genetic programming works on tree structures. Maybe just splitting hairs here though.

pmoriarty3• 4 years ago

"The main difference is that genetic algorithms represent solutions as linear strings / vectors whereas genetic programming works on tree structures."

Yes, that and that the tree data structure used by genetic programming tends to have functions in every non-leaf node, and static values or variables in the leaf nodes.

optimalsolver2• 4 years ago

It should also be emphasized that genetic programming is just one approach to program synthesis, i.e. automatically deriving computer programs from data.

You don't have to use genetic/evolutionary algorithms to search the space of programs, it's just the most popular method.

You can even try pure random search if you're feeling particularly lucky:

https://github.com/pySRURGS/pySRURGS

nyc111• 4 years ago

A constant implied that it had identified two proportional quantities — in this case, period squared and distance cubed. In other words, it stopped when it found an equation.

Very interesting article. Thanks for posting.

I've been thinking about the physics equation and its relation to proportionality. Not all physics equations are proportionalities because in physics the equality sign is loaded.

To me, proportionality is fundamental not the physics equation. So I would have written the last sentence of the quote as "...it stopped when it found [a proportionality].

lagrange77• 4 years ago

Also have a look at SINDy (Sparse Identification of Nonlinear Dynamics) by Steve Brunton et al.

https://youtu.be/NxAn0oglMVw

amelius4• 4 years ago

Do they apply Occam's razor, where simpler equations are favored over more complicated ones?

7thaccount• 4 years ago

Yes. The article states something about statistics and Bayesian and other terms I only barely understand. It uses "+" over "hyperbolic cosine" for example (to quote the article).

franknstein• 4 years ago

"One might define simplicity as the length of the equation, say, and accuracy as how close the curve gets to each point in the data set, but those are just two definitions from a smorgasbord of options."

"..the algorithm evaluated candidate equations in terms of how well they could compress a data set. A random smattering of points, for example, can’t be compressed at all; you need to know the position of every dot. But if 1,000 dots fall along a straight line, they can be compressed into just two numbers (the line’s slope and height). The degree of compression, the couple found, gave a unique and unassailable way to compare candidate equations."

mensetmanusman2• 4 years ago

“ These algorithms resemble supercharged versions of Excel’s curve-fitting function, except they look not just for lines or parabolas to fit a set of data points, but billions of formulas of all sorts. In this way, the machine scientist could give the humans insight into why cells divide, whereas a neural network could only predict when they do.”

It’s very cool witnessing the consequences of Moore’s law; as the cost of billions of calculations goes to pennies, this becomes easier and easier.

Since the complexity of nature is unbounded, I wonder if the slow down of Moore’s law will represent a plateau of what can be symbolically discovered.

For example, will the next set of scientistic discoveries require quintillions of combinatorial checks that can’t be accomplished in a human lifetime?

jas-• 4 years ago

Wouldn't this be a "what we know about physics" distillation? I am basing this assumption on the "Raw data" as being that of already performed experimentation etc.

plaguepilled• 4 years ago

For those interested, this is the relevant article (I think): https://www.science.org/doi/10.1126/sciadv.aav6971 - I'm partway through it now and am having a blast.

eggy3• 4 years ago

I am hoping this also opens up more opportunity to leverage Lisp's symbolic powers. I had great fun with Structure and Interpretation of Classical Mechanics (SICM), and recently a paper on analyzing music using symbolic regression [0], and another on symbolic regression with Lisp [1]. Julia and CL seem perfect for this with Mathematica as another option for quickly playing with ideas.

[0] https://www.researchgate.net/publication/286905402_Symbolic_...

[1] https://towardsdatascience.com/symbolic-regression-the-forgo...

ChrisRackauckas3• 4 years ago

Yeah, Julia does great in this domain. DataDrivenDiffEq.jl (https://datadriven.sciml.ai/) is a comprehensive symbolic regression package out there, which pulls together all of what's mentioned in the article (SINDy and variants, EQSearch, SymbolicRegression.jl (which is the core of PySR), OccamNet, etc.) all into a single API to allow comparisons between each of the methods.

The Julia automated discovery of physical equations work showcases these symbolic regression techniques quite heavily, for example in the State of SciML talk (https://www.youtube.com/watch?v=eSeY4K4bITI) we discuss how this has been used now across hundreds of different scientific use cases. The universal differential equations paper which describes the Julia SciML organization (https://arxiv.org/abs/2001.04385) demonstrates these cases where neural networks in differential equations are mixed with symbolic regression to allow for mixing prior physical knowledge with data for discovering just the unknown higher order physical equations. And there's a lot more directions we're going next.

eggy3• 4 years ago

Chris, the work of you and others on Julia is nothing short of amazing. Thanks! I tried Julia when it first came out, and then picked it up again to replace my MATLAB habit, and I have not looked back. I can't wait until the sims.jl package and others can be used like Simulink/Modelica.

pmoriarty3• 4 years ago

"They started by downloading all the equations in Wikipedia. They then statistically analyzed those equations to see what types are most common. This allowed them to ensure that the algorithm’s initial guesses would be straightforward — making it more likely to try out a plus sign than a hyperbolic cosine, for instance. The algorithm then generated variations of the equations using a random sampling method that is mathematically proven to explore every nook and cranny in the mathematical landscape."

Wikipedia contains equations from many different branches of math.

Aren't mathematical equations written in all sorts of notations (really obscure notations sometimes, for obscure branches of math), depending on the branch of math they're used in?

How would this program even be able to use equations written in a notation it doesn't understand? Even if it somehow understood the notation, that doesn't mean the algorithm understood the branch of math the equation was for.

This all sounds unworkable to me, unless they're somehow limiting the equations they use to some branch of math they already understand.

nh23423fefe• 4 years ago

If you imagine it's nonsense, then you probably don't understand it. You deduced a false conclusion from assumptions you've inserted.

pmoriarty3• 4 years ago

Could you elaborate on this? What assumptions do you object to?

Are you saying that different branches of math don't have their own special notations?

Sure, they have notation in common, but they have their own notation too, to express objects, relations, or operations that are of special relevance to them.

That's not to mention that any given symbol could mean different things depending on the branch of math it's used in.

kloch• 4 years ago

This article seems to be about ML but good old fashioned brute force searches are also possible.

It's trivial to find formulas that fit data to within the variable's uncertainties. You can easily drown in such output. There are two main challenges with this approach: 1. make sure you have the right "ingredients" (factors) for the search, and 2. filter the output so that human reviewers don't waste time on obvious nonsense equations.

Filtering output is not terribly difficult. Some assessment of complexity is required and depending on the formula a symmetry score might be useful. Some actual physics equations are quite complex so you have to be expecting a simple equation or filtering output will be less effective.

Making sure you have all of the right potential factors available is very hard. Expanding the input factor set extends the search space and time.

ChrisRackauckas3• 4 years ago

Brute force costs grow exponentially (well, combinatorically) with the number of terms. It's viable for the small cases, but not for the larger cases. There's a nice paper from 2016 which demonstrates this very directly (http://www.cogsys.org/proceedings/2016/paper-2016-2.pdf), though I'm sure there's some stuff from the 70's on it as well.

YeGoblynQueenne• 4 years ago

Aw, no mention of the two Robot Scientists, Adam and Eve, who don't just invent theories but also design and run their own experiments to verify their theories?

Wikipedia page:

https://en.wikipedia.org/wiki/Robot_Scientist

Nature publication:

https://www.nature.com/articles/nature02236

I guess US media don't realise there's science and technology outside of the Americas eh?

compacct27• 4 years ago

The people in the article worked in Catalan, so I guess maybe they do.

djdsol• 4 years ago

This is also how a few hedge funds ( WorldQuant and spinoffs ) get alpha signals/factors for their long/short portfolios. They get 95% of their aggregate alpha/returns from Genetic Programming and not human quant researchers.

‘Machine Scientists’ Distill the Laws of Physics from Raw Data

Related Stories