Roboticists discover alternative physics

https://phys.org/news/2022-07-roboticists-alternative-physics.html

by doener • 4 years ago

184 111 4 years ago

Ugh... Sorry but this is not physics.

thriftwy2• 4 years ago

Why not? It will surely have the same bounding box as our own physics, but the laws may be represented in completely different fashion. They would mostly map to each other, though.

Or maybe not. Maybe the "AI physics" has predictions for phenomena which our physics never bothered to figure out, whereas it would ignore large chunks of our physics area as irrelevant.

aaaaaaaaaaab2• 4 years ago

This is more like the empirical "laws" that various engineering disciplines use. I.e. some number crunching that has no theoretical foundations whatsoever, but seems to be working anyway.

Daub2• 4 years ago

Agreed, if only for the reason that the AI seems to be making observations of the physical world based on videos thereof.

samloveshummus10• 4 years ago

I don't see why that's relevant, a video camera is just another instrument that records data, not essentially different from the detectors at the LHC, albeit completely un-optimized - which is necessary for this experiment to work.

Shorel• 4 years ago

Apart from some very slow atom and particle simulations...

Is not most physics based on very extreme oversimplifications?

They are still very useful because errors average out, but for example: we don't know why fluid simulations work, in fact aerodynamics is an entire empirical field.

This was seen in F1 this year. All cars bouncing on straights, it was even given the name porpoising by the pilots. The teams never expected it, because the simulations never discovered anything about it.

Also: all the work of AlphaFold predicting protein structures.

Physics are awesome at predicting planetary motion, but stuff at smaller scales require a lot of empirical engineering to figure out.

JPLeRouzic• 4 years ago

but stuff at smaller scales require a lot of empirical engineering to figure out.

And that made the fortune of CAD software designers.

mhh__4• 4 years ago

The porpoising is apparently because the more expensive higher speed wind tunnel analysis is banned.

Engineering needs engineering,. Physics does not. The interesting thing about aero in physics is the emergence of extremely complicated phenomena, not the equations as per se.

samloveshummus10• 4 years ago

I'm not sure why you don't think it's Physics. It's about formulating laws that describe the behaviour of physical systems - that's the essence of what Physics is. I have a PhD in high energy theory and this really seems like Physics to me.

mjburgess3• 4 years ago

Sure, if the pseudoscientific description of factor analysis applied to images is correct, then it's phyiscs.

As-is, it's pseudoscience. What happens when you do a factor analysis on images? You get some measure of the axes of geometrical variance across those images.

Are those axes "related" to any physical variables, sure -- but almost never directly. To suppose the system itself had these properties is to suppose, for example, constellations actually exist and cause your personality traits.

Everything we want to know is what phyical properties of the system give rise to the observed consistent correlations in geometrical properties. *THAT* is physics.

Showing these geometrical properties exist and are consistent is just what we're trying to explain.

You cannot go from images to the domain of physics -- there are an infinite number of theories consistent with these images domains. And this is pseudoscience.

YeGoblynQueenne14• 4 years ago

> It's about formulating laws that describe the behaviour of physical systems - that's the essence of what Physics is.

I didn't see any attempt to formulate laws. The researchers trained a neural net model to predict the next event in a sequence. That is not a natural law, it's a maximum probability estimator.

To clarify, a natural law would be a formula with variables that one can plug in numbers to, in order to predict the behaviour of a system. For example, Newton's law of gravitation is a natural law, Kepler's laws of planetary motion are natural laws, the laws of thermondynamics are natural laws. But a neural net model trained to predict the next frame in a video? How is that a "law"?

mhh__4• 4 years ago

The difference is parsimony

ndsipa_pomu• 4 years ago

I'm not seeing the significance of that. Surely there's plenty of alternate descriptions of physics/systems that can be equally predictive and certainly there's lots of equivalences to shift from one framework to another.

samloveshummus10• 4 years ago

It's true that there are many mathematically equivalent ways to describe physical systems. But the important point is that some are more useful than others. For example, Lagrangian mechanics and Hamiltonian mechanics are equivalent to Newtonian mechanics, but they can give much better intuition for certain problems. Feynman diagrams are equivalent to grinding out the QFT algebra by hand à la Schwinger, but they give a completely different intuition for the underlying Physics.

More importantly, though, they could use this NN on systems that have not yet successfully been modeled, perhaps complex dynamical systems, to discover good parameters and conserved quantities.

MauranKilom• 4 years ago

More importantly, though, they could use this NN on systems that have not yet successfully been modeled, perhaps complex dynamical systems, to discover good parameters and conserved quantities.

That would only make sense to try if the model could do this for systems we already understand. By the sound of the article, it can't even do that. Despite many efforts the researchers couldn't even understand the second pair of parameters. That doesn't correspond to my understanding of "good parameters".

sbf5012• 4 years ago

For example, Lagrangian mechanics and Hamiltonian mechanics are equivalent to Newtonian mechanics, but they can give much better intuition for certain problems. Feynman diagrams are equivalent to grinding out the QFT algebra by hand à la Schwinger, but they give a completely different intuition for the underlying Physics.

I just read about Langrangian and Hamiltonian mechanics. I didn't encounter those at all in my EE physics, and they are fascinating. Great examples! Are you a physics professor, or is this stuff undergrad physics majors learn?

jack_riminton• 4 years ago

Aren't they just placeholders that help to predict a physical outcome rather than objective measurements?

radarsat12• 4 years ago

This reminded me a great deal of a paper I read like 10 years ago, which I just looked up because I wanted to see if it was cited here, "Distilling Free-Form Natural Laws from Experimental Data" [0]. Lo and behold it is not only cited, but is written by the same authors. So the novelty here, as discussed in the paper [1], is that they are doing something similar but from video instead of from sensor streams. Which is quite interesting, as it opens up the available information to systems that are hard to 'sense' apart from pointing a camera at them, like the lava lamp example.

I did see a presentation a few years ago on a similar topic by Erwin Couwans, author of Bullet physics engine, who was discussing doing neural inference of physics. He basically wanted to replace Bullet with a neural network, which I thought was kind of funny at the time, but cool if it worked. (Physical laws at this level of mechanics are mostly quite well understood, but the devil is in the details and actually solving time-discretized non-smooth dynamics, as well as performing good contact detection is less than obvious the more you get into it, not to mention friction, so I could see why a "learned" solution could be attractive, but at the time I found it funny that he was proposing to learn the physics that were already simulated by Bullet.) Looking back, it appears that those ideas culminated in a paper [2] two years ago, which I'll have to read now -- from the abstract, the differentiability of the simulation has benefits not only for learning the physics (mostly friction models it appears), but for learning controllers for plants within the simulated physics. It seems to be not cited in the linked article but maybe it is a slightly different, but related, topic. In any case, fascinating stuff.

[0] https://www.science.org/doi/10.1126/science.1165893

[1] https://arxiv.org/abs/2112.10755

[2] https://arxiv.org/abs/2011.04217

ok_dad2• 4 years ago

That’s interesting thanks for sharing. I always was bearish on neural networks when advertised as “AI”, but now I think that perhaps when used as a tool rather than a panacea they could be very useful. Humans learn to operate heavy machinery in industrial settings through experience and use via our own neural networks, and we have incomplete understanding of physics, so why not use a neural network to learn the physically best way to drive an industrial machine and use the physics as backup for safety? Seems to work well based on all these stories so far, so maybe learning neural networks are the next differential equations for engineering.

radarsat12• 4 years ago

The trick is to remember that a neural network is a function approximator. A good deal of AI research is in the business of casting "intelligence" as a "function" so that you can pose a problem and figure out how to feed it data, ie., well-defined input and output. That's why different function approximators can be used for AI, such as decision trees, etc., not just neural networks. What to model, and how to model it, are orthogonal problems, both interesting in their own right. It just happens that NN are particularly good at handling high-dimensional inputs, which are necessary for perception tasks, as well as for handling large vocabulary language modeling, so they are doing rather well lately.

On the other hand, there are lots of places where "functions" are useful, that have nothing to do with "intelligence", but where actually writing the function with full fidelity can be difficult or intractable. These are also opportunities where learnable function approximation can provide some great benefit, provided you can figure out how to pose the problem such that data is available for the learning part.

A good example is in physically-based rendering, you have lots of complicated aspects of light transport for which we can model the physics quite well, but when you get into complicated reflections and scattering, etc., this can all be modeled by a complicated function called the BRDF [0]. Hand-writing a good BRDF is possible, and quite typical of high-end renderers in fact, but it's no surprise that there's been research in replacing it with a "neural BRDF" [e.g. 1].

That's just to give an example of a place where a single, very targeted and small (but complicated) part of a larger framework can stand to benefit from data-driven modeling, and a neural network can be one good way to do that. Another example is similar usage in computation fluid dynamics [2], where we can hand write a pretty good model, but to capture what is missing, it can be useful to have an approximator. The problem there is that having approximated the function well, it doesn't necessarily lead to better human understanding of the phenomenon. Discovering the true, sparse latent variables in a way that is interpretable, instead of a black box, is a useful step towards that. (Which I guess the current article is aiming at, but I haven't read it in full yet.) But sometimes all you want is results, as in the case of synthesizing a good controller. For example in CFD, if you can use the blackbox model to generate a good stabilizer for an ocean platform, you don't really care about the physics, as long as it's accurate enough to be trustworthy. So the utility of these methods, like most things, is relative to what your goals are.

[0] https://pbr-book.org/3ed-2018/Color_and_Radiometry/Surface_R...

[1] https://arxiv.org/abs/2111.03797

[2] https://github.com/loliverhennigh/Computational-Fluid-Dynami...

qikInNdOutReply2• 4 years ago

Out of interest, how would such a system derive material properties from a still 3d scene?

We know the leaves of a house plant will bow in the wind, because you have observed it in the wild, as a child several times. Even given non-still image training material, the nn would have to learn all of the physical properties of all the things from scratch, like a human.

To train a physics model like this to derive the properties of material from the world is going to be bound to hilarious failures. A mountain with smoke hanging over it, is obviously a house plant.

digdugdirk• 4 years ago

I'd imagine you could do a frame by frame to identify points of interest and then "predict" the track they should follow into the next frame, training the model on each point of interest as you go.

In this case, you wouldn't be aiming to identify the material properties specifically, but given a model that's tightly fitted enough, there should be constants that stand in for the missing material properties when it's all said and done.

Just an initial thought, I'd love to hear why I'm wrong from those in the field.

defterGoose• 4 years ago

I would think that what you're suggesting could be done by doing a bunch of Fourier transforms. If there is no relative motion of the camera you could assume that objects with high frequencies would be less massive than objects with low frequencies that could give you a useful bound on their mass/density. (Assuming a fairly uniform energy density in the scene)

For my own curiosity, I wonder if some of the variables they're missing are based more on the context of the video, rather than the content. Like, colors, for instance. Maybe if you ran everything through in black and white you could get it closer to an integer lower bound?

t_mann• 4 years ago

Thanks for those links! Back in school, I loved wondering about whether things like the inverse square law for gravity could be (have been) discovered from raw experimental data via ML - cool to see that it's actually been done already.

dls20162• 4 years ago

Piggy backing here... but this dimension-reduction or system-identification stuff always reminds me of Takens' theorem [0]. He even did some neural network stuff back in the "dark ages" of the topic [1].

[0] https://en.wikipedia.org/wiki/Takens%27s_theorem

[1] https://clgiles.ist.psu.edu/papers/NC-2000-learning-chaos-nn...

adamnemecek• 4 years ago

This is not surprising considering this is just a fixed point or diagonalization. I have written up a bit on this topic

https://github.com/adamnemecek/adjoint

YeGoblynQueenne14• 4 years ago

So these guys trained "an AI" to identify hidden variables by looking at videos of physical phenomena, without prior knowledge of physics (except what was implicitly and obscurely encoded in "the AI's" neural net architecture, and its training set and hyperparameters, by the researchers).

The "AI" came up with a slightly off number and the researchers struggle to understand why. They struggle because "the AI" is a black box that lacks explainability and cannot produce an explanatory model, only a predictive model [1]. The simplest conclusion is that their "AI" has not learned any useful models, and has only learned to accurately predict their test set, to which the authors had acces throughout the training of "the AI" [2]. The simplest explanation is that their model isn't very good at doing what they wanted it to do, but they opt to explain it as a scientific mystery, instead. Why?

It seems the only reason to try and see a scientific mystery where a simple failure would suffice as an explanation is because the model is "an AI". There seems to be some kind of expectation that "an AI" must have some deeper understanding of physics than humans, even when we don't understand what it's doing, even whe it's not doing very well.

In short, this seems to be based on very wrong assumptions and to be coming up with very wrong conclusions, but then again it makes for a great headline and so here we are, discussing it on HN.

____________________

[1] To clarify: a predictive model is one that can predict novel events. An explainable model is one that can explain how it made a prediction. An explanatory model is one that explains how the world works and why certain predictions are true, or not. Predictive and explainable models are useful, but most scientists aim to build explanatory models, because an explanatory model is necessarily also explainable and ultimately better at predictions, while a predictive model is not necessarily explainable or explanatory and an explainable model is not necessarily predictive or explanatory. Deep neural nets can only build predictive models, which are sometimes explainable, but they can't build explanatory models. That's because they can only identify correlations in data, but not explain those correlations with generalised theories, based on previous knowledge.

To put it plainly, neural nets can't come up with scientific theories, but they can estimate probabilities of things happening. But scientists can and want to come up with scientific theories, not just predictions.

[2] When the experimenter has access to the test data and can tune the learner's model until it scores highly on the test data, that leads to a model that overfits to the test data. Such a model is useless for prediction over unseen data (i.e. data not available to the researchers during training).

nootropicat2• 4 years ago

To clarify: a predictive model is one that can predict novel events. An explainable model is one that can explain how it made a prediction.

Sorry but that distinction doesn't mean anything falsifiable. Did you mean an explanatory model is one that's easier to understand for a human? If so, that's certainly useful, but it may well turn out that the simplest working model of physics is incomprehensible.

YeGoblynQueenne14• 4 years ago

No, I distinguish between "explainable" and "explanatory". To give an example, the theory of epicycles is a predictive model, that is also explainable, but it is not explanatory. Kepler's laws of planetary motion are an explanatory model that is also explainable and predictive.

The theory of epicycles is predictive because it predicts the motions of the planets, as they are observed in the night sky. It is explainable because anyone can perform the necessary calculations and understand how the predicted motions are, well, predicted. The theory of epicycles is not an explanatory model because it does not explain why the planets should move in circular orbits with epicyclical sub-orbits. Kepler's laws are explanatory because they explain planetary motion as a result of Newton's law of universal gravitation, are predictive because they can be used to predict the motion of the planets and are explainable because anyone can plug in the numbers and see how the results are calculated.

So an "explanatory" model is a theory that explains why things happen they way they happen. A predictive model only predicts that some things will happen. An explainable model explains why it made a prediction, but it does not explain why this prediction should hold or with what frequency.

Another way to see this is that an explanatory model explains past observations and predicts future observations, while a predictive model only predicts future observations and has no explanatory power, cannot explain why past phenomena were observed.

An "explainable" model is just a model that people can understand. It's not more complicated than that.

YeGoblynQueenne14• 4 years ago

> If so, that's certainly useful, but it may well turn out that the simplest working model of physics is incomprehensible.

I just re-read your comment and I notcied that.

I think by "the simplest working model of physics" you mean quantum mechanics? I hope I clarified how I mean "explainable" vs. "explanatory" but yeah, I think that's absolutely spot on. Quantum mechanics is predictive, but not explanatory. I do think it's "explainable" though in the sense I sort-of define it, because it's a bunch of formulae that anyone can plug in the numbers to, and see how they come up with results. I don't reckon there's many theories in the sciences that are not explainable. The lack of explainability only becomes an issue with black box models like neural nets.

Perhaps I should have used the word "comprehensible" or "interpretable" instead of "explainable" since it causes confusion by being too close to "explanatory". But those words have their own problems ("comprehensible"... by whom?)

As an aside, I'm on the camp that hopes that quantum mechanics is somehow fundamentally wrong and we'll figure it out in the next big paradigm shift. It bothers me deeply that we seem to have hit a wall where we can predict, but don't have a clue and can't understand why. I think every other big leap in scientific ... understanding (hint) has come from the ability to make explanatory theories, that tell us how things work and why.

drran• 4 years ago

To put it plainly, neural nets can't come up with scientific theories, but they can estimate probabilities of things happening. But scientists can and want to come up with scientific theories, not just predictions.

Current science is famous for "shut up and calculate" principle.

- Foo, bar, baz, ...

- But why?

- Shut up and calculate!

YeGoblynQueenne14• 4 years ago

As I understand it, that just applies to quantum mechanics. For example, in computer science, we certainly try to prove the properties of algorithms, such as their computational complexity or their soundness and completeness etc.

taneq• 4 years ago

OK so we know the answer to this question is 4, let's check our new software against that.
[software returns 4.7]
Oh my GOD, it's discovered new physics!

toxicFork• 4 years ago

At least it wasn't 42

Rerarom• 4 years ago

The new physics is not in the 4.7 but that in that the model's third and fourth variables seem to be new, compared to the known models.

YeGoblynQueenne14• 4 years ago

They seem to be uncorrelated with real-world physics. Whether they are "new" is anybody's guess. They're variables identified in a virtual environment, rather than the real world so there's very little chance they correspond to something in the real world, let alone physics, much less "new physics".

coliveira• 4 years ago

Good summary. There's no end to the kind of delusion some AI proponents will cling too. In a few days they'll say this is another proof we live in a simulation!

JackFr• 4 years ago

But can they play the tape backwards and come up with entropy.

danwills• 4 years ago

What am interesting idea!

jug• 4 years ago

I've thought about this myself before so this was an interesting read! I've thought in particular about quantum theories and how they are so hard to reconcile with gravity, and the hierarchy problem. Maybe we do simply struggle because this reality is so distant from the one we experience that we have trouble understanding it. Or maybe it is because we formalized it empirically but we got off on the wrong foot unlike Einstein's part that seems to come together better. Max Planck built his framework, not because he understood why quantum physics is the way it is, but because he observed how it worked. Since it works, we build upon that, what else can we do... So it looks like it's right to build on but we're having the damndest time to move forward. This can be very hard to later backtrack on!

alzyx• 4 years ago

since it's paywalled and genlib doesnt seem to already have it, here's the ArXiv link: https://arxiv.org/pdf/2112.10755.pdf

CrimpCity2• 4 years ago

This is cool and I personally believe this type of work may lead to breakthroughs in messy data rich fields like biology where we can arrive at a higher levels of abstraction maybe not exactly to "laws" like physics but highly correlative rules around phenomena. I think this is more on the side of knowledge creation and is human friendly as opposed to being more of a black box prediction like deep neural networks. Though I think both things are complimentary since human curiosity isn't satisfied by prediction alone.

If anyone else is interested in this line of work I recommend checking out Kathleen Champion, Steve Brunton, and J. Nathan Kutz's work on Discovering governing equations from data by sparse identification of nonlinear dynamical systems(https://www.pnas.org/doi/full/10.1073/pnas.1517384113).

Also this intro video is great! https://youtu.be/Z-l7G8zq8I0

jamesakirk• 4 years ago

Thank you for mentioning J. Nathan Kutz! Reading through this article, I saw similarities to Dynamic Mode Decomposition (I am not literate enough on the topic to elaborate). His Coursera courses and book were a fascinating dive into orthogonal basis functions, lower-rank approximations like PCA... I'm not sharp enough anymore (over a decade since grad school) to fully grok it, but damn his work is so cool!

rch• 4 years ago

From the paper:

code to reproduce our training and evaluation results is available at the Zenodo repository and GitHub (https://github.com/BoyuanChen/neural-state-variables).

jovial_cavalier2• 4 years ago

State variables of dynamic systems can be arbitrary. The fact that two that were selected by an automatic process happened to correspond to the two angles is probably more interesting than the other two being uncorrelated with the momenta.

mike_hock• 4 years ago

I guess it's because it's a serendipitous discovery by roboticists rather than an experiment explicitly set up to discover "alternative physics," but wouldn't it be much more efficient to feed the NN actual physical measurements rather than a video if discovering physical models was the goal?

It's amazing that the AI found two variables that corresponded to actual physical phenomena just from video footage but it seems highly unlikely that it found an alternative description of physics given how far off the answer was. The NN could be modeling any kind of weird "fantasy" for how the visual signal it was being fed could be produced that might not even correspond to 3D space in a meaningful way.

daveguy4• 4 years ago

It's interesting that the angular explanatory variables are also those that are easily measured in a 2d still image. Whereas the angular velocity requires differential information and is not be immediately represented in each individual image.

jovial_cavalier2• 4 years ago

There should be four state variables for a double pendulum system - angle 1, angle 2, momentum 1, momentum 2. But they don't have to be neatly divided up like that. Angle 1 could be represented by some proportion of all four quantities. The individual variables can have no physical significance in of themselves whatsoever as long as there is a mechanism for turning them into the final answer.

hsnewman• 4 years ago

I posted this 4 days ago.

motoxpro• 4 years ago

And multiple other people posted it 7 days ago, when it came out.

daveguy4• 4 years ago

Here is a basket of internet points just for you:

  \•••••/

   \•••/

    \•/

     -

lisper2• 4 years ago

Worth noting: the concept of "energy" was controversial for a very long time.

https://en.wikipedia.org/wiki/Conservation_of_energy#History

comboy• 4 years ago

Well, it does seem like a "made up", not directly measurable variable. We just have it for convenience of understanding the world. Or am I missing something?

lisper2• 4 years ago

Nothing is "directly measurable". Everything in science is a story that we make up to explain our personal experience, including "measurement" and "observation".

alok-g• 4 years ago

According to Noether's theorem, law of conservation of energy is same as symmetry of laws (equations) of physics with respect to time.

_hcuq• 4 years ago

My current thinking (I’m a total dilettante….) is that energy is a state in closed system. The system of course designed by the observer. For example, a cart half way up a hill. Does it have positive or negative potential energy? Depends on how you define your system.

a_nop• 4 years ago

I'll be tickled when they discover the AI is measuring the lens distortion of the camera or some other artifact of the signal chain.

a-dub2• 4 years ago

more cameras. more physical systems. different settings. more data. :)

Tao3300• 4 years ago

I was also wondering if one or more of these variables pertain to perception. Digital artifacts in the video being mistaken for machine elves.

bufferoverflow• 4 years ago

Or the noise specific to that camera sensor.

twobitshifter• 4 years ago

Seems likely that it would, our eyes have to do the same thing dealing with saccades, blind spots, inversions, distortion and more.

lowbloodsugar• 4 years ago

Having been programming a while, when confronted with "Well, it got the answer 4.7, and we figured out 2, and we can't figure what the other 2.7 are", I wouldn't leap to "Wow, the AI has found a whole new physics". I'd go with "bug", until I can very concretely describe otherwise.

a-dub2• 4 years ago

A particularly interesting question was whether the set of variable was unique for every system, or whether a different set was produced each time the program was restarted.

indeed! i'm assuming it finds the same variables over multiple runs with different rng seeds. if that is true, it seems the next interesting question is if multiple kinematic problems can be used for the training of the same net. one key feature of physics is that it was invented to describe general principles that apply to multiple problems, so it doesn't surprise me that it might find some more optimal representation that doesn't necessarily generalize to other physical problems.

ijidak• 4 years ago

the AI produced the answer: 4.7

This gave me a Hitchhikers Guide chuckle.

solarmist• 4 years ago

This is an extremely insightful comment. LOL

It does make sense though. Answers a much easier to arrive at than questions or processes.

7speter• 4 years ago

So uhh, is this how skynet figured out time travel?

mikewarot• 4 years ago

:::Tangent::: So if you had a new neural network, observing the physical world, and put a simple chaotic physics system within it's field of view, it would learn to encode such systems, and perhaps begin to understand the laws of physics?

So that's why we put dangly things above cribs for babies to focus on.

Roboticists discover alternative physics

Related Stories