HN user
That's a great article! 1) I'd seen a lot of other cool stuff from the same author over the years (see https://math.ucr.edu/home/baez/README.html), but had somehow missed this one.
1) "Struggles with the Continuum" by John Baez
If you use the theory of nonlinear partial differential equations to analyze the behavior of compressible materials, you can find what are called the "characteristic" speeds, which are the speeds that various types of waves propagate at.
Compressible materials tend two have two different characteristic speeds, one for sound waves and one for shock waves.
The speed of sound basically works out to speed = sqrt(stiffness / density). So as a material gets stiffer, the speed of sound goes up. An infinitely stiff (i.e. incompressible) material by implication would have an infinite speed of sound, though this can't happen in any real material.
Shock waves travel faster than sound, at a speed related to the pressure difference across the shock. The greater the pressure difference, the faster the shock travels. So if you had an infinite pressure difference, you could have an infinitely fast shock wave, but again this can't happen in the real world.
However, sound and shock speeds only apply to pressure waves in a material. Other influences like gravity and electromagnetism travel at the speed of light. So for example, if you're doing fluid dynamics for plasma, then you'll have a third characteristic speed, the speed of light, because of the charged nature of the material attracting and repelling itself.
There are also more exotic characteristics, like the speed of a propagating combustion front in a flammable material. But when you get to this level you're no longer just solving one simple set of differential equations.
If mathematicians could solve these kinds of problems, they could answer valuable questions like "Will this equation always have a physically meaningful solution?" If the answer was "No", then we would know that the equation can't be a faithful model of reality.
We already know that the incompressible Euler equations can't be a faithful model, for reasons I've mentioned elsewhere in the thread. But I think the hope is that if they can answer these questions for incompressible Euler, then they can eventually extend their techniques to more complex fluid equations like Navier-Stokes, which people generally assume (but can't yet prove) is physically reasonable.
Simulation has great practical value, but it doesn't give you any guarantees about the behavior of the solutions for all the cases you haven't actually tried.
It's the equations themselves that are singular. When we write simulators, we usually have to paper over the singularities that are inherent in the math.
For example, if you're simulating charged particles moving around, and you use a force equation F = k q1 q2 / d^2 (1), then when d approaches 0 (i.e. when the distance between particles approaches zero), then the force F goes to infinity.
For atoms, it works the same way. If you use a force law like Lennard-Jones (2), it also has the interatomic distance in the denominator, so the equation has a singularity baked right in.
You could always adopt a more complex force equation that doesn't have a singularity in it. But in practice, it's easier to use a simple but singular equation, and then selectively ignore its bad behavior.
The density can be constant, but it doesn't have to be. If the density field starts out with some variation in it, then those variations move around as the fluid flows. Incompressibility just means that those density variations can't get bigger or smaller, they can only move, shear, and rotate.
Pretty much any mathematical model of a real phenomenon can have some sort of singularity or discontinuity in it.
If you model atoms as dimensionless points (1), then any kind of force law with the distance between atoms in the denominator can lead to a singularity when that distance is zero. In practice, you write the simulator to disallow this, but it's still there in the equations, you're just ignoring it.
If you model your atoms as finite-sized but incompressible billiard balls, then when they hit each other it's a discontinuity, since they instantly change direction when they collide. These collisions conserve total momentum and energy, but they're unphysical because real physical quantities can't jump from one value to another (in classical physics).
Even if you model your atoms as little rubber balls, the model can still be singular. Linear elasticity (the most common choice) allows you to compress a finite-sized object down to zero size with finite energy, which yields infinite energy density. Again, you'd have to disallow that in the simulator, which is very practical, but not theoretically satisfying.
1) https://en.wikipedia.org/wiki/Molecular_dynamics is the typical method of atomistic simulation.
Wow, I'm honored :) These days, I try to only comment when an article is really in my wheelhouse, but that's not very often, given my narrow interests in fluid dynamics and computational physics.
The incompressible Euler equations model a fluid as a two-valued field. This means that at every point in space, the field has two values, density and velocity (1).
To me (2), a singularity in a field like this means that one or more of the field values "blows up", i.e. goes to infinity as you run the time variable forward.
But how could this ever happen? The Euler equations model the "conservation" (i.e. constant-ness) of three real physical quantities: mass, momentum, and energy. If these three quantities are finite and constant when you add them up over the whole field, how can any part of it "blow up" into an infinite value?
The answer is that the blow-up must occupy a volume that shrinks as the blow-up grows, so the conserved quantities are still constant. The singularity would be infinitely small in space, and have an infinite value of density or velocity (or both).
The hard question is, are these blow-ups merely artifacts of a particular numerical simulation technique, or are they essential somehow to the incompressible Euler equations themselves? That's what these papers are trying to figure out.
To me, an "essential" (i.e. inherent-in-the-equations) blow-up seems intuitively reasonable because of the acausal nature of the field. When you simulate the incompressible Euler equations, it superficially looks like it's a physical fluid doing physical-fluid things, swirling and flowing around. But in a real fluid, a change in one part of the fluid propagates to the other parts at finite velocity, creating real cause and effect.
An Euler fluid's time evolution is not a phenomenon that ripples forward through time in a normal way. Instead, every point in the fluid responds to every other point simultaneously. If you poke a cube of incompressible Euler fluid with your finger, there is no pressure wave that ripples through it, where the fluid parcels push each other along and get out of each other's way. Instead, the whole cube of fluid somehow instantly adopts a new flow pattern that conserves mass/momentum/energy in response to that finger-poke.
1) Note that velocity is a vector, since it has a direction. This means that in 2D the velocity is two numbers, and in 3D it's three numbers. So technically the 3D incompressible Euler equations have four values at every point: one density, and three velocity components, one each in the x, y, and z directions.
2) I'm a numerical simulation guy, not a mathematician. Real math experts have rigorous definitions of a singularity, e.g. in https://arxiv.org/pdf/2203.17221.pdf "Singularity formation in the incompressible Euler equation in finite and infinite time," Theodore D. Drivas and Tarek M. Elgindi.
The question that the referenced paper (1) is trying to answer is "do the 3D incompressible Euler equations develop a finite time singularity from smooth initial data of finite energy?" This is an important question in the theory of nonlinear partial differential equations, but is probably not as relevant to real fluid flow as a lay reader might imagine.
The incompressible Euler equations model a very strange and unphysical kind of fluid. Incompressibility means that the speed of wave propagation in such a fluid is infinite, which means that normal causality is not respected. Effects in such a fluid happen simultaneously with their causes.
For example, if you apply a force to one end of a pipe full of Euler fluid, the fluid instantly starts coming out of the other end of the pipe, with no time taken for this effect to propagate from one end of the pipe to the other. You could use a long pipe full of Euler fluid as a superluminal communication device!
Intuitively, it seems reasonable that in such an unphysical fluid, it would be possible to form a singularity even from smooth initial conditions. The difficulty, of course, is proving that intuition, which is what the paper is trying to do.
1) https://arxiv.org/pdf/2210.07191.pdf "Stable nearly self-similar blowup of the 2D Boussinesq and 3D Euler equations with smooth data", Jiajie Chen and Thomas Y. Hou.
That's a good point. Guess-and-verify could be a handy additional heuristic method if Mathematica's other methods came up empty on a problem. I've also heard of machine learning being used to choose between internal algorithms available in formal proof systems, to try to pick the algorithm that's most likely to work instead of just trying them all sequentially.
The authors have shown a very nice and (to me) non-intuitive result. But they're playing a little fast and loose with their comparison to Mathematica. They're comparing their algorithm's accuracy (solution correctness vs. incorrectness), with Mathematica's ability to find the correct solution in less than 30 seconds. This is a very important distinction! Mathematica will never silently return an incorrect solution (barring software bugs, of course). And Mathematica can often take minutes to evaluate what appears to be a simple integral, so a 30-second timeout is far too short, unless you're simply trying to compare the computational efficiency of the two approaches.
There may be other subtleties as well. Mathematica works in the complex domain by default, which makes many operations more difficult, but the authors discard expressions which contain complex-valued coefficients as "invalid", which makes me think they're implicitly working in the real domain. Do they restrict Mathematica to the real domain when they invoke it? Perhaps, but they don't say one way or the other. And do they try common tricks like invoking FullSimplify[] on an expression/equation before attempting to operate on it? I'd like to see more details of their methodology.
I suspect that at least some of these "flaws" are intentional, and are meant to make programming easier, at the expense of some performance.
For example, three of the poster's points (not allowing device property querying, not allowing the programmer to choose where a kernel runs, and not exposting local memory to the programmer) all make programming easier, though they also disallow some types of performance tuning.
One big potential reason for doing GPGPU on a mobile device is to get better energy efficiency per gigaflop, rather than to get huge overall performance like on a desktop GPGPU. In this context, squeezing out all possible performance may not be as important.
It does look like mobile GPU vendors are about to start offering OpenCL support. For example, ARM submitted OpenCL 1.1 Full Profile conformance test results for the Mali-T604 last year (http://blogs.arm.com/multimedia/775-opencl-with-arm-mali-gpu...), and Imagination Technologies showed mobile OpenCL demos last year at CES (http://www.youtube.com/watch?v=sDrz-w1jzEU).
It's easy to see why OpenCL hasn't rolled out fully on mobile GPUs yet: writing and debugging a full OpenCL software stack is very expensive and time-consuming, and there's still not that much real programmer demand for OpenCL on mobile.
As for Renderscript, it's always sounded like a bit of "not invented here" syndrome Google's part -- we've already got CUDA and OpenCL, and RS doesn't really bring much new to the table. They've already deprecated the 3D graphics part of Renderscript in Android 4.1, so perhaps they'll do the same to Renderscript Compute soon.
When I saw the TEDx guys' caution about a "physics-related speaker [who] has a degree in engineering, not physics", it struck a bit of a nerve for me.
My Ph.D. is in electrical engineering, not physics. I recently got a computational physics paper accepted in a peer-reviewed journal, and I'm hard at work on a second paper. And in a different world, I could speak at a conference and no one would have to worry about my bona fides.
But I think the TEDx caution is well-founded.
Engineers are often accustomed to knowing more about science than the average person. It can be very easy for them, with the best of intentions, to convince themselves and others that they know more than they really do. It's easy to think you've got some great new idea if you don't engage existing experts in the field via peer review and reading papers.
This is not to say electrical engineers can't be authorities on physics topics -- quite the contrary! But I agree with the TEDx guys that it does merit a bit of extra checking, especially in their situation.
I work in the R&D division of a microprocessor company. Most of what we do, I would call "research", but not quite "science". We investigate things that are too risky or time-consuming for our product design groups to look into, with an eye towards making our company money in the future. We fund Ph.D. students in electrical engineering and computer science at university labs, and collaborate with them. But we don't (generally) push back the frontiers of human knowledge in our day-to-day work.
Things might be different at (say) IBM Research Zürich, doing work on atomic force microscopy, but that sort of thing seems to be the exception rather than the rule in industrial R&D. I don't know if I would have expected to see hardcore science at Sun Labs where the article writer's friend worked.
Stross' comparison between Cray 1 performance and that of a modern smartphone seems off. He says: "A regular ARM-powered smartphone, such as an iPhone 4S, is some 12-13 orders of magnitude more powerful as a computing device than a late 1970s-vintage Cray 1 supercomputer."
A Cray 1 could peak at about 250 MFLOPs/s (http://en.wikipedia.org/wiki/Cray-1), and a modern smart phone like the Galaxy Nexus peaks at about 9.6 GFLOPs/s (using ARM Neon instructions on both cores). That's less than two orders of magnitude difference.
Floating-point power efficiency seems to have improved by about 6-7 orders of magnitude in that time though, which is very nice :)
It sounds like the NCSU guys are using the CPU as a prefetcher to speed up GPU kernel execution, not using the GPU to speed up normal CPU programs as the ExtremeTech article implies.
The CPU parses the GPU kernel and creates a prefetcher program that contains the load instructions of the GPU kernel. This prefetcher runs on the CPU, but slightly ahead of kernel execution on the GPU. This warms up the caches, so that when the GPU executes a load instruction, the data is already there.
Humorously enough, early television in the UK (from 1929-1935) had only 30 lines of resolution and so could be recorded at audio frequencies onto a "Phonovision" disk. You can see some examples at http://www.tvdawn.com/recordng.htm of the low quality of the results.
For a little while, the BBC did broadcasts of 30-line video using two AM radio frequencies (one for the video, one for the audio) that could be picked up by special recievers.
Having to reject someone can be a very stressful experience. I don't like the idea of purposefully inflicting that stress on others unless there's a chance we both could benefit.
"Getting rejected on purpose" sounds like I'd be picking situations where I know for sure I'll be rejected -- an inappropriate proposition, an undeserved request. I don't want to do that to someone else just to try to desensitize myself to rejection.
It looks like a big linear motor like in the video is probably $1000 to $2000 dollars. Angle sensors are pretty cheap (just a few bucks). Then you'd need an A/D-D/A board to control it with ($500 to $1000).
The guy whose thesis this was comments on it here: http://www.reddit.com/r/math/comments/ebql6/double_inverted_...
I had guessed they were using visual sensing because of how the two halves of the pendulum were painted in contrasting, bright colors, but I was wrong. There are angular sensors at the joints of the pendulum, and a position sensor in the wagon at the bottom. The control system also needs the velocities of the parts of the pendulum, which it gets through a state observer (http://en.wikipedia.org/wiki/State_observer).
The interesting part for me was how the control system stands the pendulum up: it figures out the potential energy the pendulum will have when upright, then jerks the wagon around to add that amount of energy into the system as kinetic energy, then guides the system around a constant-energy landscape until it's upright. Pretty clever!
I agree that component count and the price of other logic chips were important too. Woz was a genius at that kind of design where you have to do something super-creative to save that last tiny bit of cash.
However, I think I can make a good argument that memory price was the main factor. As a hardware engineer, the only reason you'd ever use tricks like indexed color (palettes), foreground/background color per character (like CGA or the TMS9918), even/odd column color restrictions (like the Apple ][), and all the rest is to save memory.
Once you can afford enough memory to store your full color depth for every pixel in the frame buffer, your display hardware actually gets simpler -- it's just a straight pipe from memory to screen with no extra logic (other than your D-to-A conversion in the old days). No indexing, no complicated bit-packing schemes, no switching of color palettes during horizontal or vertical retrace.
Tons of effort and creativity went into that stuff, but it was all obsolete once memory got cheap enough.
It's true that the Apple ][ was double-buffered, but as another commenter mentioned, it wasn't a game console :)
But the Apple ][ is actually a good illustration of my point. Due to the cost of memory back then, Wozniak had to make many compromises in the hardware design, which resulted in a number of very odd visual artifacts. For example, if you tried to move a colored sprite around on screen, the colors of pixels in odd and even columns would flicker between two different four-color palettes.
The retro-engineered video subsystem in the article is essentially the same resolution as the Apple ][, but with greater color depth and none of the compromises, all because the cost tradeoffs have changed so much in the last 30-odd years.
I liked how today's low memory prices completely changed the hardware design tradeoffs. This system uses only 1980s-level technology, but you could never have afforded to build a bank-switched double-buffered memory system like this for a home game console back in the day.
As a result, the animation is super-smooth and doesn't show any tearing artifacts. Which is funny, considering that you could still see tearing on the Windows desktop until Vista (the underlying hardware had long been double-buffered, but operating systems took forever to make full use of it).
The founder's thesis mentions that they use a linearizer in their analog circuit, but all that does is give the same precision over the entire logical value range from 0 to 1 (by that I mean the same amount of voltage swing equals the same amount of "logic value change" anywhere in the range).
I suppose they could use a "non-linearizer" to put more of the precision near 0 and 1, but it would come at the expense of precision in the middle. The less voltage swing is involved, the more susceptible you are to noise from various sources.
I think it is likely we will see more of these. But this probability chip sounds more like an analog computer than what D. E. Shaw has done.
The Lyric web site says that they "model relationships between probabilities natively in the device physics", where D. E. Shaw's Anton chip sounds like it uses traditional logic gates the same way a GPU does.
P.S. Sorry, I downvoted you by accident -- I meant to upvote you.
It would be very hard to accurately account for a microprocessor's out-of-order instruction issue in a compiler.
A modern desktop-class microprocessor can have scores of instructions "in flight" at once. Each new instruction that's decoded gets reordered or stalled differently depending on the status of all of those previous instructions and their hardware requirements.
A compiler could try to model the processor's microarchitecture and simulate how it would execute the program, but that simulation would be inaccurate because of data-dependent branching, interrupts, and other unpredictable behaviors. Plus, the microarchitectural details you'd need to create such a simulation are usually not available.
That said, compilers can still tailor code to the microarchitecture in a general way. For example, if you know a processor can only decode one branch instruction per cycle, you can try to reorder the compiled code to avoid back-to-back branches in the instruction stream.
That might very well fix the anisotropy problem (I haven't seen it done yet, but it sounds reasonable). However, that leaves you with another problem -- how do you create a stable particle-like pattern that can travel over a randomly-connected grid of connections without disintegrating :) Something like a Game of Life glider will explode if it hits a differently-connected area of the grid.
But say you solve that problem too -- there are many more problems after that. How do you encode the other properties of a particle like mass, charge, spin and momentum into this pattern? How can patterns attract and repel each other at long distances like real particles do?
These problems are probably all solvable, but my point to the original poster was that it's harder than it seems at first, and it's not something that's amenable to a simple search of the state space of possible automata.
You're right, NKS does cover more than cellular automata, like network systems (as your reference shows). But all the examples Wolfram gives of network systems have the same anisotropy problem as a cellular automaton. He uses mostly hexagonal grids for his network systems, which are better than square grids, but still not isotropic.
Wolfram doesn't give even a simple example of how two particle-like structures might repel or attract each other in an isotropic fashion in a network system (or any other system in NKS). That doesn't prove it's impossible, but if it is possible neither Wolfram nor anyone else seems to have any idea how to even get started.
It's harder than you might think to model real physics with a cellular automaton.
Cellular automata are simulated on regular grids of cells, which gives them anisotropic (direction-dependent) behavior. For example, moving patterns in most automata can only travel in certain preferred directions (like gliders in Conway's game of life). And patterns that can move in multiple directions usually travel with different speeds in each direction.
In the real world, we don't observe any anisotropy in space, so none of the cellular automata I've seen proposed up to this point can model real physics, even in principle.
Lattice gas automata use hexagonal grids instead of square grids to alleviate this problem somewhat, but the anisotropy never really goes away, it's just reduced.