Introduction to spherical harmonics for graphics programmers

by luu • 3 months ago

150 29 3 months ago

spherical harmonics can have uses beyond lighting

This math is also used in Ambisonic surround sound though newer techniques use planewave expansion.

For games, the full-sphere encoding of Ambisonic B-format can be decoded for arbitrary speaker locations and the soundfield rotated around any axis. I'm not sure if its ever been used for a game though.

analog31• 3 months ago

... also quantum mechanics. The textbook solution for the wavefunction of the hydrogen atom involves spherical harmonics.

hackingonempty3• 3 months ago

That fact is betrayed by the the similarity of the shapes of atomic orbitals and the sensitivity patterns of Abisonic B-format channels.

https://en.wikipedia.org/wiki/Ambisonics#Higher-order_ambiso...

https://en.wikipedia.org/wiki/Atomic_orbital#Orbitals_table

...and the same patterns appear on the unit disk with the Zernike polynomials, used to describe optical aberrations and more.

https://en.wikipedia.org/wiki/Zernike_polynomials

vatsachak2• 3 months ago

Ambisonics can give one end game audio

bschwindHN• 3 months ago

end-game audio?

Or it can give game audio to one end?

vatsachak2• 3 months ago

Awesome write up. Is the appeal of spherical harmonics in graphics compression?

Like, if you know the third order harmonics that's only 16 values you have to pass around

djmips• 3 months ago

For sure that's a big reason but it's also a useful basis for doing lighting calculations because of their sphere like nature. They are quite efficient in dynamic scenes and historically used in a lot of precalc to do something akin to real time Global Illumination

maho• 3 months ago

+1 for introducing them as real-valued functions over cartesian coordinates!

Typically, spherical harmonics are introduced as a complex function over spherical coordinates, which makes them much easier to derive, but imo hides their beauty.

The real-valued, cartesian form of regular spherical harmonics is also called "solid harmonics" or "harmonic polynomials", in case you want to dig deeper.

nice_byte2• 3 months ago

Author here. Please let me know if the sample code doesn't work for you. It's all single threaded dumb JavaScript which makes it very easy to read, but definitely not performant. I decided to stick with it for didactic reasons, but still worried that it may hang someone's browser.

geon• 3 months ago

Would it be practical to use high resolution spherical harmonics as a replacement for cube maps?

nice_byte2• 3 months ago

Not really. Besides the problems with ringing outlined in the post, the number of coefficients required to capture higher frequency detail grows quadratically, requiring not only more storage but also operations to evaluate. Which makes straightforward cubemap replacement impractical.

lukko3• 3 months ago

This is really great. I always saw those harmonic shapes as electron orbitals, I had no idea they could be used in lighting too - so cool.

It made me wonder - why do the electron orbitals take those shapes in say a hydrogen atom? Is there a constraint on the electron and proton together that make it fit only to spherical harmonic functions?

rsfern2• 3 months ago

The reason is that electrons (like all quantum mechanical objects) are wavelike. In an isolated hydrogen atom, the electron is in a spherically symmetric environment, so the solutions to the wave equation have to be spherical standing waves, which are the spherical harmonics. The wave frequencies have to be integer divisions of 2pi or else they would destructively interfere. (Technically each solution is a product of a spherical harmonic function and a radial function that describes how fast the electron wave decays vs distance from the nucleus)

What’s interesting is if the environment is not spherically symmetric (consider an electron in a molecule) the solutions to the wave equation (the electronic wave functions) are no longer spherical harmonics, even though we like to approximate them with combinations of spherical harmonic basis functions centered on each nucleus. It’s kind of like standing waves on a circular drum head (hydrogen atom) vs standing waves on an irregular shaped drum head

Of course the nucleus also has a wave nature and in reality this interacts with the electrons, but in chemistry and materials we mostly ignore this and approximate the nucleus like a static point charge from the elctrons perspective because the electrons are so much lighter and faster

lukko3• 3 months ago

Ah amazing - thank you for the response! I have a couple of related questions - is it that the non 2 pi frequencies exist, but they destructively interfere so we can't see them? My understanding is that the radial function for the electron is zero at the nucleus - there is no possibility of it being found there - but why is that the case?

crispyambulance• 3 months ago

Spherical harmonics are basically a fourier series. They're a complete orthonormal set of basis functions for functions for the unit sphere. Whereas the fourier series from calc 101 is a complete orthonormal set of basis functions on the unit interval (eg [0,1]).

In other words you can express any reasonable function on the unit sphere as a series of spherical harmonic terms. That makes them ideal for working with differential equations (eg schrodinger's equation for the hydrogen atom, or, emission from an arbitrary light source).

groundzeros20152• 3 months ago

And the number of terms you need to get a good approximation is related to the frequency. Low frequency signals like lighting work well.

hasley• 3 months ago

If one needs to describe (and maybe compress) functions or data on a sphere, spherical harmonics are really a thing.

An alternative would be to construct a new function (or matrix) that is not only periodic in azimuth, but also in elevation (i.e., extend elevation to a full circle -pi to +pi). Then, one can simply compute two independent Fourie r transforms: along azimuth and along elevation. [1] The same idea works on matrices using the Discrete Fourier transform (DFT/FFT). However, you then have to accept things like that your data points are all equal at the poles.

[1] https://en.wikipedia.org/wiki/Double_Fourier_sphere_method

evanb• 3 months ago

In case the author is reading this, if you're going to introduce the complex-valued harmonics you should be careful to put the complex conjugate in the inner product

    <f, g> = ∫ f(ω)^* g(ω) dω

which does match the corresponding linear-algebra inner product if the vectors are over the complex numbers

    p . q = Σ_i p^*_i q_i

which guarantees that p.p ≥ 0 even for complex p (and does not change the only-real case).

luxuryballs• 3 months ago

what level of math do I need to understand this? or the rest of the math in the post, something I can catch up on in a weekend? barely remember the last math class I took seriously, trig like 18 years ago

throwway120385• 3 months ago

This is basic integral calculus, and the sigma symbol indicates discrete summation.

yeoyeo42• 3 months ago

I think the steps would be like this:

- get understanding of ordinary vector linear algebra.

- understand what vector dot product does and why

- understand why an orthogonal set of basis vectors for the space you're working in is useful / what properties it has / how its used. like basic euclidean 3d space (1,0,0) (0,1,0) (0, 0, 1) basis vectors.

- get a refresher on basic calculus, in particular integrals

- understand this inner product, it's a generalization of dot product, except you can think of your vectors having infinite number of dimensions now.

- the properties of the dot product you know (like that two vectors are perpendicular if their dot product is 0) work for the inner product too. or perhaps its better to say that the general inner product is defined to have similar properties

- there are functions that are orthogonal to each other in the same way vectors can be orthogonal to each other, and you can use the inner product to tell which ones.

- spherical harmonics are constructed / by design orthogonal to each other. how to show this and where the intuition for finding them could come from is a whole topic...

- but once you have it, just like you can project vectors onto basis vectors (to essentially transform them into the coordinate system described by those basis vectors), you can project functions into the coordinate system represented by those orthogonal functions.

- then you have to figure out why you would even want to do this. in short is has a lot of useful properties/applications. in the graphics case you can compress some quite complex functions into just a few coefficients using this (not perfectly, there is some 'information loss', but still). integrating over two functions becomes cheaper when they are projected to SH basis. it lets you do some unintuitive stuff like combine light that goes into different directions into one common set of coefficients.

gucci-on-fleek• 3 months ago

what level of math do I need to understand this?

A basic understanding of differential equations is all that's really necessary, but knowing about orthogonal polynomials would be helpful too.

something I can catch up on in a weekend?

Probably not. If you know any calculus (even a basic high school class should be enough), two weekends would probably be enough; if you don't know calculus, then double it.

My advice would be to use an introductory level quantum physics textbook or an advanced chemistry textbook, since the spherical harmonics are used quite a bit in those fields. You could use a math textbook too, but those will tend to focus on details that are irrelevant to you.

An alternate path would be to learn about Fourier series/transformations, then what's discussed in the article will follow as a natural consequence. This is probably a harder option, but there's lots of really good learning materials online for Fourier transformations (and comparatively little for spherical harmonics), so it may end up being easier for you.

smath• 3 months ago

Loved the writeup.

I'll just drop a note here to say that these spherical harmonics are also used in creating specialized neural network layers that are useful for modeling 3D objects like point clouds and moleculues, proteins, etc. Basically whenever we want to make sure that rotating / translating the object doesnt make a new object. [0] is a good reference for this.

Even more interesting is that these are the same spherical harmonics that appear as solutions to Schrodinger's equation in quantum mechanics (s, p, d, f orbitals in an atom) [1]

[0] https://arxiv.org/abs/2312.07511. [1] https://en.wikipedia.org/wiki/Atomic_orbital

Introduction to spherical harmonics for graphics programmers

Related Stories