What is an eigenvalue?

https://nhigham.com/2022/11/08/what-is-an-eigenvalue/

by RafelMri • 4 years ago

262 154 4 years ago

AdamH12113• 4 years ago

Does anyone know of an example of a simple physical system where eigenvalues have a physical interpretation? The examples I know of are all in quantum mechanics, which is a bit abstract for me.

qntty• 4 years ago

Natural frequencies of mechanical systems are eigenvalues of it’s equation of motion.

_spduchamp2• 4 years ago

Any references to help me unpack what you just said there?

antegamisou• 4 years ago

Most are introduced to the interplay between physics and linear algebra through the study of the mass - spring system where the type (real or complex), sign and amount of the eigenvalues determine its behavior and stability. For example, complex eigenvalues with positive real part indicate an unstable, or chaotic, in terms of amplitude convergence oscillation.

contravariant2• 4 years ago

If i recall correctly you can represent a harmonic oscillator as a linear differential equation with a 2x2 matrix. The imaginary part of the eigenvalues of this matrix correspond to the angular frequency of the oscillator.

I like this example because it gives a physical meaning to both eigenvalues and imaginary numbers. It also shows the connection between the sine and cosine and the complex powers of e comes from (since you can show that all three solve the differential equation).

hn_throwaway_992• 4 years ago

Don't know if this counts as a "physical" system, but Google's original PageRank algorithm famously uses eigenvectors and eigenvalues: https://math.stackexchange.com/questions/936757/why-is-pager...

williamcotton• 4 years ago

I have a brief overview of eigenvectors as a 2D shear transformation in this overview of PageRank:

https://web.archive.org/web/20130728183938/williamcotton.com...

FabHK• 4 years ago

Take a linear map from some space to itself, and ask:

What lines (through the origin) are mapped back to themselves? Those are the eigenvectors, and the amount by which they're elongated or shortened are the eigenvalues.

So, if we talk about 3d space, and we rotate things - the rotation axis is unchanged. That's an eigenvector (with eigenvalue 1).

If we mirror things - any vector in the mirror plane remains unchanged, that's an eigenvector (with eigenvalue 1), the vector perpendicular to the mirror is unchanged, but flipped, so that's an eigenvector (with eigenvalue -1).

If we dilate everything along the x axis by a factor of 2, say, then the x axis is an eigenvector (with eigenvalue 2), while the y and z axis and any vector in that plane is an eigenvector (with eigenvalue 1). Any other vector is "tilted", so not mapped to itself, so not an eigenvector.

MichaelZuo2• 4 years ago

What does 'through the origin' mean in a physical system?

atty• 4 years ago

When dealing with the Schrödinger equation, the eigenvalues are the energy levels of the quantum system.

_spduchamp2• 4 years ago

I read an interview with Australian wire music composer Alan Lamb that a stringed instrument with multiple overtones vibrating on the string can be analyzed by breaking down the vibration into eigenvalues, but I've never found any reference material that explain that. I'm wondering if he was referring to FFT.

auxym• 4 years ago

If you discretize the string into a bunch of tiny masses, linked together by a bunch of tiny springs, you can build a mass Matrix (diagonal) M and a Stiffness matrix K (element ij = stiffness of spring that links mass I and mass j).

I can't remember the next part exactly, you can look it up in a textbook, but you multiply the matrices KMK, or similar, and the eigenvalues of this are the natural frequencies of the string. The eigenvectors represent the mode shapes, ie the displacement of each mass element.

The same technique is used in Finite Element Analysis to find the modes and modeshapes of complex structures (a car frame, a bridge, etc)

alanbernstein2• 4 years ago

Complex exponentials are the eigenfunctions of the Fourier transform. In other words, frequency component values are the eigenvalues.

https://en.m.wikipedia.org/wiki/Eigenfunction#Vibrating_stri...

sfpotter3• 4 years ago

See my other reply.

xhkkffbf• 4 years ago

Think of a fun house mirror that for the sake of this example make you look twice as tall but 20% skinnier. This can be modeled by a two-by-two matrix with eigenvalues of 2 and 0.8. (Indeed, it will have them on the diagonals which makes it easier to study.)

planede• 4 years ago

Moment of inertia tensor [1], and principal axes of rotation [2].

Principle axes are the axes where a weightless body can rotate around without "wobbling". These axes are orthogonal to each other. If a rigid body has I_1 < I_2 < I_3 moments of inertia, then rotation around the first and third axes is stable and rotation around the second axes is unstable.

[1] https://en.wikipedia.org/wiki/Moment_of_inertia#Inertia_tens...

[2] https://en.wikipedia.org/wiki/Moment_of_inertia#Principal_ax...

ok1234562• 4 years ago

Markov probability matrix where the entries are probabilities of some physical event happening.

The the eigenvectors will be the long term stable state probabilities.

cscheid2• 4 years ago

The values associated with each vertex on the _dominant eigenvector_ (the eigenvector associated with the dominant eigenvalue) are the long-term stable state probabilities. That's from a single eigenvector, not "the eigenvectors".

litoE• 4 years ago

Not quite. For a Markov probability matrix, 1 is always an eigenvalue, and all other eigenvalues are less than or equal to 1. For each eigenvalue that is equal to 1 you get a long term stable state probability. These distributions contain disjoint subsets of the states, and the system will converge to one of those subsets, depending on the initial state. The eigenvalues that are strictly less than 1 do not add any information to the long term state of the system. See Stochastic Processes and Their Applications, V4 (1976) pages 253-259. I wrote it while still in grad school.

bodhiandphysics3• 4 years ago

Take an airplane... it’s dynamics are described by a series of differential equations. We want to know if it’s stable! If the wife values of the dynamics are real and greater than 1 it’s unstable. If the eigenvalues are complex and have a modulus greater than 1 it will oscillate instability. If one is equal to one, it will cause everyone to vomit.

bodhiandphysics3• 4 years ago

Also in computer science.. a web sites page rank is the eigenvalue of the connectivity matrix.

cscheid2• 4 years ago

That's not true. The page rank is read from the eigenvector, and is the value associated with the given vertex (ie web page). There are as many page rank values as there are web pages, but only one eigenvector from which to read: the dominant eigenvector of the transition matrix, which is the one with the largest eigenvalue. So, only a single eigenvalue for the entire pagerank computation.

sampo2• 4 years ago

Does anyone know of an example of a simple physical system where eigenvalues have a physical interpretation?

Oscillation modes in mass-spring systems. Here is a simple one with 2 masses and 3 springs, so the matrix is only 2-by-2.

https://math24.net/mass-spring-system.html

With more than 2 masses, you don't need to arrange the masses on a line, but you can have a 2d or 3d arrangement, with interconnecting springs. I am sorry I failed to find an example image.

The theory is explained, for example, around page 479 in this Thornton and Marion Classical Dynamics textbook. But you need to read about Lagrangian mechanics (chapter 7) before it makes sense.

https://eacpe.org/app/wp-content/uploads/2016/11/Classical-D...

syrrim• 4 years ago

The vibration of a bell (say) could be modelled by a matrix, with a state vector to represent position, velocity, and acceleration, and the matrix modelling the differential equations describing their evolution over time. The eigenvectors represent a basis of the system, so that we can describe any potential state vector as a sum of eigenvectors. If we do so, then each step of the system can be modelled by multiplying each of these eigenvectors by its corresponding eigenvalue. If an eigenvalue happens to be complex, then we can describe it in phasor form as the product of an amplitude and a angle. The amplitude tells us how it will decay (or amplify) over time. The angle tells us the frequency of oscillation, and thus the note that the bell will typically sound.

idiotsecant• 4 years ago

Get a sheet of rubber. Grab it in both hands and stretch it. Inspect your sheet and find a line on the sheet that you could draw in with a marker and when you stretched the sheet the line would grow and shrink, but would not change what it was pointing at (probably a line from one of your hands to the other, in this simple example) That is an eigenvector of your sheet stretching transformation. The eigenvalue is how hard you're stretching the sheet.

einpoklum• 4 years ago

I'll upvote any post beginning with "get a sheet of rubber" :-)

montecarl• 4 years ago

I think a system of springs is a good example. I think having a bunch of springs hooked together is a bit abstract so let's instead think of a molecule and model the bonds between the atoms as springs. If you were to squeeze this molecule together or try to pull it apart and then let go, it would vibrate in some complex way. By complex I mean that it wouldn't just bounce back along the direction that you compressed or stretched it.

However, if you write down the matrix of spring constants for the system and solve for the eigenvalues and eigenvectors of this system you can do something special. If you compress or stretch the molecule along the direction of the one of the eigenvectors then let go, the molecule will continue to vibrate along that same direction. The motion will not spread out to all other degrees of freedom. It will also vibrate with a frequency given by the eigenvalue of that eigenvector.

Additionally, any complex vibration of the system can be broken down into a combination of these independent vibrational modes. This is a simple fact because the eigenvectors form an orthogonal basis for the space.

cat_man• 4 years ago

There are a couple of other comments that have mentioned oscillation modes, vibrations, etc. The first 7 pages of this series on sound synthesis might help give an idea of where these might come from:

https://drive.google.com/file/d/12SM0SAOvMq166gc8B1b81Y_S7HP...

The third page in particular shows a plot of "amplitude" versus "frequency" to show the "harmonic spectrum of a sawtooth wave". The "frequencies" correspond to the modes of vibration (i.e., sine waves of different frequency), which are the "eigenvectors" in this case. The "amplitudes" are the relative contribution of those vibrations to the overall sound, and these correspond to "eigenvalues".

The article is talking purely about constructing sounds via synthesis, so there's not necessarily a linear system associated with it, but there is a connection. Wave equations represented by linear partial differential equations can often be analyzed as a linear system that has these "modes of vibration" (i.e., series of orthogonal sinusoids at different frequencies). If you were to, for example, model a plucked string (like a guitar), you can model the solution as a weighted sum of eigenvectors (in this case, "modes of vibration" or sinusoids of different frequencies). The "weights" would be the eigenvalues, which determine the spectrum and ultimately the timbre of the sound produced.

That might seem more involved, because it's an infinite-dimensional linear system (i.e., the vectors are functions on a interval, rather than finite lists of numbers). It turns out, though, that the finite-dimensional discretization of an infinite-dimensional linear system (i.e., a partial-differential equation approximated by a finite-dimensional linear system) will sometimes have eigenvectors / eigenvalues that have similar features as the infinite-dimensional case. For example, there are certain finite-difference operators that can be written in matrix form whose eigenvectors will work out to be sampled sinusoids.

I'm not totally sure of the history, but I think a lot of the interest in eigenvectors / eigenvalues as a topic in matrix theory originated from this are (i.e., numerical solutions for partial-differential equations that were used to model physical systems).

CamperBob22• 4 years ago

Wow, that's an awesome introduction to music synthesis. Bookmarking for future referral to others.

simplotek2• 4 years ago

Does anyone know of an example of a simple physical system where eigenvalues have a physical interpretation?

Yep, vibration modes. Vibration frequencies represent their eigenvalues while the shape that the structural system exhibits when subjected to said vibration corresponds to it's eigenvector.

If a structural system is modelled as a linear elastic system it's possible to apply an eigendecomposition of that system and represent it in terms of linear combinations of it's vibration modes/eigenvector, and consequently we can get very accurate representations by using only a hand-full of these eigenvectors.

You know swing sets? We would start to swing back and forth just by moving our legs in a particular frwquencey, and without much effort we could move more and more? It turns out the frequency we moved our legs was the system's vibration frequency/eigenvalue for the vibration modes/eigenvector representing the we swinging back and forth.

dr_dshiv• 4 years ago

Does this relate to the normal modes or eigenmodes of a system?

Actually, trying to understand how eigenmodes and eigenfrequencies — which I understand well — relate to eigenvalues and eigenvectors.

johnbcoughlin• 4 years ago

The speed of sound is the eigenvalue of a particular matrix (the "flux Jacobian") in the Euler equations, the 5-component system of partial differential equations that describe gas dynamics.

crdrost• 4 years ago

So my actual favorite first example is to do this with Fibonacci numbers as a linear recurrence relation, but that's not really a "physical" interpretation. Let me give you my favorite physical one:

The essence of special relativity is that acceleration is a bit weirder than you think. In particular when you accelerate by amount a in some direction x, even after accounting for the usual Doppler shifts you will find that clocks separated from you by that coordinate, appear to tick at the rate 1 + a x/c² seconds per second, where c² is a fundamental constant. Clocks ahead of you tick faster, clocks behind you tick slower (and indeed appear to slow down and approach a ‘wall of death,’ more technically called an ‘event horizon,’ at a distance c²/a. (This effect is called the ‘relativity of simultaneity,’ and it is in some sense the only real prediction of special relativity, as the rest of this comment will show—the other effects of ‘time dilation’ and ‘length contraction’ are second-order and can be derived from this first-order effect.)

This means that the transformation equations for moving into a neighboring reference frame are not the ones that Galileo and Newton proposed,

    t' = t
    x' = x – v t

but slightly modified to (to first order in v, so only considering small velocity changes)

    t' = t – (v/c²) x
    x' = x – v t

where w = c t is a measure of time in units of distance using this fundamental constant. How do we generalize and get the full solution? We can do it by looking in the eigenvector basis. Consider new coordinates p = x – c t and q = x + c t, given any (x, t) you can find a unique (p, q) which describes it and if you want to get back those values you would say x = (p + q)/2, t = (q – p)/(2 c). But feed these magical coordinates that come from eigenvectors into the above transform and it "diagonalizes",

    p' = (1 + v/c) p
    q' = (1 – v/c) q

and therefore if you want to make a big change in "velocity" c φ (here instead φ turns out to be "rapidity") out of N smaller changes, you can repeat this transform N times with little boosts by v/c = φ/N, and you will stitch together the full Lorentz transform out of little first-order Lorentz transforms:

    p' = (1 + φ/N)^N p = e^φ p
    q' = (1 – φ/N)^N q = e^{-φ} q

Transforming back and using the hyperbolic sine and cosine, sinh(x) = (e^x – e^{-x})/2, cosh(x) = (e^x + e^{-x})/2, the full formula is

    w' = w cosh(φ) – x sinh(φ)
    x' = x cosh(φ) – w sinh(φ)

where w = c t is a simple time-in-units-of-meters coordinate. Usually we denote cosh(φ) = γ, sinh(φ) = γ β, which gives this the more familiar form you'll find in textbooks, and the identity cosh²x = 1 + sin²x gives a formula γ = 1/√(1 – β²) for the latter... but this ‘rapidity form’ is in some ways more elegant. Anyway, point stands, from the "first-order" transform you can derive the "full" transform just by building any large velocity change out of an infinite number of infinitesimal velocity changes, and this is the source of the factor γ which describes time dilation and length contraction.

Okay, now for physical interpretation. You asked what physical meaning these eigenvalues and eigenvectors of the Lorentz transformation have, and the answer is this: the eigenvalues (1, 1) and (1, -1) of the Lorentz matrix represent light rays, the p/q description we came up with above was a description of spacetime in terms of light-ray coordinates where we identify an event at a particular place and time with the light rays that it casts, announcing that the event has happened, in the +x and -x directions. On the negative side, these are also the last light rays that were able to touch the event before it happened, so represent "everything it could have possibly known about" -- there is a space between these two "light cones" which is its "relativistic present," the things that anything which was there at the event cannot know about until the future.

The eigenvalues, exp(φ) = sinh(φ) + cosh(φ) = γ + γ β = √[(1 + β)/(1 – β)] and exp(-φ) = √[(1 – β)/(1 + β)], are the Relativistic Doppler shifts of those light rays. Indeed one can read them as e.g. exp(-φ) = 1/γ * 1/(1 + β) , here 1/(1 + β) is the standard Doppler shift formula from nonrelativistic physics and 1/γ is the decrease in frequency due to time dilation.

6gvONxR4sf7o• 4 years ago

Eigenvalues of covariance matrices are a famous example. You can get PCA from it.

j2kun• 4 years ago

Here's one: https://pimbook.org/demo/index.html

Eigenvectors are the fundamental frequencies of a spring system

photochemsyn• 4 years ago

I like this explanation, which puts the concept of operator at center and has some nice computational relations, and is related to the physical world by the notion of an observable, with an eigenfunction being a special kind of operator-function relationship. At the end, there's a nice non-QM application (Fourier transforms).

source: PW Atkins & RS Friedman, "Molecular Quantum Mechanics 3rd Ed"

An observable is any dynamical variable that can be measured... in classical mechanics... observables are represented by functions* (such as position as a function of time), in quantum mechanics they are represented by mathematical operators... We shall not in general distinguish between the observable and the operator that represents that observable (such as the position of a particle along the x-axis)"
"An operator is a symbol for an instruction to carry out some action, an opeeation, on a function...in certain cases, the outcome of an operation is the same function, multiplied by a constant" [that constant being the eigenvalue]

Form: (operator) (eigenfunction) = (eigenvalue) (eigenfunction)

"An important point is that a general function can be expanded in terms of all the eigenfunctions of an operator, a so-called complete set of functions... then a general function can be expressed as the linear combination [a sum over a complete set of functions, each function having its own coefficient]"

Finally, we get to a practical real-world example (non-QM):

"...for instance, the straight line g = ax can be recreated over a certain range by superimposing an infinite number of sine functions, each of which is an eigenfunction of the [differentiation] operator, d2/dx2. Alternatively, the same function may be constructed from an infinite number of exponential functions, which are eigenfunctions of d/dx."

Extending this general concept, we can go into the famous Fourier Transform, used widely for all kinds of classical problems in converting waveforms into frequency peaks, such as in musical analysis, electrical engineering, etc. The wiki page on Fourier transforms has a very brief mention of this view, i.e. The Fourier transform decomposes a function into eigenfunctions for the group of translations.

Here's what looks like a deep dive into this approach to the Fourier Transform (2008):

http://www.systems.caltech.edu/dsp/ppv/papers/journal08post/...

billfruit2• 4 years ago

I do think, the term eigenvalue is rather opaque, and should be replaced by a more plain-english terminology that readily conveys its meaning.

jxy• 4 years ago

Try to name the bones you used to type this sentence?

st_goliath• 4 years ago

As a native German speaker, I don't understand your problem. It's very much not opaque, plain terminology that easily conveys meaning. ;-)

Perhaps, we should compromise and name it after Leonhard Euler? That should clear up the confusion.

NotYourLawyer• 4 years ago

“Characteristic value.” I guess that’s a little better actually.

hdjjhhvvhga2• 4 years ago

Can you propose one?

billfruit2• 4 years ago

Linear scaling factor?

Scale of aspect?

Aspect factor?

Scale along Axis?

Axial scaling factor?

Natural scaling?

Propensity?

Leaning factor?

In geography for example(quoting from Wikipedia):

"In physical geography and physical geology, aspect (also known as exposure) is the compass direction or azimuth that a terrain surface faces."

gpsx• 4 years ago

You know what's another one from physics whose name has nothing to do with the actual meaning - "Gedanken experiment"

gmfawcett• 4 years ago

Huh? It literally translates to "thought experiment" in English, which is exactly what it means.

frozenlettuce• 4 years ago

In romance languages you have autovettore/autovetor/autovectore, as in "self-vector"

Tainnor5• 4 years ago

It used to be called "proper value" in English (you can still find that in old textbooks), but the (semi-)German word has basically entirely replaced it.

vmilner2• 4 years ago

3blue1brown on this:

https://m.youtube.com/watch?v=PFDu9oVAE-g&vl=en

raydiatian3• 4 years ago

Literally all you need.

vmilner2• 4 years ago

I think anyone starting a lin alg course could do a lot worse than watch all his "Essence of Linear Algebra" series before starting - then watch the relevant (c. 15 min) episodes as you take each lecture.

jackconsidine• 4 years ago

I was learning principal component analysis a few years back which uses Eigenvectors to reduce feature dimensions while minimizing information loss (sorry if I butchered this)

I was really struggling to grok what Eigenvectors and Eigenvalues were and found this video to be the best intuition primer. I wish I had 3b1b when I was in high school and college

hdjjhhvvhga2• 4 years ago

For a completely different approach, see this answer:

https://www.reddit.com/r/explainlikeimfive/comments/1avwm7/c...

hackandthink• 4 years ago

Machine Learning (LDA):

"By finding eigenvectors we’ll find axes of new subspace where our life gets simpler: classes are more separated and data within classes has lower variance."

https://medium.com/nerd-for-tech/linear-discriminant-analysi...

_gmax0• 4 years ago

Also, if you come from a computing background, I think Eigenfaces is a great, illustrative use of eigenvalues.

https://en.wikipedia.org/wiki/Eigenface

Waterluvian3• 4 years ago

Something that frustrates me, and maybe I’m just confessing my stupidity, is the extra layer of indirection in any discipline when things are named after people and not the thing’s characteristics.

My doctor once told me “if you learn enough Latin, a lot of names in medicine will hint at what they are, so you have less to memorize.”

I find that these names often lend a sense of complexity to concepts that turn out to be rather simple. In high school this really contributed to my struggles.

Edit: apparently Eigen isn’t a person’s name so I sure picked an embarrassing moment to bring this up.

martin_balsam• 4 years ago

But it is named after its characteristic, albeit in German

Waterluvian3• 4 years ago

Well… boy did I pick the wrong example to bring this up with. Alas, I’ll leave my shame here for all to see.

sfpotter3• 4 years ago

If you learn a lot of math, a lot of names will hint at what they are so you have less to memorize. :-)

Waterluvian3• 4 years ago

Except when some smart jerk discovered like eight different things!

jpmattia3• 4 years ago

is the extra layer of indirection in any discipline when things are named after people and not the thing’s characteristics.

"Eigen" in German has same English root as "own": "Eigenvalue" is Germanglish for "Own/inherent value", so meets your spec of naming a thing after its characteristics, as long as "naming" is allowed to be in multiple languages.

filmor• 4 years ago

It doesn't mean "same". It means "own" in the sense of "inherent" or "characteristic".

DocTomoe• 4 years ago

A common language fosters research and common understanding.

In IT, that language is English. In diplomacy, before interpreters were plentiful, that language was French. And in many classical, medieval-era sciences, that language was Latin (as a commonly-understood language that came from it's ease of being learned by romance-language speakers and being rather relevant in the (then church-run) universities).

So, there's no indirection intended. It's just an artefact of the past - an artefact that helps Chinese, Spanish and American doctors communicate (in broad strokes) even today.

323• 4 years ago

But you assume that there is one word to describe the characteristics.

If such a word doesn't exist, you might as well name it after a person instead of trying to invent a new word.

ravi-delia2• 4 years ago

I recently embarked on a journey to come up with a math vocabulary for Toki Pona, a lovely little artistic conlang which deserves better than what I'm doing to it. In Toki Pona, words are build up from simpler ones to describe a thing as it is. A friend is 'jan pona', a person who is good (to me, the speaker). So I've had to come up with names which describe math topics.

It's awful.

You know how many same-xs there are?! Eigenvalue, eigenvector, homomorphism, isomorphism, homeomorphism, homotopic. Which one gets to actually be "same shape"? Worse are when well meaning mathematicians use descriptive names anyway. Open and closed are not mutually exclusive, giving rise to the awful clopen (and don't pretend like ajar helps. an ajar door is an open door!). Groups, rings, and fields all sort of bring to mind the objects they describe, but only after you know the archetypal examples. Math is the study of giving the same name to different things, and that gives rise to more names than there are short descriptions.

So do you know what I did? Whenever I could, I used a real person's name. It freed up a limited vocabulary, and gave enough wiggle room to translate most undergrad math without too much loss. I suspect a similar thing is in play with math. Maybe the category theory people have abstractions to usefully describe "same-functions" without confusion. But in general, things are named poorly because it's genuinely a hard task.

constantcrying3• 4 years ago

It is sometimes very hard to name things well. The name either becomes so unspecific that it is just as useless, or it gets so long that nobody will use it.

This gets worse the "deeper" the math goes, but for me it never was a real problem, as you usually learn the definition together with the name.

cogman10• 4 years ago

You see this sort of thing crop up in chemistry.

For really simple compounds, names are more or less settled and consistent (with some exceptions).

But as soon as your compound starts to get more complex (think organic chemistry) all the sudden, it becomes nigh impossible to consistently name things. There are tons of compounds with the same chemical formula that are regionally named differently. Even worse, there are tons of compounds with the same chemical formula that are actually different things due to how the compound is arranged. (Good ole carbon chains).

ProjectArcturis• 4 years ago

Who is this explainer aimed at? If you can understand the first sentence, you probably already know what an eigenvalue is.

Jyaif• 4 years ago

Right, but it's great to refresh your memory about eigenvalues.

techwizrd• 4 years ago

Often, papers or terse textbooks will list a definition like the first sentence without the added detail below. I think this is great for undergraduate students or folks who'd like to refresh their memory a bit on eigenvalues, how they're derived, and what they may imply. I certainly found it helpful.

CamperBob22• 4 years ago

The thing about Higham is that he's sort of a one-man Wikipedia of linear algebra. Many of the terms that he uses also have their own pages that (eventually) break the concepts down into comprehensible terms.

See https://nhigham.com/index-of-what-is-articles/ for a useful listing. Or, in an alternative form, https://github.com/higham/what-is . Notice that if you go all the way back up the rabbit hole you'll find user-friendly articles like "What is a matrix?" that clearly define the terms used farther down.

I really dig Higham's pedagogic style, in case it's not obvious.

oifjsidjf2• 4 years ago

I was blown away in my Digital Signal Processing (DSP) class that eigen "values" exist for certain systems in the form of "waves".

Basicaly you put in a wave made from multiple sine and/or cosine waves through some function f(x) and the output is STILL a wave, though its frequency, amplitude and phase might change.

Technicaly if I remember correctly this applies to all complex exponentials, since those can be rewritten in the form of e^(ix) = cosx + i*sinx.

This formula also beatifuly shows how rotations and the complex exponentials are connected.

So basicaly you don't just have eigen values, eigen vectors: you also have eigen FUNCTIONS (sine and cosine above are the eigen functions of f(x)).

DSP basicaly revolves arounds functions that don't "corrupt" wave-like inputs (wave in -> wave out).

sverona2• 4 years ago

In fact functions are just infinite-dimensional vectors. Almost all of the theory goes through unchanged. This is the basic idea of functional analysis.

constantcrying3• 4 years ago

Notably, one very important part which does not go through is that for mappings between infinite dimensional spaces linearity does not imply continuity. (E.g. a series of functions bounded by a constant can have arbitrarily large derivatives)

A large part of functional analysis is dealing with that fact and its implication for PDEs.

OJFord• 4 years ago

Or rather size-of-their-domain-dimensional?

bobbylarrybobby2• 4 years ago

If you're into eigenfunctions, pick up any textbook on quantum mechanics. The hamiltonian is a linear operator whose eigenfunctions are the stationary states of the system ("stationary" because an isolated system in a stationary state will never leave that state) and whose eigenvalues are the observable values of the energy of the system. In general, there is a correspondence between observable quantities and Hermitian operators on wavefunctions: "measurement" is the application of a Hermitian operator to the wavefunction, and the values you may observe are the eigenvalues of the operator. So, for instance, energy is quantized in some systems because their hamiltonian has discrete eigenvalues.

c-baby5• 4 years ago

Maybe I'm missing what's interesting about this, but a function like f(z) = 5z + 2 would output a wave with changed amplitude and phase when z = sin(x). That doesn't seem that interesting to me, so f(z) must have some other interesting properties?

oifjsidjf2• 4 years ago

Honestly I forgot the details, but basical the ENTIRE field of DSP stands on this fact.

Basicaly there exist some functions into which you can feed in sound waves and the output is guaranteed to still be a sound wave.

If you'd feed in a sound wave and if the function would corrupt it you would not be able to do any digital signal processing, since the output must be a wave.

Sound(wave) in -> Sound(wave) out, guaranteed to always be true.

geysersam• 4 years ago

It's even worse than you describe it!

f needs to be linear, but the function in your example is not linear.

However, there are quite interesting linear functions. Example: f(x(t)) = x(t-2) + 4dx/dt - \int_0^t 2x(s) ds

mturmon• 4 years ago

I think what the comment is missing is that the class of linear f(z)'s is much larger than you might expect. Partly because it just is, and partly because we like it that way.

In the land of analog signal processing: any combination of capacitors, inductors, and resistors [https://soundcertified.com/wp-content/uploads/2020/04/speake...] is linear.

In the land of math abstractions of signal processing: differentiation, integration, finite-impulse-response (FIR) filters, IIR filters, frequency-domain equalization, etc. All linear. Remember, linearity is with respect to the full time history, so f(z) = z(t) - z(t-1) + z(t-2) is still linear.

So we already know the eigenvectors of that whole arbitrary pile of componentry! Given any box of the above components, we can exactly characterize its response to any input -- for all time -- by knowing a list of that system's eigenvalues -- one for each of the already-known eigenvectors.

That's the "frequency response" -- the eigenvectors are sinusoids, and the frequency response is the eigenvalue corresponding to each eigenvector (sinusoid). And of course the Fourier transform takes you back and forth from the time domain to the eigen-domain.

We liked this analytical framework so much that when we could fabricate nonlinear devices (transistors) easily, we purposefully arranged things so these devices were only used in a linear part of their response curve. Hence, amplifiers: f(z) = 11 z.

And then the musicians introduced distortion and f*cked it all up -- our system isn't in a linear regime, our old eigenvectors are meaningless, and we can't predict what will come out. Pure chaos.

mrfox321• 4 years ago

https://en.wikipedia.org/wiki/Spectrum_(functional_analysis)

WastingMyTime89• 4 years ago

I’m not sure I understand but it seems to me you are just talking about eigenvalues in C.

That’s interesting but not particularly remarkable because eigenvalues are defined for linear transformations of any vector space over a field.

topaz0• 4 years ago

Small correction: eigenfunctions are analogous to eigenvectors, not eigenvalues. In fact they _are_ eigenvectors, in the sense that they are vectors in a vector space of functions (or some restricted set of functions, e.g. continuous functions or periodic functions).

adamnemecek2• 4 years ago

All these points fail to mention that they are fundamentally self-relationship

Lawvere's fixed point theorem is I think the best formulation of the idea https://ncatlab.org/nlab/show/Lawvere%27s+fixed+point+theore...

I've been putting together a brain dump on the topic

https://github.com/adamnemecek/adjoint/

Join the discord

https://discord.gg/mr9TAhpyBW

hither_shores2• 4 years ago

There's not really a connection here, the category of vector spaces isn't cartesian closed.

adamnemecek2• 4 years ago

The category of Chu spaces is through.

raydiatian3• 4 years ago

A far easier to digest primer on eigenvalues is available from 3Blue1Brown [+]. His presentation format is undeniably approachable, so much so that I think you could probably use it to teach linear algebra and eigenvectors to 9-year olds.

[+] https://youtu.be/PFDu9oVAE-g

Kalanos• 4 years ago

You lost us at lambda

mpaepper• 4 years ago

I explained it in a more coding oriented style here: https://www.paepper.com/blog/posts/eigenvectors_eigenvalues_...

kiernanmcgowan2• 4 years ago

Eigen-things can also be thought of "fixed values" of a "thing" transformation.

For example - the eigenfunction of a derivative is e^x since when you run the derivative function on e^x you get.... e^x

marviel• 4 years ago

After watching the 3blueonebrown video linked below in the comments, I'm inclined to agree with you -- nice way of putting it.

Tainnor5• 4 years ago

They don't have to be fixed, a scalar multiple is allowed too. e^ax is an eigenfunction of the derivative too.

rmeno12• 4 years ago

In this case, we could say that all functions e^ax are eigenfunctions of the derivative, with the eigenvalues being a.

patrick451• 4 years ago

Another good example is that sinusoids are eigenfunction of a linear dynamical system. If you drive the system with a sinusoid, the output is also a sinusoid of the same frequency, but different amplitude and phase. In other words, the frequency response is the eigenvalue at a given frequency.

monktastic1• 4 years ago

Functions satisfy the laws of a vector space (well, certain classes of functions, such as L^2 anyway -- I'm certainly no expert here). I.e., functions are vectors. And the derivative is a linear operator on that space: d/dx(af + bg) = ad/dx(f) + bd/dx(g). Therefore d/dx has eigenvalues, one of which you properly identified as e^x (and more generally, e^ax, as others have pointed out).

Tainnor5• 4 years ago

(well, certain classes of functions, such as L^2 anyway -- I'm certainly no expert here)

For any vector space V over a field F, the set of functions V -> V is a vector space over F. This is essentially because the vector space properties are carried over "point-wise".

When you consider only certain classes of functions (such as L^2), the question of whether they form a vector space boils down to the question whether those classes are closed under addition and scalar multiplication. All other vector space properties are satisfied because they are in the larger vector space of all functions.

pclmulqdq2• 4 years ago

This is the best explanation of the eigenthings in my opinion. Too many people, this article included, reach for more advanced concepts to explain something as simple as "eigenvectors are how an operator transforms stuff and eigenvalues are how much."

kiernanmcgowan2• 4 years ago

Thank you - its too easy to get into the weeds with mathematics and lose the "why" of something. Connecting these higher level concepts with the algebra that _most_ people have experience with ends up being a more direct way to share knowledge vs diving into the higher level stuff that enthusiasts really enjoy.

kridsdale24• 4 years ago

God knows how I passed 4 years of engineering math without ever being able to describe to anyone what eigen* actually are, but I think that's actually the norm.

ukj2• 4 years ago

So… fixed points?

https://en.wikipedia.org/wiki/Fixed_point_(mathematics)

bobbylarrybobby2• 4 years ago

Yes except eigenvalues need not be 1. e^(kx) is an eigenvector of d/dx for all k but is only a fixed point when k=1

sampo2• 4 years ago

So… fixed points?

Fixed directions. The operator operating on an eigenvector doesn't change the vector's direction, but can change its magnitude (i.e. length). (Except, if you change the length from positive to negative, then you kind of change, or flip, the direction to 180 degrees opposite.)

dboreham• 4 years ago

It's the thing that explains everything.

w0mbat• 4 years ago

Eigen not understand a word of this.

codeulike• 4 years ago

Eigen regret trying to understand this

djmips• 4 years ago

Why not get Eigen Steve to weigh in? https://youtu.be/ZSGrJBS_qtc

papaver-somnamb• 4 years ago

I went through Linear Algebra I & II where eigenvalues & eigenvectors played a central role many a time and I was left with no elucidation regarding their practical applications.

Now I know that eigen*s (at risk of egregious oversimplification) can characterize systems and transformations, they are fundamentally widely applicable. Think of stretching narrow an arbitrary 2D image on a non-cardinal axis; an eigenvector can be a key factor that describes that transformation.

scmitton• 4 years ago

By far the best explanation of eigenvalues is found on 3blue1brown: https://www.youtube.com/watch?v=PFDu9oVAE-g&list=PLZHQObOWTQ...

Seriously, the course on the essence of linear algebra did more for me than my entire college linear algebra course. It's only a pity I discovered the videos at the end of my mechanical engineering degree and not sooner.

azepoi• 4 years ago

Steve Brunton and Nathan Kutz have great content for applied math also https://www.youtube.com/watch?v=QYS-ML_vn4k&list=PLMrJAkhIeN...

https://www.youtube.com/watch?v=2Z3bYlXJx3U

What is an eigenvalue?

Related Stories