cat_man

You're right, I did mistakenly mix a few concepts together. The series representation for the function is infinite, like you say. What I was misapplying was the idea that a convergent infinite sum of matrix products (c0I + c1A + ... + cnA^n + ...) can be written as finite polynomial, which relies on Cayley-Hamilton like you say. So f(A) for any particular choice of A, can be written as a finite polynomial, but the coefficients of that polynomial will change as A does.

So writing

exp(A) = I + A + A^2/2 + ... + A^n/n!

like I did isn't correct. The concept I was thinking of was a Taylor series like this

exp(A) = I + A + A^2/2 + ... + A^n/n! + ... (to infinity)

has an equivalent finite polynomial like:

exp(A) = c0I + c1A + ... +c_{n-1}*A^(n-1)

(usually expressed as order n-1, not n like I was said earlier). The c0, c1, ... c_{n-1} will depend on A, so not as generally useful as what I was misremembering. It does let you build representations for things like exp(tA) (where t is a scalar multiple) in terms of a finite polynomial (the terms depending on t, like the "exp(lambda*t)" in your diagonal matrix example, end up in the c0, c1, ..., c_{n-1} coefficients).

Thanks for the correction and sorry if my garbled attempt at explaining confused anyone. The point about matrix functions being based on series representations is independent of those mistakes.

My memory is fuzzy on the details, but there's a way to show that if you have a series representation of a function that equals that converges to a=that function (i.e., an infinite sums of x^0, x^1, x^2, ..., x^n, ... with n going to infinity that equals f(x) - like the Taylor series of an exponential), that when you apply that to square matrices to represent a matrix function, there's an equivalent representation as a finite matrix polynomial. You can take the highest power of the equivalent finite polynomial as low as one less than the order of the matrix (so a matrix function of NxN matrix can be written as a N-1 degree matrix polynomial).

In other words, for a square NxN matrix A and functions satisfying the appropriate conditions, this convergent infinite sum of increasing powers of A:

c0 + c1A + c2A^2 + ... + cnA^n + ...

can be written as the n-1 degree matrix polynomial:

b0 + b1A + b2A^2 + .... + b_{n-1} A^(n-1)

The coefficients b0, ..., b_{n-1} are different than the coefficients c0, c1, ... for the infinite series, but they evaluate to the same thing.

Critical to the point you're making, the b0, b1, ..., b_{n-1} are different for differing choices of A, because the bn coefficients depend on the eigenvalues of the matrix. So you really have b0(A) + b1(A) * A, etc. The coefficients of the infinite series (c0, c1, ...) do not have this dependence on A.

That makes the 1x1 case tautological, because

b0(A) = f(A)

That is, the finite series representation is the value of the function at its argument.

There's some info on some of that here:

https://en.wikipedia.org/wiki/Cayley%E2%80%93Hamilton_theore...

Matrix functions (at least ones I learned about way back when) are Taylor series representations of a function with the matrix plugged in. For example:

exp(A) = I + A + A^2/2 + ... + A^n/n!

For matrices, you can show these end up as finite series (i.e., it can be written with the highest n being the dimension of the matrix)

The A^n factors can be computed with fewer flops than dense matrices, I believe. The "Q's" don't involve little work since they're "unitary" and satisfy QQ = I, so that for example

A^2 = (QTQ)^2 = (QTQQT) = QT^2Q*

That's where the f(A)= Qf(T)Q* part comes from.

It sounds to me like he's quoting what someone described as a "fundamental tenet of numerical linear algebra" as "anything Jordan decomposition can do, Schur can do better". He doesn't seem to be defending "this is a fundamental tenet", but is saying matrix functions illustrate the "Schur can do better" concept. Both Jordan decomposition and Schur are QTQ* decompositions, so I think he's justifying "Schur can do better" with his comments on poorer stability of Jordan form (i.e., the matrix function property depends on A = QTQ*, which both cases satisfy, but Schur has better numerical properties).

I don't think the post said you need the Schur decomposition if you want an eigendecomposition (I'm taking to mean the full eigenvalue + eigenvector pair). It just pointed out that the diagonals are eigenvalues, the first column of Q will be have the eigenvector for the first eigenvalue, and you can obtain eigenvectors from the remaining info if you want it. I'm not sure what the typical approach is for doing eigendecompositions, it might not directly involve Schur decompositions at all.

So I agree partially that you could add a little motivation to make it more self contained, but also agree it's probably aimed at a less general audience, like students who'd have more context around some stuff where the details are light.

I've only watched about five of the videos on it, but Keenan Crane's intro to computer graphics series is pretty great (he's a CS professor at Carnegie Mellon):

https://www.youtube.com/playlist?list=PL9_jI1bdZmz2emSh0UQ5i...

He makes excellent use of visuals and well-crafted examples to get core concepts across without getting bogged down in details that aren't well-suited to a lecture format.

There are a couple of other comments that have mentioned oscillation modes, vibrations, etc. The first 7 pages of this series on sound synthesis might help give an idea of where these might come from:

https://drive.google.com/file/d/12SM0SAOvMq166gc8B1b81Y_S7HP...

The third page in particular shows a plot of "amplitude" versus "frequency" to show the "harmonic spectrum of a sawtooth wave". The "frequencies" correspond to the modes of vibration (i.e., sine waves of different frequency), which are the "eigenvectors" in this case. The "amplitudes" are the relative contribution of those vibrations to the overall sound, and these correspond to "eigenvalues".

The article is talking purely about constructing sounds via synthesis, so there's not necessarily a linear system associated with it, but there is a connection. Wave equations represented by linear partial differential equations can often be analyzed as a linear system that has these "modes of vibration" (i.e., series of orthogonal sinusoids at different frequencies). If you were to, for example, model a plucked string (like a guitar), you can model the solution as a weighted sum of eigenvectors (in this case, "modes of vibration" or sinusoids of different frequencies). The "weights" would be the eigenvalues, which determine the spectrum and ultimately the timbre of the sound produced.

That might seem more involved, because it's an infinite-dimensional linear system (i.e., the vectors are functions on a interval, rather than finite lists of numbers). It turns out, though, that the finite-dimensional discretization of an infinite-dimensional linear system (i.e., a partial-differential equation approximated by a finite-dimensional linear system) will sometimes have eigenvectors / eigenvalues that have similar features as the infinite-dimensional case. For example, there are certain finite-difference operators that can be written in matrix form whose eigenvectors will work out to be sampled sinusoids.

I'm not totally sure of the history, but I think a lot of the interest in eigenvectors / eigenvalues as a topic in matrix theory originated from this are (i.e., numerical solutions for partial-differential equations that were used to model physical systems).

I got a B in Linear Algebra, and I still can't describe why you'd need that in the real world, while calculus/diff eq/discrete math were clearly tied to physics/thermodynamcis/computer science problems I knew.

I thought this was an interesting comment, because I personally believe linear algebra is one of the most applicable topics in math and relates a lot to the topics you contrasted it with. For example, in multivariable calculus, derivatives of functions with multiple variables end up being linear maps, and understanding properties of those maps and how they're transformed helps a lot with understanding the properties of derivatives and how to apply them. Differential equations are solved in practice by approximating them as linear systems and solving those equations, so again, understanding linear algebra helps a lot there (e.g., eigenvalues are intimately connected to the ways differential equations behave). I'm not so familiar with discrete math, but I do know there are connections between linear algebra and some areas of graph theory (not sure how critical they are to those areas, though).

That's not meant as a criticism of your comment, because I think the way linear algebra courses are taught doesn't do much to make those connections clear. Intro courses focus a lot on mechanical problem solving and do a poor job of motivating concepts (e.g., I remember eigenvalue problems showing up mostly out of nowhere). In courses beyond the introductory level the presentation and focus is more abstract and does little to demonstrate why you would care beyond intrinsic interest. I think if more motivation or context were provided, it would help encourage those more interested in applications than math for math's sake to go deeper into a topic that can be very useful in a lot of applied areas.

I think it's a typo. \int_a^x f'(t)dt should be f(x) - f(a), and that would be the fundamental theorem of calculus (or some corollary of it). It looks like either f(a+x) should be f(x) or the upper limit x should be x+a. The rest of the post deals with Taylor series, which are often written with f(a+x) for the function being expanded, so that might have mixed the author up when writing that. They don't directly use that expression elsewhere as far as I can tell, so it doesn't affect the rest of the post.

If it's f(x) = f(a) + \int_a^x f'(t)dt, then you can think of f(x) - f(a) as the area under the curve of f'(t) on the interval from a to x. For example, if f(x) = x, f'(x) = 1. The area of a function with height of 1 from a to x would be x - a, which is the same as f(x) - f(a) = x - a.