The Matrix Cookbook (2012) [pdf]

https://www.math.uwaterloo.ca/~hwolkowi/matrixcookbook.pdf

by sebg • 3 years ago

183 44 3 years ago

math.uwaterloo.ca

porphyra• 3 years ago

The section on multivariate Gaussians was a real life saver at work recently. Highly recommended!

yootyootr• 3 years ago

For those clicking in expecting "Neo's protein porridge," "Morpheus Meatballs," and "Trinity's Tripe," you are going to be disappointed in that this is neither a delicious pop-culture recipe book, nor does it relate to hit Wachowski trilogy "The Matrix."

kemiller2002• 3 years ago

In the Wachowskis' defense on that one, it would be a really short cookbook.

Oatmeal: tastes like chicken.

done.

defulmere• 3 years ago

Oatmeal: tastes like chicken.

Surely you mean tuna.

mxuribe• 3 years ago

Don't try to "taste" nor try to identify the "taste" of the oatmeal...Because, there is no oatmeal.

keiferski• 3 years ago

"It's a single-celled protein combined with synthetic aminos, vitamins and minerals. Everything the body needs."

tjpnz• 3 years ago

It doesn't have everything the body needs.

__void• 3 years ago

yes I was a little disappointed, ngl

a post-apocalyptic recipe book would be really interesting (canned cultivated meat, mycotissues, texturised hydroponic soya and boiled grains, yummm)

giardini• 3 years ago

__void says: "a post-apocalyptic recipe book...":

That would be "To Serve Man":

https://en.wikipedia.org/wiki/To_Serve_Man

You can view that "Twilight Zone" episode at:

https://www.youtube.com/watch?v=wJjvg-Gq1LE

purpleflame1257• 3 years ago

I actually came in hoping for the chat protocol >_>

euniceee3• 3 years ago

Same

bee_rider3• 3 years ago

Neo : What are you trying to tell me? That I can dodge ZERO PIVOTS?

Morpheus : No, Neo. I'm trying to tell you that when you're ready, YOU’LL HAVE AN SPD MATRIX.

cratermoon• 3 years ago

I was at least hoping for a cookie recipe.

bee_rider3• 3 years ago

There’s also the famous Numerical Recipes book. That is more like a set of steps you can follow when writing your code, which seems like what a recipe/cookbook ought to contain.

Since this work is about identities, aka representations that can be swapped in for one another, IMO they should have called it the “matrix ingredient substitution guide.”

CoastalCoder• 3 years ago

IIRC the Numerical Recipes book was a bit of a copyright Trojan Horse. (I don't know if that's really true, I just heard it mentioned somewhere.)

Presumably this book doesn't carry that risk.

hgsgm2• 3 years ago

Everything written is copyrighted, whether it's in a published book or not.

darthoctopus• 3 years ago

previous discussion of these problems: https://news.ycombinator.com/item?id=30895307

The real problem is the Numerical Recipes license, which is extremely restrictive - probably more so than most users realize. I won't dignify it with a link, but you can find licensing terms on their website. NR routines are copyrighted (fine) and cannot be redistributed as source (annoying but not uncommon for COMMERCIAL software, somewhat unusual for scientific software). There is no exception for noncommercial or scientific use, which is Grinchy and irritating, especially given that two of the authors acknowledge funding from the NSF for work on numerical methods. Beyond that, the single-CPU/single-screen terms of their license are almost impossible for a working scientist to comply with, especially in a networked environment.

BeetleB2• 3 years ago

IIRC the Numerical Recipes book was a bit of a copyright Trojan Horse. (I don't know if that's really true, I just heard it mentioned somewhere.)

Oh yes, this brings back memories. The first edition, I believe, had a fairly permissive license. The second edition had a more restrictive license (it probably wasn't that bad - the authors wanted a payment for use, and I don't think it was a lot).

I was an undergrad doing a summer internship with a professor, and he wanted me to implement the method of steepest descent (or something similar) in Fortran for his computation. He told me to just copy it from the NR book. I dutifully looked at the license and told him we needed to pay to use it. He looked at it and said "Wow, they've gone hard core." Then he handed the book back to me and said "Well, we need an implementation in Fortran. Find a way, and don't come back to me until it's done."

I'm pretty sure that was code for "Do it and don't tell anyone." I instead spent days finding code online that was not restricted. Finally got one that worked.

Later on I spoke to numerical computation researchers and they had a universal disdain for that book. Apparently a lot of the algorithms were outdated - even when the book was published. Better algorithms existed, and they were numerically stable.

liorben-david• 3 years ago

My favorite matrix problem that I've had to do was showing that when viewed as a function from n^2 Real space to the Real number line, the determinant is continuous and differentiable. Computing its derivative is a particularly satisfying exercise in pattern parsing

From that you get that the set of Orthogonal matrices(Matrices with determinant 1) end up forming a manifold

clircle• 3 years ago

Blast from the past!

I actually used this Cookbook to look up a matrix derivative that I used in my PhD thesis!

tnecniv2• 3 years ago

I used the derivatives section on basically every homework assignment in grad school.

I wish CAS systems were better at linear algebra!

hoosieree3• 3 years ago

I love the "Notation and Nomenclature" section. So many academic papers assume readers will infer the type of a variable by its font styling, which is frustrating if you aren't already very knowledgeable in that particular field.

To clarify - using an actual programming language (any language, I'm not picky) would still be better than inventing an ad-hoc, unverifiable font-based type system every time you publish a new paper. But having a "Notation" section is still vastly better than the usual approach of "guess what I mean".

hgsgm2• 3 years ago

Mathematical notation is a programming language.

The notation section is type definitions.

BeetleB2• 3 years ago

Mathematical notation is a programming language.

It actually isn't - the notation has lots of ambiguities that don't confuse mathematicians because they are aware of the context.

On Proof and Progress in Mathematics[1][2] is a great essay by a Field's medalist. Part of the essay discusses this very topic, and why mathematicians choose not to use a formal notation for every day work. Some quotes from it:

The standard of correctness and completeness necessary to get a computer program to work at all is a couple of orders of magnitude higher than the mathematical community’s standard of valid proofs. Nonetheless, large computer programs, even when they have been very carefully written and very carefully tested, always seem to have bugs.
When one considers how hard it is to write a computer program even approaching the intellectual scope of a good mathematical paper, and how much greater time and effort have to be put into it to make it “almost” formally correct, it is preposterous to claim that mathematics as we practice it is anywhere near formally correct.

[1] https://arxiv.org/abs/math/9404236

[2] I discovered it via HN comments a while ago.

tnecniv2• 3 years ago

To be fair, fields have conventions that help quite a bit. Matrices are almost always (I have seen exceptions) capital letters.

I used to be an advocate for some kind of universal notation, but honestly it is not practical. A useful notation for one paper can be incredibly cumbersome in another paper in the same field. This gets worse if you’re paper is at the border of two fields and their notational conventions are mutually inconsistent. It’s best to just have a notation section that sets forth your convention in the document and stick to it. Ultimately, if you’re reading a paper in an area in which you are knowledgeable and can’t parse the notation quickly, the author failed to write the paper well.

As Whitehead said, “By relieving the brain of all unnecessary work, a good notation sets it free to concentrate on more advanced problems, and in effect increases the mental power of the race.”

hoosieree3• 3 years ago

I agree with the Whitehead quote. Ken Iverson opened his 1979 Turing award lecture ("Notation as a Tool of Thought") with it. And APL would be a suitable replacement for 90% of the bespoke pseudo-notations I've seen in hundreds of academic papers in my domain (machine learning and reverse engineering).

Contrast this with 2 papers I've read recently which independently defined a "function" that returns 1 when its argument is true, and 0 when its argument is false. In both cases, the purpose of this function was to either keep or cancel part of an arithmetic expression by multiplying by either 1 or 0. Each paper defined this function with different notation (one used prefix notation: "Fn(condition)", the other used brackets: "[[condition]]"). Each paper spent several sentences defining the meaning of this notation and the associated function.

Meanwhile in APL, true and false are 1 and 0. If these authors had used APL, they could both have used the same notation, and neither would need to explain beyond saying something like "we use APL notation here". This behavior is not unique to APL - JavaScript, C, Python, and many others have "truthy" values which can be treated as 1 or 0 for the purpose of multiplication.

I don't care if authors use APL (or Python, or Idris, or whatever). I just would prefer that (when possible) they use an existing language which is actually learnable. If it's acceptable to not define your notation (relying instead on tacit conventions and context) why not use an existing language?

mock-possum2• 3 years ago

So many academic papers assume readers will infer the type of a variable by its font styling

wait seriously??

_dain_• 3 years ago

Yes, have you never read a math paper before? Bold = vector is a common convention, for example.

SpaceManNabs• 3 years ago

Numerator vs Denominator layout. The great debate of matrix algebra.

mbustamanter• 3 years ago

This cookbook was insightful in the sense that I once tried proving several of its identities via Kronecker products but then found that it was much easier using tensor notation, so easy that I almost forgot Kronecker products altogether. I put together some proofs in my blog [0] (note: it's incomplete and the URL might change!).

[0] https://mbustamanter.github.io/ssg-blog/2022/matder1/

dryanau• 3 years ago

FYI the headings in this blog are barely legible in bromite for me, I can imagine they're supposed to look that way

nsajko• 3 years ago

Seems to look fine in both Firefox and Chromium.

jey• 3 years ago

Tom Minka's "Old and New Matrix Algebra Useful for Statistics" is also really handy: https://tminka.github.io/papers/matrix/minka-matrix.pdf

I used to always stare at these two PDFs while deriving machine learning algorithms back in the day. (At least my hand rolled code C++ still beats PyTorch's reverse autodiff performance.)

dang• 3 years ago

The Matrix Cookbook (2012) [pdf] - https://news.ycombinator.com/item?id=18566449 - Nov 2018 (39 comments)

The Matrix Cookbook (2012) [pdf] - https://news.ycombinator.com/item?id=14726223 - July 2017 (9 comments)

mock-possum2• 3 years ago

honestly pretty disappointed that this doesn't have Neo's famous Buckeyes recipe - even Morpheus's Enchilada Casserole or that juicy and delicious steak Cypher snacks on during his meeting with Smith would've been appreciated. Hell even Trinity's Ranchwater would be nice, even if it's only a couple ingredients, there's just something about the way she makes them.

jpfr• 3 years ago

You don‘t get funny points on technical topics. HN ain’t Reddit. ;-)

dTal• 3 years ago

It should have been just endless different ways of serving Taystee Wheat

nomorepdfs• 3 years ago

This pattern emerging of people posting PDF downloads should be discouraged and discontinued. I dont need to explain that.

The Matrix Cookbook (2012) [pdf]

Related Stories