Linear algebra for game developers

http://blog.wolfire.com/2009/07/linear-algebra-for-game-developers-part-1/

by Mithrandir • 15 years ago

143 30 15 years ago

blog.wolfire.com

5hoom• 15 years ago

Very nice.

Linear algebra really is the bread & butter of game programming.

Showing the theory alongside examples of how it's actually used to make games 'do stuff' should be very helpful to those getting started. I could have really used a simple set of examples like this when I was first getting a sprite to flit about the screen.

Keep them coming :)

ColinWright9• 15 years ago

I've just re-read what I've said, and while I'm going to leave it as written, I'd ask please not to take it the wrong way. My background is in pure math, so my understanding and expectation is different from most people here.

So ...

I'm surprised to see this sort of thing described as "Linear Algebra" because to me it's just utterly basic vectors, pretty much the first 20 minutes of the Linear Algebra course I did. I've always been confused in the past when people have said that learning linear algebra is critical to games and similar, but if this is what they meant, the confusion is largely cleared.

So I'd ask the HN community:

* Is this what you all mean by Linear Algebra?

* What other topics do you want to know about?

Many thanks in advance, I always welcome the opportunity to learn, even when painful at the time.

granitepail• 15 years ago

I'm with you. This is more geometry than it is linear algebra. When I think of linear algebra I think of vector spaces and determining their relationships, not simple vector math.

freakwit• 15 years ago

note that this is just part 1 of 4. (I am assuming you didn't read all four parts, because that material would have to be covered impressively quickly to fit in 20 minutes)

ColinWright9• 15 years ago

Well, 20 minutes was perhaps an exaggeration, but in our first lecture we covered everything up to the end of part 2 and most of part 3. I think in the second lecture we did a significant amount of part 4, but then we went in a different direction.

We certainly did changes of basis and arbitrary dimensional rotation in our first lecture. We didn't do quaternions, and in fact never did in the context of these sorts of things.

Of course, we had done basic vectors and matrices in secondary school - we certainly knew how to multiply N dimensional matrices, and were expected to invert 2-D matrices by hand and compute the determinant of 3-D matrices. They were on the final exam.

jonsen2• 15 years ago

I don't see this thing described as "Linear Algebra". I see it described as "Linear algebra for game developers". To me that's two rather different things.

ColinWright9• 15 years ago

I think it's elementary vectors for game developers.

That's not to put it down, it's just to point out that for 20 years I've heard people saying that they want to learn Linear Algebra because it's needed in games programming, and this is the first time I've seen that perhaps they're talking about something completely different from what think of as Linear Algebra.

This really is vector geometry.

alttag2• 15 years ago

The author jumps in to what I see as linear algebra (matrix manipulation) in part 3, and more heavily in part 4 (rotating axes).

The first two parts are a necessary foundation. Yes, it's basic vector stuff, but not everyone gets that vectors can be represented as matrices.

ColinWright9• 15 years ago

Well, thing is, Linear Algebra isn't matrix manipulation. Matrices are just one aspect of Linear Algebra - it's like saying that symbols such as 'x', 'y' and 'z' are "Algebra."

Matrices are representations of (linear) transformations between Linear Spaces. Linear Algebra is talking about the Linear Spaces, and manipulating matrices is sort-of like finding roots of or factoring polynomials.

Which brings me to your second part. Vectors are elements of vector spaces. Matrices are transformations between vector spaces. The one is not the other, and while there are similarities and overlap (and sometimes identifying aspects of each to the other is useful), conflating the two at an early stage is a recipe for getting no further than simple matrix manipulation. If that's all you want then that's OK, but you miss out on a lot of insights that can be really valuable and make you stand out from the crowd.

But remember, I'm saying this as a mathematician, and my terminology will be different. I mostly want to make people aware that they are using terms in a different way, and open the opportunity for the kind of confusion I've suffered for decades.

DamagedProperty• 15 years ago

I appreciate your response because I have spent many nights trying to connect the dots between linear algebra and game development. When I read about straight linear algebra I can not see the connection to make it useful for me. Now I understand that I have been focusing on the wrong things and really need to simplify it more. Thanks.

tptacek2• 15 years ago

This is an awesome thread.

Since you asked:

Give me the shortest path in Khan videos, reading assignments, and exercises to LLL lattice reduction starting at "the first 20 minutes of a linear algebra class".

(You asked! Lattice basis reduction is used in several practical RSA attacks).

ColinWright9• 15 years ago

OK, I really don't know much about the LLL since it was invented at the time I was studying LinAlg and never got as far as our course work. It was being used by Conway, Norton and Parker when I was doing my PhD, so I heard something of it, but never implemented it, as I was working in a different area.

However ...

Here's (some/most of) my understanding.

We're working in R^n, and suppose you have n linearly independent vectors b_1 to b_n with integer coefficients in the obvious basis. These vectors span R^n, but you need to use real numbers as coefficients to do that. If you only take linear combinations using integer coefficients then you get a lattice in R^n.

Exercise: Prove that (1,5) and (2,3) are linearly independent in R^2

Exercise: plot a portion of the lattice spanned by (1,5) and (2,3) in R^2.

The same lattice might also be spanned by different vectors, and in particular, you may be able to find a collection of shorter vectors that span the same space.

Exercise: Find two shorter vectors v1 and v2 that span the same lattice as the one above.

The new vectors must also be in the same lattice, and so must be representable as an integer linear combination of the original vectors.

Exercise: Represent v1 and v2 as linear combinations of (1,5) and (2,3)

Basically LLL consists of a measurement of how well you're doing so far, and a way of replacing one of your existing vectors with a linear combination so as to make your collection improve.

I'll stop there - ask more questions if anything is unclear.

Ogre• 15 years ago

I'm a game developer. My specialty is network programming, so I don't deal with this stuff a lot, but it's impossible to completely avoid it.

I also majored in CS at a college where the CS department was part of the Mathematics department (though it moved to Engineering in my last year.)

What I mainly remember of Linear Algebra was that it and I didn't get along. I struggled a lot with all the heavy math classes. When I started picking up matrix operations much later, from looking at other programmers' code, it didn't really occur to me to call it "Linear Algebra". When I saw the title of this article, I still didn't think that's what it was going to be about.

So, I'm a game developer and I'm with you, it's not what I'd think when I hear Linear Algebra either, though if I thought really hard and fought off the flashbacks, I could probably find matrix operations mixed in somewhere with my scant memories of Linear Algebra in college.

That said, this is a great series of articles for aspiring game developers. Everyone needs to know these basics these days (Ok, maybe not if you're making match 3 games or word puzzles or the like). I wish someone had just shown me this 15 years ago.

Dn_Ab3• 15 years ago

Hi Colin thank you for making this place so interesting. I struggle to teach myself maths but greatly enjoy it.

As I see it the most applicable branch of maths for games falls under the umbrella of Clifford/Geometric Algebra - stuff like quaternions, cross products and bivectors. Indeed graphics programmers would gain a lot of power and elegance from the unified view provided by geometric algebra - rather than the hodgepudge of seemingly unrelated concepts as it is currently presented.

Topics covered in a linear algebra class that are typically used in games are basic - vector arithmetic, dot products, linear and affine transformations and projection.

Below you note that a vector and a matrix are different things. But I think the word is polymorphic. Sometimes, the notation aspect - table based syntax for arranging numbers is the focus. So someone will say that matrices are made up of n column vectors. Other times the focus will be on the linear transformation aspect. Where a matrix is more like a higher order function and is evidently different from the entities it is transforming. It simply requires a change of frame.

ColinWright9• 15 years ago

The ability to move freely between different conceptualizations is both important and useful, but if used too early it comes at a cost. Thinking of vectors as elements of the space and matrices as ways to transform the space gives a clear separation of concerns, and can make it easier to grasp what's going on.

Later you can discover that by dividing up the matrix the right way you can think of it as 1xN matrices, and then you can associate those with vectors and gain further insight as to what's, but trying to do that early can be very confusing.

After all, an int is a kind of number, and a float is a kind of number, and we can move between those, but there are dangers unless you've got the difference clear in your mind. How many people have complained that 5*0.2 isn't 1.0? Knowing what's happening underneath, and then creating the mappings in your understanding seems to be the fastest way to mastery.

joshAg• 15 years ago

I can only speak for myself, but when i hear of linear algebra, I think of my 'linear algebra for scientists/mathematicians' course in college.

The math behind signal processing; I only took 2 classes in college, and boith were pretty hand-wavey (granted, it's hard to rigorously cover S and Z transforms when you know 100% of the class hasn't taken and probably never will take complex analysis.

simon_kun• 15 years ago

If people are interested in learning about vectors in a visual programming environment, then I highly recommend this tutorial on 2D vectors by Daniel Shiffman (of processing fame) http://processing.org/learning/pvector/

wingerlang• 15 years ago

I would recommend following this blog if you like to see impressive progress and tech that no other game can show (from what i've seen)

sidcool• 15 years ago

Ah, I missed you by 8 hours in posting this.

Linear algebra for game developers

Related Stories