Why there is no Hitchhiker’s Guide to Mathematics for Programmers

http://jeremykun.com/2013/02/08/why-there-is-no-hitchhikers-guide-to-mathematics-for-programmers/

213 75 13 years ago

vorvzakone• 13 years ago

tl;dr: i'm a snobby research student who is trying to trash developers who didn't need college. (yeah. that was an ad hominem. come at me.)

sthatipamala• 13 years ago

That's not at all what the author is trying to say. He's pointing out the shortcomings of mathematics notation and why that can be off-putting to otherwise motivated programmers.

NegativeK• 13 years ago

Not only is it not at all what the author is trying to say, I'm honestly confused at how you could arrive at that.

tunesmith3• 13 years ago

I read the article and did not come away with that impression. :-)

zepolud• 13 years ago

Actually, there is something that comes pretty close to this--it's called The Princeton Companion to Mathematics, curated by T. Gowers but mostly written by about a hundred of the top mathematicians in their respective fields, and is one of the most beautiful books I've ever read.

danso• 13 years ago

> Perhaps one of the most prominent objections to devoting a lot of time to mathematics is that it can be years before you ever apply mathematics to writing programs. On one hand, this is an extremely valid concern. If you love writing programs and designing software, then mathematics is a nothing more than a tool to help you write better programs.

Calling it an "extremely valid concern" is understating it. The dilemma is that programmers have many, many roads to follow if they want to improve their product, all of them discrete, deep fields in their own. Math is great, and certainly studying it will expand your mind...but so will studying engineering practices, statistics (which yes, is a kind of math, but not as theoretical as the kind the OP is advocating for), user interface design, and knowledge domains (medicine, legal, earth sciences, etc.)...all of which, as far as most programmers can tell, will yield concrete rewards much more easily than will mastering higher level mathematics.

doktrin• 13 years ago

This is a very succinct breakdown of the problem as I see it. If only it were so easy as being able to rationally and empirically determine that X is the most efficient investment of time and energy.

tikhonj2• 13 years ago

Abstract math is somewhat special just like studying programming languages and semantics is somewhat special: it's something of a meta-science, applicable to all the programming you do.

Everything else is specific to the domain you're programming in; these "meta" fields (I need a better name for them) have programming or reasoning itself as the domain.

cschmidt3• 13 years ago

I don't know about the OP's point on the value of proof to programmers. I do optimization and machine learning, and I use all kinds of math on a regular basis. But I don't prove things, and rarely need to follow someone else's proof either. You can get a lot out of applied math without touching a proof. And if it isn't applied, then you really don't need it ;-).

I agree there is value in learning how to read math notation, so there is a valid point in there somewhere.

currywurst• 13 years ago

I think that theorems are just like libraries you can pull in for your purposes, and the proofs are the unit tests that you can dive into if needed ;) !

mjw• 13 years ago

In the sense of http://en.wikipedia.org/wiki/Curry–Howard_correspondence, the theorems are the type signatures of the API, and the proofs are the implementations!

tuxdev• 13 years ago

Just because you don't explicitly write it down doesn't mean you didn't prove something. For optimization, you have to prove to yourself that the new code behaves equivalently to the old code. Anytime you use algebra you're doing a small proof to show that two expressions should be treated equivalently. Writing a proof down is the very last step, the hard part is thinking it through.

cschmidt3• 13 years ago

I do derivations all the time. I just do a series of algebraic transformations, starting from what I have, and getting to what I need. Some might say that I'm doing a proof. But I never use proof by contradiction, contrapositive, induction, or proper proof techniques.

Mgccl2• 13 years ago

You are restricted by the set of tools you already have right?

I can't believe how many times a great advancement in theory in different field comes from pure math advancement (or applying pure mathematical results). Say in economics, if Nash didn't apply topology, how would one ever know Nash equilibrium exists. Stable matching, compressed sensing and others also comes to mind.

I have a hypothesis that any field would benefit if we introduce some pure mathematicians to them.

j2kun10• 13 years ago

To be completely honest, the more common case is that the mathematicians follow behind physicists and engineers to verify that their applications are theoretically sound. But yes, great breakthroughs do come from pure theory.

coolSCV• 13 years ago

An analogous case is the programmer who relies on libraries and abstraction and "knows just enough to be dangerous." Statistics and optimization mathematics in particular are vulnerable to this. If you haven't read the proofs or you cannot prove it yourself, then you truly don't understand it. I'm not claiming this is you since your post doesn't imply that.

However as an example, if someone is solving linear or non-linear programs using <insert software or library here> but hasn't bothered to thoroughly study the mathematics underlying either and is just plugging in numbers from the constraint equations they came up with, then I would not want that person doing any sensitivity analysis. There would just be no way to be confident that they are interpreting the results correctly.

cschmidt3• 13 years ago

I've benefited from lots of theoretical classes, and you're absolutely right that you need to understand the theory of what you're doing. For example, I took a lovely class from Prof. John Hooker (at CMU) that derived Linear Programming duality as a natural consequence of Farkas' Lemma, rather than just mechanically grinding through LP tableau. I certainly followed the proofs discussed in many classes.

However, I can honestly say I got that understanding without doing hardly any proofs of my own. Graph theory was the only class where I "did proofs", and I didn't find it my favorite. (Later I did algorithmic graph theory and network flows - and that was fantastic, with no proofs.)

I guess my point is just that an engineer can understand and use a lot of pretty advanced math without spending much effort learning about proof techniques. That certainly wouldn't be how I'd spend my time. (Of course, if you enjoy doing that then knock yourself out.)

tunesmith3• 13 years ago

I've come up against this recently and I think a lot of imperative programmers will. For me, the basic pattern, starting from a career based on perl/php/java was this series of realizations:

- EECS circuit design is based on a conceit that gives chips a speed limit

- as a result, chips aren't getting much faster and that multi-cores are happening instead

- parallel programming will become more and more essential to make a buck as a programmer

- imperative languages aren't great for parallel programming due to mutable variables

- functional languages are better for parallel programming due to immutable variables

- functional languages are a lot closer to math concepts than imperative languages

- uh-oh

So for me it's been a fun few months of stumbling through self-directed study in stats, probability, and math. Bayesian probability from some articles by Yudkowsky. Fast, clumsy review of 0th-order, 1st-order, and zfc set theory. Anki decks out the wazoo. Now I'm plodding through Learn You A Haskell.

I would like to work more with actual mathematical proofs but I'm finding that tough to get into myself. It was fun paging through metamath for a few days, though.

bcoates3• 13 years ago

Message passing is an ancient and well-understood mechanism for producing parallel, scalable, mutable-state imperative programs. Feel free to continue to not learn math :)

tunesmith3• 13 years ago

But that's not as much fun. :-)

victorh• 13 years ago

There's no reason you can't have immutable variables and pure functions in imperative languages, though.

Not that I don't support your quest to learn Haskell, Haskell is awesome and I hope you enjoy it :)

reycharles• 13 years ago

> I would like to work more with actual mathematical proofs but I'm finding that tough to get into myself.

A professor at my CS dept. started using the Coq proof assistant[0] to teach some (functional) programming concepts and techniques. It has had a side effect of making the students more in to proofs.

Personally I think it's great - it shows that proving things is just programming.

I think the Software Foundations book[1] (or tuturial?) is a good place to start. There is also another good tutorial, but I can't remember its name. It's more about "getting work done" so I don't think it's as good as a starter.

[0] http://coq.inria.fr/ [1]: http://www.cis.upenn.edu/~bcpierce/sf/

andrewflnr• 13 years ago

This seems like a problem of terminology. It is valid to desire a better acquaintance with the theorems others have proven, just as it is valid to desire a better acquaintance with the user-facing features of Chrome, iTunes, Excel, etc. I guess "using math" is different than "doing math", and most people are more interested in the the former.

There is a bit of a continuum, though. Using Excel effectively involves a smidgen of programming, similar to how "using math" effectively really means using other peoples' theorems to "prove" things, very informally, about your own domain. Maybe that's why people are more likely to get confused between "using" math and "doing" math.

ekm2• 13 years ago

I think contest math is the best kind of math for a programmer since it sets out to solve problems and not to necessarily come up with theories like regular math.

thyrsus• 13 years ago

He's analogizing to the fictional "Hitchhiker's Guide to the Galaxy". There does exist an analog to the real world "Hitchhiker's Guide to the Galaxy": Carl E. Linderholm's "Mathematics Made Difficult". Utterly hilarious.

Regrettably, it's out of print, and you have a choice of paying upwards of $138.50 (abebooks.com) for a used copy of a book of jokes, or finding the pirated pdf I once encountered in a search (I've no idea how complete that was).

j2kun10• 13 years ago

My upstairs neighbor has this book, and I've been meaning to steal it from him for a few months now :)

hdivider• 13 years ago

"For the programmer who is truly interested in improving their mathematical skills, the first line of attack should now be obvious. Become an expert at applying the basic methods of proof"

This may be good advice if you want to get into pure mathematics, but huge parts of maths are not pure.

Applied mathematics is very relevant for programmers - especially game programmers. It's not a sin to sacrifice mathematical rigour for the sake of discovery.

It doesn't even have to be a direct map (e.g. actual implementations of calculus or such). Just understanding various bits and pieces in granular detail can help you get precise and testable quantitative ideas for problems.

For instance, a while ago I wanted to implement a kind of smooth up-and-down motion for an object in my game. Almost immediately I thought of equations of this form:

(y position) = (some amplitude)* sin((some frequency)*(time)) + (some value to denote the origin)

This is a first step for some fairly interesting up-and-down motion - the rest is all a matter of finding suitable constants.

Similar things come to mind when you want to move an object smoothly to another position along a line, with a decreasing speed:

position = ((position you want to move to) - (current position)) / (some constant) + (some minimum speed)

This stuff just flashes before your mind if you've done plenty of physics (or 'mechanics' in mathematics) before. What's more, you'll know countless other ways of increasing the complexity of your equations without having to do much trial & error hacking (except to find things like constants and such).

For games, I'm convinced that there's still an incredibly huge variety of mathematical behaviour that has yet to be harnessed for the implementation of actual game mechanics. (Games usually just use some game engine's implementation of only some parts of classical mechanics, for instance.)

Never underestimate the relevance of applied mathematics to real-world programming, even though it doesn't really occupy a spectacularly prominent position in CS-focused courses.

rcfox• 13 years ago

I think the biggest stumbling block for me was all of the stupid tricks involved in the proofs I had to do for school.

"If I add sin(x) to both sides of the equation, then I can use such-and-such identity which makes this equal foo."

If you don't make the correct mental leap, then you get completely stuck and it's really easy to give up.

davorak• 13 years ago

It is sometimes useful for me to break down this type of problem into:

* enough memory to remember the identities * fast enough recall of identities from memory @ * fast enough exploration of possible solutions @

@ Fast enough is when you can do it all within the amount of time it takes for solving the problem to give enjoyment rather then ending a frustration.

Without the above I have to use different, usually more intentional, strategies.

bcoates3• 13 years ago

If this is intended as a defense of Math culture, I think it falls short. He does a good job of pointing out the corresponding problems in the state of programming:

Indeed, the opposite problems are familiar to a beginning programmer when they aren’t in a group of active programmers. Why is it that people give up or don’t enjoy programming? Is it because they have a hard time getting honest help from rudely abrupt moderators on help websites like stackoverflow? Is it because often when one wants to learn the basics, they are overloaded with the entirety of the documentation and the overwhelming resources of the internet and all its inhabitants? Is it because compiler errors are nonsensically exact, but very rarely helpful? Is it because when you learn it alone, you are bombarded with contradicting messages about what you should be doing and why (and often for the wrong reasons)?

The difference is that the CS community recognizes that these are problems; every single thing he's complaining about are open problems being taken seriously and attacked from multiple directions, and there is hope for serious improvement in the coming decades. Anyone who thinks rude snobs, bad documentation, or useless compiler errors are a beneficial is rightly ridiculed as a smug weenie or accused of having an ulterior motive.

By contrast, mathematicians are defensive and complacent about their arcane, non-inclusive notation and communication: "At this point you might see all of this as my complaining, but in truth I’m saying this notational flexibility and ambiguity is a benefit." Look at the litany of problems he just presented. Consider the fact that mathematics is not the only complicated subject that requires complicated, flexible, and rigorous notation. It just isn't credible that the shitty state of mathematical notation is either necessary or unavoidable. The occasional counterexample, where someone with a good understanding of a subject presents it in full rigor without resorting to the usual obfuscation, is a hint of what could be.

If your publications cannot be read without an expert interpreter, they are defective. Hypertext has been around for decades, if you're going to invent your own ad-hoc (or even standardized!) syntax to solve a problem your readers have a right that you document the meaning of your notation.

john_b• 13 years ago

This isn't entirely true. Numerous mathematicians have publicly bemoaned the difficulty of communication between mathematicians in different specialties. So it's definitely not something that is ignored.

The question is what to do about it. We're not just talking about confusion arising from different notation between mathematical specialties (resolving that would be as easy as defining your notation in an appendix), but different, equally valid ways of mathematical thinking.

In some ways, it may be better to think of different mathematical specialties as different programming languages. Proficiency in one will help you, but won't guarantee that you can interpret another. Except that in mathematics, the differences are more extreme. If you have two Turing-complete programming languages, then you have two different tools that can solve the same class of problems. But different fields in mathematics deal with entirely different mathematical objects which require a conceptual instead of notational leap on the part of the reader. It's not simply a matter of figuring out how to write for loops or manipulate strings in the new language. You actually have entirely different ideas in each, and trying to impose some common notational standard among them is fraught with problems.

Mgccl2• 13 years ago

I second this.

In programming languages, different paradigms differs as much as how different field in math. So a more concrete analogy would be one jumping from different paradigms of programming languages. Like someone using a imperative language like Java for their entire life and must learn to read a Haskell program.

skybrian• 13 years ago

Fair enough. But even though there's no hope of coming up with a single programming language for everything, we do have quite good tools for popular areas. It seems like being able to mechanize the error-checking of proofs in certain of the more useful and popular subfields of math might be just what's needed by non-mathematicians?

bcoates3• 13 years ago

I agree with you and the OP that creating a standard notation for all math is problematic. And you're right that there are conceptual difficulties behind the math that are not going to be automatically resolved by clearer communication. But a lot of the difficulties are accidents of culture.

To continue the programming language/mathematical notation allegory, you can use whatever notation you want to describe your program in a Lisp with macros. But you have to actually define what your notation means. Neglecting to provide the definition of the macros would be the equivalent of a logical argument that leaves off its premises or the state of math publications today from the perspective of the poor sap that has to read them (and the mind of the person who wrote it).

j2kun10• 13 years ago

I certainly agree that communication can be improved (and this is one of the main reasons I spend so much of my time improving my technical communication skills via blogging). But I stand by my defense of the need for flexibility. It's extremely hard to explain why without delving into technical details, but there are some times when abuses of notation are honestly much more helpful than they are hurtful. Usually they exist to decrease the amount of clutter in syntax, and they are often stated in words during presentation (and the abuses mostly go assumed only in the higher-end research papers, in fields where the fundamental book every practitioner is obligated to read spells out all of those assumptions in full).

Of course, the more of these you add on the harder it becomes to explain it to a newcomer. So I'm not trying to defend the math culture as being right in all ways. I'm just trying to explain why things are the way they are, and rationally see that there is a good reason to do so (or else why would it have been done?). When you learn the stuff in real life, each of these "abuses" are added on one small step at a time (maybe I learn about one new abuse of syntax every month or two) and so absorbing it all simultaneously is not such a big deal.

j2kun10• 13 years ago

It's kind of like list comprehensions in python. For loops do just fine, but they're such a pain to write all the time and for some cases we just want a clear one-line way to say "add 1 to everything in this list."

DennisP• 13 years ago

Changing the notation is fine. Changing it without explaining what the new notation means is where the problem lies.

Maybe it would help if we put papers online with the ability for anyone to annotate and add explanations, or links to other papers for context.

dxbydt2• 13 years ago

> “The proof is trivial.” This is the mathematician’s version of piping output to /dev/null.

Heh heh. Never thought about it like that. Amazing write-up, btw. As somebody who dabbled in both CS & Math, I'd say the cultures are vastly different. You can spend years, decades even, just teaching undergrad calc courses while having barely 1-2 papers to your name, and yet you'd be considered a legitimate mathematician & get paid & all that. I know scores of math professors who are in that category. With CS, if you don't have productive output in a week, you are just idling & companies will fire you. CS academia isn't a whole lot different...the paper output is a lot more, though much of it is backed by programatic machinery.

As a consequence, a standardized notation/syntax has developed out of sheer pragmatic necessity. Everybody knows what you mean when you say its a hash, or a bst, or an lfsr, or a trie or a monad ( you wish :) Well atleast some of these are standard concepts across all pgmming languages.

With math, you can labor for years in some obscure field ( heh heh a mathematician will kill for that pun!) in which less than ten people know what you mean. At a conference, Hilbert was supposed to have gotten up and asked a speaker "What do you mean by a Hilbert space?". That should tell you something. The syntax is remarkably nonstandard & even simple things like edges, nodes, edge weight, graph correspondingly become arcs, vertices, payload, network depending on where the literature originates from. When I studied a few math papers in grad school, I had a lurking suspicion the author was going out of his way to obscure his thought process & result. With CS papers, you atleast get straightforward pseudocode & you can run off to your favorite language whether php or haskell & give it a shot. There are math texts out there where you pick up one of them, you feel like a complete fool, you pick up the other, you instantly get the point, and they are both talking about the same exact thing !! artin vs dummit & foote vs herstein comes to mind...i got 1000% more out of herstein than the other two.

I agree with the author that in the absence of standard math terminology, I'm afraid programers struggling to learn advanced math just learn bits & pieces necessary to get their job done & move back to fighting their daily battles with git rebase & jira tickets.

bad_user• 13 years ago

I've seen CS papers that use mainstream programming languages directly, instead of pseudocode, so you can just copy/paste it.

However, many CS papers aren't an easy lecture either, even if they talk about something really practical. I remember reading the paper on String B-trees, which is a data-structure for string search optimized for external storage (so you don't have to fit the whole index in RAM) and even if I understood all the concepts, the language was rather obtuse for no reason.

jfb• 13 years ago

Writing prose, even highly structured prose, is an unevenly distributed skill. Thankfully, it's also perfectly possible to get better at it. Would that more academics (lawyers, &c.) would try.

derefr• 13 years ago

> It’s as if the syntax of a programming language changed depending on who was writing the program!

This isn't unique to mathematics; it's the reason it's hard to scale the number of programmers working together on a Lisp codebase. Each programmer writes their own notation (macros), and ends up down their own rabbit-hole of a sub-language that nobody else can immediately understand.

DanBC• 13 years ago

Sorry, but Helvetica Neue and font weight 300 make the page unreadable for a number of people.

jholman4• 13 years ago

There's a lot of interesting and valuable stuff in this post (enthusiastically upvoted!), but I think there are a lot of errors, too.

First of all, let's acknowledge that many (the great majority of?) programmers have little or no use for math, and know it, and are correct. That's totally fine. I think danso's comment captures this well, with references to UI design, domain knowledge, engineering practices, etc, and dxbydt similarly mentions "git rebase and jira tickets". So let's assume we're restricting ourselves to programmers who do have a use for math (and hopefully know it).

FIRST PROBLEM

TFA's introductory claim is apparently that there are programmers who say "math is useful, I want to use it, I don't want to read or write proofs", and that this is ludicrous. It's absolutely not ludicrous.

Yes, a working mathematician's job is to explore and understand the space of mathematics, and yes, that is inextricably bound up in proof. And I would suggest that learning more about math proof is a soul-enriching use of anyone's time (in the same broad vein as learning Latin). But techniques of calculation are one of the valuable outputs of mathematicians, and it is reasonable and effective for a person with a task to say "I would like to have more powerful techniques of calculation, and I don't want to learn proofs".

For example. Certainly it is possible to learn how to do Gaussian elimination, and use it to solve systems of linear equations, to good effect, without being able to prove that elementary row operations preserve the system. Maybe it helps? Maybe it doesn't? If you want to invent a new technique, I imagine that in that case being able to prove the correctness of the existing technique probably helps a great deal. But what's unreasonable about the person who says "I have these equations, I need to find the values (if any) of these variables here that make all the equations true, and I'm too busy to learn any proofs today (and too busy to write any proofs ever)"?

For a simpler example, we used arithmetic just fine before ZFC.

SECOND PROBLEM

Maybe this is actually two problems, intertwined. And maybe both of these problems are quibbling.

Primus, he says "Mathematics is cousin to programming in terms of the learning curve, obscure culture, and the amount of time one spends confused". I think this is highly questionable. Mathematics is far harder and more baroque than programming, not least because as a discipline it's nearly one hundred times older.

Secundus, after acknowledging the insanely annoying tendency of math papers to make up their own notation without ever defining it, he claims that the non-mathematician's confusion is comparable to the non-programmer's confusion at for(;;);. No, it's not comparable. The math case is hard because someone failed to tell you what you need to know, and no one wrote it down anywhere. The for-loop is only hard if you did the opposite: failed to read what was actually written down (e.g. in K&R) and took someone's word for it that the standard and general form of the for-loop is for(int i=0;i<LIMIT;i++){ /do stuff with i/ }

My claim here is that the learning math - even just the calculation techniques - is genuinely inherently hard. Partly that's because it's so overwhelmingly useful that if it wasn't, we'd already know it; clever students know more calculation technique by age 15 than most professional programmers know about programming. But on the other hand it's also contingently hard, because mathematicians aren't generally very interested (nor incented) to export their product so that outsiders can make use of it. Which I guess isn't so terrible; once someone else labours mightily to learn all that very-hard math, you can hire her at an exorbitant rate to make use of the calculating techniques she has learned. Or if you want to get that benefit without paying the exorbitant rates to outsource the math, you can roll up your own sleeves. Good luck!

btilly• 13 years ago

I agree.

I would say that people who have not spent a lot of time playing with math are not going to have developed the kind of intuition that will let them understand the subject, or see how to apply it to novel problems. Also there is a dearth of math material aimed at people who don't already have a strong math background. Therefore people who you find who can handle math usually also have a background in proofs.

But it is perfectly possible to explain useful and subtle math points to people who can think well, even though they may lack math background. However as I've found writing articles like http://elem.com/~btilly/ab-testing-multiple-looks/part1-rigo..., it is deceptively harder to do than I'd have thought. Unless a mathematician has specific reason to believe that it will be read by people who do not normally choose to read math, they can't be blamed for not making that effort. (Incidentally the next installment in that series should arrive next week.)

j2kun10• 13 years ago

OP here. My point was that if you want to be better at math, then you need to get comfortable with proofs. There are lots of instances of using math for useful things, but if you don't do proofs you won't get any better at understanding mathematics. There are a lot of times when people want to know more about a particular mathematical fact (they genuinely want to understand why it is the way it is) but the minute you tell them, they stop wanting to hear it. This is what I think is ludicrous.

The for loop analogy may be a bit of a stretch, but the reasons for it are as wacky as for the mathematical syntax of inventing or dropping indices. Literally the C99 standard says that if a test expression is missing in a for loop, it's replaced by a nonzero constant. In my mind as a programmer and as a mathematician, that's as arbitrarily confusing as dropping the limits of an indexing variable in a summation. They both just allow you to write/type less and have the expression still mean something.

dxbydt2• 13 years ago

>you need to get comfortable with proofs

True, but this is simply not the point of programming/CS. Back in grad school in CS, we used leiserson. Leiserson has proof of quicksort! I mean, that's fucking crazy. He mathematically derives amortized runtime of quicksort.I really loved leiserson. But even my professors hated it...they'd say, lets skip over the proof section, its not important. Nowadays people use Skeina. Skeina is like a storybook. It tells you - do this, get that, if you want to know more, read leiserson! So skeina basically punts. This is why everybody loves skeina, because no proofs. Just straight algorithms that can be translated to code. Skeina is slightly better than Numerical Recipes cookbook, in that it tells you what to do & points you to the why, but doesn't tell you the why. But math is all about why.

Now look at a typical grad math course. They love artin. artin is full of proofs. Well, show me how the direct product works out. Ok, install GAP, & bang out some 10 line script, & boom, done! No refactoring, no nice variable names, no explanation...its as opaque as it gets. You just call the function & it'll do its thing. You want to know why ? Well, that's programming, & we are mathematicians, why do you want to know those things, its some messy bullshit C code, they have this thing called data structure in programming, you have to know about pointer, reference,...dude just type the script & get back to Artin.

So each culture actively repels the other, even though they are massively co-dependent. Even department level politics are driven by that rivalry. Math depts are money losers net net. CS otoh brings in tons of dough. Guess who teaches all the difficult discrete math courses in CS ? Yeah, its the math profs! Crazy, I tell you.

EvanMiller• 13 years ago

> if you don't do proofs you won't get any better at understanding mathematics

There is a vast world of mathematical understanding available to programmers who don't like proofs. I think there is some miscommunication here about what constitutes mathematics. Mathematicians tend to think of the art of proving things, whereas engineers (and programmers, and scientists) think of equations and how to solve them. I think mathematicians actually do a disservice to engineers by trying to beat them over the head with real analysis and set theory (which is why most physics and engineering departments end up having to teach their own mathematical methods courses).

The "foundational" knowledge of mathematics -- epsilons, deltas, Cantor sets, Banach spaces, etc. -- is irrelevant to almost all practitioners. Newton, Taylor, Euler, Gauss, Laplace, Bessel, Maxwell and the gang got pretty far without it. When programmers say they want to learn math, they mean they want practical knowledge: interesting functions, how to compute them, and their properties that can be applied with utility. They don't want -- or need -- to spend time poring over proofs.

jholman4• 13 years ago

I suggest that your point is that if you want to be a better mathematician, you need to get comfortable with proofs. Sure, well, so what?

If a programmer tells you they're interested in math, and they don't want to learn proofs, it's possible that they're saying the ludicrous thing that your article claims they're saying... but it's more likely that they're saying "I want to use the calculating techniques of math". Speaking for myself, I have a very modest interest in getting better at proving, but significant interest in getting better at calculating.

As for the for-loop, the reasons may be wacky (wackier, I admit, than interpreting for(;1;); ), but there's something fundamentally different between the for-loop and the summation. In the for-loop case, everything you need to know to interpret that code is written down in an obvious place.

johnbender• 13 years ago

"Secundus, after acknowledging the insanely annoying tendency of math papers to make up their own notation without ever defining it"

"The math case is hard because someone failed to tell you what you need to know, and no one wrote it down anywhere."

I agree with a lot of what you're saying but I don't agree with these two assertions.

I'm working developer and have been for 6 years (Ruby, JavaScript). About a year ago I wanted to learn a bit more about how math (logic in particular) gets used to prove interesting things about programming languages. It was a very short search for books/material on the subject that were approachable for me and contained plenty of fully explained notation (eg, inference rules in Pierce's Types and Programming Languages).

After convincing myself the math was nothing to be afraid of, approaching papers was difficult but not at all impossible. It really boiled down to putting in the time to fill in the gaps in my knowledge through reading referenced papers and taking notes. I have yet to come across a paper that does not provide, through references or explicit notation semantics, all the information necessary to understand the content.

Is doing the reference reading easy? No, it's very time consuming. That doesn't mean that these papers are deliberately evasive in their definitions.

[edit] If it's not clear I'm reading CS papers mostly to do with type theory. It may be that things are very different for pure mathematics papers.

jholman4• 13 years ago

I agree with what you said, except I'm not totally thrilled with how you took one and a half sentences I wrote out of context, incorrectly claimed they were in error, and then wrote a bunch of unrelated stuff that had no bearing on them, as if it was a disagreement.

But, yeah. Textbooks are far clearer for a novice than papers are, and hard work is more effective than whining. That's right!

stiff2• 13 years ago

Majority of mathematics isn't at all concerned with calculation techniques, this is a fundamental misunderstanding many people get from school, mathematics is the study of concepts and relationships between them. If you do not understand the proofs, you do not understand the concepts and you will not be able to apply them. People who learn mathematics as calculation are the ones that most often claim that have no use for mathematics, where in fact they did not really learn any. Proofs are also crucial in Computer Science, for example you do not really understand recursion until you see how it is related to an inductive proof. The widest areas of fruitful application of mathematics do not arise simply from "calculation techniques", whatever that would mean.

aaronblohowiak• 13 years ago

Plenty of math has nothing to do with quantities or measurements. Relational algebra and type theory come into mind as particularly useful for workaday programming.

plainOldText2• 13 years ago

I know this is a bit off topic, but does anyone know of a source (book, video, website, etc) which you could use to review the math from 0 to Calculus in one day? A sort of big picture overview of how math evolves from simple to complex. I've always wanted to do it, just to keep the math in my head fresh.

Permit• 13 years ago

Khan Academy might be exactly what you're looking for. It has videos ranging from addition all the way through to second year undergrad calculus. I've found the most important (and difficult) part of learning math is to force myself to practice.

Much like programming, mathematics is something best learned actively. Simply reading/watching videos rarely gets me anywhere.

nborwankar3• 13 years ago

I'll do an outline here - then follow each outline item to Khan Academy for steps and details

* numbers - natural numbers, integers +ve/-ve numbers addition subtraction

* numbers - rationals, reals, multiplication/division

* numbers - fraction and decimal representations - addition/subtraction/mult/division of fractions and decimals

* sets and functions - mappings, single valued and multivalued, domain and range

* powers and exponents

* algebra - functions and equations linear equations, quadratic equations, simultaneous equations

* geometry and trig

^^ althat was pre-calculus and I may have condensed the steps a lot

* limits <-- this is basic stuff for calculus - all of calculus is about starting with finite expressions and then taking limits as something becomes infinite or infinitesimal.

* continuity of functions

* derivatives

mean value theorem

* integrals

integration and differentiation is often taught in that order but integration is much easier to understand intuitively since areas and volumes are more concrete and tangible velocity as a vector tangent to the direction of motion is less tangible to most.

After this point you take a sidestep and start the process again at a higher level with * sequences and series (limits on steroids) * linear algebra (simultaneous equations on steroids) * multivar calc (calculus on steroids)

Then there's complex analysis, differential equations, partial differential equations and so on.

Note here I am taking the usual more 'applied' approach to math which is tangible. The more 'pure' approach' - abstract and much harder for most is to go

logic and number systems real analysis algebra complex analysis measure theory / probability functional analysis ... which is what you'd do if you were on a pure math rather than an engineering math track.

Hope that helps.

plainOldText2• 13 years ago

Wow, thanks. You took the time to write all that. That is awesome.

gambler• 13 years ago

The argument of the author sounds like an argument of someone who is addicted to writing ugly Perl one-liners. Many, many, complicated and co-dependent one-liners. "But I need this expressive power to do things!" Perhaps you primary concern should be about explaining things you did, rather than doing them. Especially if your job it to explain something to people.

anonymoushn• 13 years ago

This is good advice for programmers who want to become better mathematicians. That seems to me to be distinct from wanting to be better at applying mathematics to programming problems. In the same way that wanting to be a better gardener is distinct from wanting to understand the aspects of gardening that will help you write programs relevant to that domain, with the obvious caveat that mathematics-related knowledge is relevant to a much wider range of problems people try to solve using software than gardening-related knowledge.

In school I appreciated the sorts of classes that stressed proof the most, like topology or real analysis 2, because they made me a better mathematician. When programming, I am frequently glad that I took some upper level math class, but generally for reasons unrelated to whether I've actually done mathematics in that area. For instance, I could take advantage of the equivalence of the square metric and the Manhattan metric in a 2-space (up to a 45 degree rotation of the space) regardless of whether my topology class had emphasized proofs. The attitude portrayed as absurd in the article ("I don't want to write or read proofs, just use theorems") doesn't seem that far off from one that would work pretty well. The main issue with that attitude is the belief that a large body of theorems can be understood without understanding any of their proofs. Programmers don't need to be able to write proofs that aren't extremely trivial, but people who want to apply ideas from some area of mathematics probably need to understand it in a deeper way than memorizing theorems.

nborwankar3• 13 years ago

Re: the original post IMHO, I find it has many logical flaws and while it is trying to draw attention to the fact that programmers need to learn math it may end up scaring people away.

Background: I have a BS in EE from IIT Bombay, An MS in Applied Math from USC and an almost PhD(everything but thesis) in Pure Math.

I also loved teaching math as a TA and was able to teach engineers multivariable calculus and differential equations so that they lost their fear of it.

But for the last 20 years have been in the database business. I am very enthusiastically returning to math in the form of data science in the last year.

I find that especially today it is necessary for math to be made accessible without having to prove theorems - that's for professional mathematicians.

Programmers use hardware often without knowing diddly-squat about semiconductors. And that's just fine if you don't want to or need to. We drive cars, use mobile phones and all sorts of machinery without needing to know anything about internal combustion or the CDMA algorithms or even how theory of relativity figures into GPS.

I think the insistence on setting a high bar for people to learn/use/apply math is unnecessary mystification and obscurantism.

Math is beautiful, useful and powerful. If you know math you should be wanting to simplify it's teaching and instruction, thinking hard about making it useful for the non-mathematician and also making it easier to access in a breadth-first fashion rather than the depth-first fashion in which it is currently practiced.

I know many people, engineers and mathematicians who believe that simplification is "dumbing down". I disagree vehemently.

Today if you are a programmer and want to learn math I have a few of suggestions

a) If you want to apply it in engineering take a couple of classes on Coursera/Udacity - it will be a slow an steady way to re-activate your math neurons (it's also insurance against Alzheimers to keep learning new and hard stuff).

b) Learn some statistics, learn to use R. (Coursera again)

c) Learn some linear algebra, minimal, then learn MATLAB (Andrew Ng's Machine Learning class on Coursera or a similar one by AbuMustaffa (sp?) of Caltech)

d) If you absolutely don't want to learn a new language then look at Machine Learning by Marsden which uses Python and then pick up NumPy/SciPy and look up the exciting stuff that's happening with the IPython Notebook.

e) If your focus is on pure CS the take the Algorithms track(s) by Roughgarden (Stanford) or Sedgewick (Princeton) on Coursera

This way you dont have the notation issues - R has a syntax, MATLAB has a syntax just learn it. Python hopefully is note even an issue.

Start using your math brain a little at a time - absolutely no proofs required - and THEN once you see the beauty and want to learn more and want to learn why, THEN dig deeper with the fundamentals. If you don't that's fine too - use it, put its power to use and become a better developer and engineer. Math is not trivial but it's not as hard as it's made out to be. There's way too much noise in the channel.

Good luck.

(feel free connect with me on twitter if you want to get more help (@nitin))

j2kun10• 13 years ago

I certainly don't think programmers need to learn math. The vast majority of programming requires no math, and there are tons of out of the box theorems and algorithms that can be used successfully by programmers to do wonderful things.

I am just pointing out why math can scare programmers away. The first step in overcoming that fear is to identify it.

And the fact is that programmers do need to tweak the algorithms they use. Your analogy with a car works perfectly: can you expect someone who only knows how to drive a car be able to convert a sedan into something that can go off road? Then how can you expect someone who only knows how to call a function to alter its contents to suit their particular application?

I also think that I haven't set a high bar. The basic four methods of proof is quite a low bar (this could, and should be taught in high school mathematics, and it baffles me why it's not). Learning common syntax will come as you learn.

aangjie• 13 years ago

> The basic four methods of proof is quite a low bar

+1.

Am just a programmer, but have always regretted my falling out with math during my college...

Have never heard of proof by contrapositive, and looking up, i am still baffled as to significance or point of separating it from proof by contradiction.

I look forward to more posts and hope you decide to write about basic methods of proof sometime in the future.

nborwankar3• 13 years ago

It's true you may have set a low bar re: proofs, I just don't think any formal proofs are needed at the first step.

Learning about numerical convergence and computer arithmetic so you can understand why your MATLAB Linear Algebra program is not giving a useful result is far more valuable and doesn't need any proofs. It needs a knowledge of how to use theorems and results and what the conditions are for a result to be applicable. But there is no need to do the proof. Especially since many many proof sare quite idiosyncratic and give no help in suggesting how one thinks like that.

Using the analogy of the car I am talking about teaching someone who rides a bike to drive a car. My explicit non-goal is being a mechanic of any kind. So your example of modifying your car is a strawman. That's exactly what I assert a student-driver doesn't need and *shouldn't have to do. It is going to be pretty much useless in actually learning how to steer and how to follow traffic rules. One is about proper usage, the other is about understanding internals. Sorry I still disagree and reassert that proofs are no use at all and a huge distraction for programmers trying to learn math. There is so much valuable and useful work a programmer can do even using existing mehods without ever modifying them. Especially with special purpose languages like R and MATLAB - the underlying algorithm are very mature and a first time user is not expected to modify them.

Quite separately I think it's extremely valuable to learn/relearn the foundations of math but this is for anyone and not specific to what a programmer needs to do useful things with math. Yes and it should be taught in high school math. It's not taught bec the teachers didn't learn it that way and so it goes ...

alok-g• 13 years ago

+1 for the book recommendation alone [1]. I finally see a book on set theory that explicitly states that the term "set" by itself is undefined.

Reading the first chapter preview at Amazon, I already see the need for a human to explain something to me:

Two sentences from the book [1]: "A possible relation between sets, more elementary than belonging, is equality." and "Two sets are equal if and only if they have the same elements". These already seem to be contradicting each other -- how can equality be more basic than belonging if the former is defined in terms of the latter.

While I agree with OP on the difficulty I face when learning advanced mathematics (notation, generally tacitly assumed by the authors), one thing that also bugs me is use of human language (!) as a part of the proofs. How do I know that the proof is correct and is not impacted by something fundamental about the language itself? Merely via inclusion of a human language in the proof, a lot more axioms "may" have been included than what meets the eye. Probably not, but I often find it hard to convince myself of this.

[1] www.amazon.com/Naive-Theory-Undergraduate-Texts-Mathematics/dp/0387900926

xyzzyz• 13 years ago

Axiom of extensionality states that the sets A and B are equal if and only if they have the same elements.

The point here is this: if A and B are the same set, then whenever some x belongs to A, it also belongs to B, because B is just a different name for A. Similarly, if something belongs to B, it also belongs to A. So, the implication "A is B implies A and B have the same elements" is just a logical tautology, axiom of extensionality does not say anything new in this case.

Now suppose you have two sets, C and D, such that whenever something belongs to C, it also belongs to D, and when something belongs to D, it also belongs to C -- in other words, they have the same elements. Can we conclude that C is the same set as D? Without extensionality axiom, no. That's why we usually assume axiom of extensionality, which tells exactly that whenever sets the same elements, they are the same, so that we don't end up with a weird situation where we have two sets that are indistinguishable with set theory, yet are different.

compee• 13 years ago

I work with mathematics applied to various applied computer science problems with a couple startups.

If anyone is interested in learning a bit of mathematics to structure programs (for real-world usage, even!) feel free to let me know what you'd be looking for. I'd love to help!

ilaksh• 13 years ago

Math is just obfuscated code that you have to run in your head.

Uncompetative• 13 years ago

The Hitch-Hiker's Guide to the Galaxy already parodies Maths:

http://hhgproject.org/entries/bistromathics.html

Clearly, Douglas Adams spent a lot of time in Bistros, as the second review on this site echoes his sentiments about the odd behaviour of waiters and nonabsolute numbers such as time of arrival:

http://www.yelp.co.uk/biz/giannis-italian-bistro-san-ramon

xwowsersx• 13 years ago

I feel like so much of this discussion is way too general. It's impossible to debate how/how much math is needed for "programming" without speaking about a particular (programming) domain and the subset of math, if any, needed for it.

In other words, "no math is needed" and "you need proofs" are probably both true. There's some realm of programming in which no math is needed, ever. And then there's other programming that requires lots of math. But the math required, might be linear algebra in some cases and some other specific branch of math in another area. This fact is a just a reflection of the fact that "programming" is an extremely wide area. There are some realms of programming, where the "programming" aspect of it is virtually subsidiary to the math. The programming is just automating or systemizing some math which is the real work is taking place.

So, my take away from seeing this discussion and similar comments so many times is that everyone can agree that math is an enriching subject. However, from the standpoint of its relationship with your programming, going really deep into one particular area of math without a clear idea of if/how it's going to help you accomplish something previously unattainable is probably not the best use of your time.

Why there is no Hitchhiker’s Guide to Mathematics for Programmers

Related Stories