Information Theory, Inference, and Learning Algorithms (free ebook edition)

One of my favorite machine learning textbooks, if you can call it that. It's a little oddball though. For any topic, it's extremely interesting and insightful, though usually not comprehensive enough to rely on it as a reference.

linhir• 17 years ago

For anyone interested in learning algorithms (and not just free books), I would also suggest Pattern Recognition and Machine Learning by Chris Bishop. Not free, but worth while. http://research.microsoft.com/en-us/um/people/cmbishop/prml/

tel• 17 years ago

I've been reading through this bit by bit the last few days, mostly to get a handle on how to implement MCMC for Bayesian posteriors, and I have to say its fantastically written. I wouldn't call it comprehensive or unbiased, but it sets up the infrastructure of interrelatedness between noisy channels, information theory, statistics, and machine learning pretty much as effortlessly as possible.

Note: I'm buying it entirely because it has wide margins. Many of the calculations he outlines deserve to be worked out in full. Wide margins are absolutely the most important publishing concern for a math/science/engineering-based text.

npk• 17 years ago

If you want to learn how to implement MCMC I recommend:

Bayesian Logical Analysis Physical Sciences by Gregory

Gregory's book explains a lot more of the engineering (autocorrelations, step size jumping, etc..). Even better, it discusses how to perform model selection using a clever annealing technique. Though model selection may not be of interest to you.

ps - MacKay's book is my nightly reading, so I'm not dissing MacKay :)

pz• 17 years ago

I've said it before and I'll say it again: this book is great. This books provides a really great foundational understanding of ML, not the toolbox approach of other textbooks. I think the exposition of coding theory is especially nice compared to, say, Cover and Thomas.

MaysonL• 17 years ago

God, this Mackay guy gets around: Sustainable Energy, this, and the Dasher project.

tejus• 17 years ago

Fantastic book. The problems are interesting, and nicely bring out the connections between topics that would on the surface, seem to be disparate.

Cover and Thomas is more textbookish, and in some ways, more detailed. Personally, I'd read this first, and then take on the interesting topics in Cover and Thomas.

I read a lot of math books, and I'd put this right on top along with Needham's 'Visual Complex Analysis'.

Information Theory, Inference, and Learning Algorithms (free ebook edition)

Related Stories