Ask HN: Good books on machine learning?

https://news.ycombinator.com/item?id=478789

by trominos • 17 years ago

13 7 17 years ago

news.ycombinator.com

I'm trying to learn about methods of machine learning. Do you guys have any suggestions for books or sources I should use?

pmbouman3• 17 years ago

Try "The Elements of Statistical Learning" by Hastie, Tibshirani and Friedman. Lots of math but an outstanding introduction.

mstoehr• 17 years ago

There really isn't very much available on the practical side. So if you are looking to implement algorithms I suggest that you make use of the machine learning at ocw.mit.edu

Alternatively, if you want a good dose of a theoretical explanation of algorithms currently in use I highly recommend "Pattern Recognition and Machine Learning" by Christopher Bishop. It is definitely the best machine learning (and statistics) textbook that I have ever come across.

bravura• 17 years ago

I second the nomination of Bishop. It is the standard text. It is only two years old, and Bishop will teach you machine learning the way that the field practices it nowadays. In my lab of fourteen people, we must have six or so copies of Bishop.

I don't understand what is impractical about Bishop. If you are looking blindly to use an off-the-shelf machine learning implementation, that's one thing. Machine Learning has been described as the study of bias. If you want to understand when to pick certain techniques, and develop appropriate biases, then read Bishop.

"The Elements of Statistical Learning" by Hastie, Tibshirani and Friedman gives more of a statistician's approach. The treatment is simply less broad, and also more dated.

You can also look at Andrew Ng's video lectures: http://www.youtube.com/watch?v=UzxYlbK2c7E He is very well-respected in the field. For certain students, watching a lecture may be preferable to reading a book.

pmbouman3• 17 years ago

In terms of "datedness," I would point out that the second edition is now out:

This major new edition features many topics not covered in the original, including graphical models, random forests, ensemble methods, least angle regression & path algorithms for the lasso, non-negative matrix factorization, and spectral clustering. There is also a chapter on methods for ``wide'' data (p bigger than n), including multiple testing and false discovery rates.

I'd be interested to hear what you mean by "simply less broad" as compared to Bishop's book, which (from having flipped through it) looks pretty comparable.

pmbouman3• 17 years ago

A few other things (sorry, not to snipe too much! :) )

-I'm skeptical of the idea of a single "standard text" in such a fast-moving field. New machine learning techniques appear constantly and are often documented online years before they appear in books. Some computer scientists say they prefer conference proceedings over academic journals because the latter take so long.

-Further, I'm not sure that the goal of any text should be to cover topics X, Y and Z in any case, which doesn't seem possible for a book to do. What does seem feasible is to set up a framework for analyzing the performance of different techniques. So I'd like to hear a comparison of how Bishop does that vs. HTF.

-You're of course correct that HTF takes a statistician's POV on the field - the authors are all professors of statistics at Stanford. They are also accomplished - Friedman was a co-author on CART, for example. I would instead ask the question: what can you get out of the book and the framework it offers?

-I think that part of the framework in machine learning is to think about bias AND variance, and how to trade them off successfully. This is an important part of model selection, for example.

martian• 17 years ago

A very readable introduction to machine learning and recommendation systems is Programming Collective Intelligence, by Toby Seagram (O'Reilly). Worth looking at if you're just getting your feet wet.

earl• 17 years ago

Here are a set of links:

http://www.vetta.org/recommended-reading/

I'd second the recommendation of Bishop if you can hack the math, and also Elements of Statistical Learning, though I wouldn't attempt to learn techniques from the latter so much as look at a very interesting mathematical take on them.

Ask HN: Good books on machine learning?

Related Stories