Can anyone recommend good books, articles or essays that introduce machine learning?

https://news.ycombinator.com/item?id=79359

by jkush • 19 years ago

27 23 19 years ago

news.ycombinator.com

I intend to find out for myself but I'm curious to know how steep a learning curve you've found it to be.

DaniFong• 19 years ago

A nice read early on would be "Elephants don't play chess" but Rodney Brooks.

http://people.csail.mit.edu/brooks/papers/elephants.pdf

Tichy• 19 years ago

I stopped reading on page 3, because it seems to me that the "physical grounding hypothesis" is complete bullshit. Where is the "physical grounding" aspect if I want to build a search engine for the web? It's all just information.

Not saying that getting physical can not yield interesting results, but to state that it is the only possible way to program intelligence seems just wrong.

pcowans• 19 years ago

Information Theory, Inference and Learning Algorithms by David MacKay is excellent, and freely available online.

http://www.inference.phy.cam.ac.uk/mackay/itila/book.html

mriley2• 19 years ago

I'll second this recommendation - I bought the printed copy, and I'm constantly going to it for reference. The fact that it's available for free is just an added bonus.

I would also suggest Elements of Statistical Learning: http://www-stat.stanford.edu/~tibs/ElemStatLearn/

As well as Duda, Hart, and Stork's Pattern Classification: http://rii.ricoh.com/~stork/DHS.html

snifty• 19 years ago

This is a great book, but it's definitely not introductory. I think Segaran's "Programming Collective Intelligence", mentioned above, is the best first bet, primarily because it's fun. And the code is good.

wastedbrains• 19 years ago

Artificial Intelligence A Modern Approach SE, by Stuart Russell and Peter Norvig.

I am a big fan of Peter Norvig in particular, he has a ton of great essays and code available online: http://norvig.com/

jmzachary2• 19 years ago

Machine Learning by Tom Mitchell for a serious academic book.

Programming Collective Intelligence by Toby Segaran for a practical approach in Python.

jgrahamc• 19 years ago

I agree on the Mitchell book.

If you are looking for a light introduction to naive Bayesian text classification then there's always the piece I wrote for DDJ: http://www.ddj.com/development-tools/184406064

John.

davidw• 19 years ago

The "Programming Collective Intelligence" book is pretty good in terms of a practical introduction to a variety of techniques.

queensnake3• 19 years ago

I disagree wrt Mitchell; too dry and out-of-date. In its place I'd recommend the Norvig book which covers everything Mitchell does (IIRC) except 'Probably Approximately Correct' (PAC) learning. And, it's more up-to-date (Support Vector Machines) and is much more readable. Also, I haven't read it except a flip-through at a bookstore, but Christopher Bishop's one I'd look at (perhaps the Amazon comments).

neilc• 19 years ago

I've seen the Norvig book recommended by others on this site for ML, but I don't understand why: the Norvig book is an AI book, not an ML book. It only has a few chapters on learning, and IIRC much of that is on reinforcement learning. Obviously the Norvig book is very well-written and is good background for learning about ML, but I don't think it is a sufficient ML book as such.

As far as ML goes, I found the "Programming Collective Intelligence" book to very readable and practical, but very light on the theoretical foundations (which is intentional, of course). I've got a copy of the Witten ML book ("Data Mining: Practical Machine Learning Tools and Techniques"), but to be honest I haven't gotten much from it yet, either: it doesn't seem to discuss SVMs in any detail, nor random forests or neural networks. But I haven't really dug into it yet.

jmzachary2• 19 years ago

If you can read and understand Mitchell's book, you will have a very good foundation for understanding modern ML techniques. The poster was looking for references to introduce him to the field.

at• 19 years ago

books about machine learning applications/tools: - "Programming Collective Intelligence" - http://www.oreilly.com/catalog/9780596529321/ - "Data Mining: Practical Machine Learning Tools and Technique" - http://www.cs.waikato.ac.nz/~ml/weka/book.html

books about machine learning background/theory: - "machine learning" - http://www.cs.cmu.edu/~tom/mlbook.html - "learning and soft computing" - http://cognet.mit.edu/library/books/view?isbn=0262112558

For a simple-to-use (Python-based) machine learning tool/API check out Orange: http://magix.fri.uni-lj.si/orange/

cschmidt• 19 years ago

If you want a fairly easy read without too many equations, try:

Data Mining: Practical Machine Learning Tools and Techniques (Second Edition) http://www.cs.waikato.ac.nz/~ml/weka/book.html

Which goes nicely with the Weka open source ML toolkit http://www.cs.waikato.ac.nz/ml/weka/

(although it is a good read without the toolkit)

If you want a bit more math, I really like the recent (Oct 2007) book:

Pattern Recognition and Machine Learning by Christopher M. Bishop http://www.amazon.com/Pattern-Recognition-Learning-Informati...

It is nicely self contained, going through all the stats you'll need.

mriley2• 19 years ago

I didn't particularly like the WEKA book, but Bishop's book is excellent.

If you're interested in introductory data mining stuff, I would recommend Tan's Introduction to Data Mining: http://www-users.cs.umn.edu/~kumar/dmbook/index.php

toddh• 19 years ago

The machine learning blog at http://hunch.net/ is a good resource.

alec• 19 years ago

Sutton and Barto's book on reinforcement learning is available online at http://www.cs.ualberta.ca/~sutton/book/the-book.html, and is probably the definitive work on the subject.

queensnake3• 19 years ago

I wish they'd make a new edition; but, probably, the field is moving too fast.

jorgeortiz852• 19 years ago

Check out the lecture notes for CS229, Stanford's class on Machine Learning. The class has no textbook, so the notes are fairly comprehensive. (Though the math notation can get a bit intense...)

http://www.stanford.edu/class/cs229/materials.html

jorgeortiz852• 19 years ago

Also, the notes assume a reasonable knowledge of probability theory and linear algebra. If you're unsure, you might do well to review those topics before approaching machine learning (or at least keep good references handy). (Edit: the link above has Section Notes that review probability theory, linear algebra, and convex optimization. You might find those useful.)

And it's not an easy course. Don't be discouraged if the problem sets seem impossible. (They very nearly are.)

marcus• 19 years ago

ML is a very interesting field. Great resources can be found on: http://www.aaai.org/AITopics/html/machine.html http://aima.cs.berkeley.edu/ai.html

Start by reading and completely understanding a few of the simpler machine learning algorithm (Backpropagation neural networks for example), how&why they work. and continue from there.

ivankirigin• 19 years ago

Mitchell, Norvig, and "Probabilistic Robotics" by Thrun.

Can anyone recommend good books, articles or essays that introduce machine learning?

Related Stories