Tools to get started in machine learning

http://k2company.com/blog/2012/09/06/toolbox-for-learning-machine-learning-and-data-science/

by nwenzel • 14 years ago

89 35 14 years ago

k2company.com

mlmilleratmit• 14 years ago

Solid summary of some powerful tools, no wonder python is the new default for academic data analysis.

taw92• 14 years ago

Word. Given Cython, Jython, etc.... this list keeps going (I use Weka in Python).

merlinsbrain• 14 years ago

Can I get your email address? Check my profile for mine please. Thank you!

FrojoS• 14 years ago

Andrew Ng, in his machine learning class [1] urges people do use Matlab instead of Python, because in his experience people develop faster with Matlab than wit any other tool/language.

Personally, I am experienced with Matlab but not so much with Python, so I am not able to judge. I definitely hate the fact that Matlab is proprietary and partly closed source. Also, I think Python syntax is much more pretty while Matlab is not even designed to be a real language. But alas, it comes with very powerful functionality out of the box.

Note, that the OP mentioned in the introduction that he had no access to Matlab over his employer or university and hence dismissed it.

[1] https://www.coursera.org/ml (one of the first videos)

otoburb• 14 years ago

It's too bad the OP didn't find Octave, which is an opensource Matlab clone that was also used in Andrew Ng's ML course.

mmaul3• 14 years ago

If you have taken Andrew Ng's Machine class the handwriting recognition system that is mentioned in the course was implemented in Lush. I think the original code is is even included in the demos distributed with Lush.

iskander2• 14 years ago

>Also, I think Python syntax is much more pretty while Matlab is not even designed to be a real language.

If you're only coding the core of an algorithm (rather than a full-featured library with lots of plumbing), and your logic fits naturally into Matlab's native array operators, then using Matlab is a joy.

0ren2• 14 years ago

Some good points for comparison:

http://www.scipy.org/NumPy_for_Matlab_Users

IMHO, it's best to prototype in Octave and then build in python. I find that the Matlab/Octave syntax is too focused on linear algebra, so it's better for small prototypes (and for people coming from non-SW fields). For big projects, I prefer the 0-based arrays, more than one function per file, and all the rest of the python goodies. I estimate that 70% of my time is usually spend preparing the data (e.g. parsing xml, or some other files, etc), for which I find python more suitable.

In fact, I usually work with them side by side, testing ideas in Octave, then implementing these pieces into a large python project.

Edit: this has also been discussed here before, e.g.

http://news.ycombinator.com/item?id=363096

http://news.ycombinator.com/item?id=689183

wookietrader• 14 years ago

> Andrew Ng, in his machine learning class [1] urges people do use Matlab instead of Python, because in his experience people develop faster with Matlab than wit any other tool/language.

Never trust academics when it comes to programming. :)

Seriously, Matlab might be a tiny bit better for scientific programming than Python. But: if you start building an eco system around your machine learning code (distributed evaluation of models, email reporting of results, web reporting of results, online tracking of training progress, data base related things, web services for other people, proper documentation, ...) you are happy if you chose python.

Also, there is theano for python which has auto diff, transparent GPU/CPU use and symbolic optimization. It makes your life easy if you are using complicated models.

mmaul3• 14 years ago

Lush is an excellent platform for machine learning. There are bindings to gnuplot ,opencv, lapack, gsl, an optimization library for gradient descent, a machine learning framework, a nerual network simulator.

It also has very nice matrix and vector manipulations features built in to the language and is very easy to bind to C code.

iskander2• 14 years ago

Some people really do seem to get a lot done in Lush, so I'm not discounting its utility, but the language is sort of a mess. I took Yann's class and gave up in frustration after a few homeworks. I was very happy working in Matlab and relieved to to never see a 'bloop', 'eloop', or whatever-loop again.

shreeshga• 14 years ago

If you are going to use python for ML. Use the python package from Entthought [http://www.enthought.com/products/epd_free.php]

It has most of the libraries, out of the box.

rrosen3262• 14 years ago

(Author) - Nice tip. Thanks - I'll update my list. (Wish I had known at the start of my learning!)

rm999• 14 years ago

While I think Python is great, I think the author discounts R too casually. I share his frustrations about R, but what's nice is R (well, S) was designed to be a statistical computing tool, and it does anything related to that quite well. Especially for data analysis, which is where I spend a lot of my time (perhaps most) in the whole model building process, R is amazing. Also, R is very usable out of the box for math and data visualization, whereas Python requires many libraries. It's good to know R.

I think R and Python can be used in conjunction very effectively. Analyze your data in R, prototype your algorithms in either, build products in Python.

dbecker3• 14 years ago

With rpy2 you can even embed R code in Python.

pfanner2• 14 years ago

>If you are a real data scientist or expert, skip this

I am a "real data scientist" and need some advanced data analysis technics and machine learning. Can someone recommend an introduction for me?

ihodes• 14 years ago

What's your background? (i.e. what do you already know?)

pfanner2• 14 years ago

Probably not much. I started working with bigger data some months ago and now I notice that I need some real techniques and not just my "ok try this and this". I'm coding in C (where I "create" the data (numerical integration of stochastic differential equations)) and Python (plotting). I need methods/algorithms/techniques to analyse the data "on the fly" because I can't save it all (it's too much data).

EvanKelly• 14 years ago

Does anyone have any experience with Octave and how it compares to the Python setup that OP suggests?

What are the benefits/pitfalls?

nwenzel• 14 years ago

I have used Octave but only for the Coursera ML Class, not actual production use. My understanding is that it is the open source version of Matlab capable of running many Matlab programs.

I learned Octave first, then R, then Python.

Octave to me feels like Python + NumPy. I'd say Octave has more in common with R than with Python.

Given the choice between Octave and R, I'd choose R for the more robust user community and incredibly diverse and thorough selections of libraries.

apl• 14 years ago

Octave sits at an awkward half-way point between MATLAB and Python/R/Julia/etc. You get the shitty syntax of MATLAB at not quite MATLAB's speed and miss out on support as well as various incompatible toolboxes. So unless you have hard dependencies or lots of MATLAB code sitting around, Octave isn't that attractive an option.

It's great for matrix/vector maths, though. If that's all you do -- go for it. Everything over and above that is a royal pain in MATLAB and, conversely, in Octave.

gallamine• 14 years ago

If you're using a Mac, please don't install all of this individually. Instead, install the Scipy Superpack: http://fonnesbeck.github.com/ScipySuperpack/

gammarator• 14 years ago

The superpack is a great one-click option. For staying up to date, I find a package manager (like macports or homebrew) more effective for managing my python packages.

dbecker3• 14 years ago

>I found the syntax baffling, the documentation copious, but written for mathematicians instead of hackers

I'm always surprised how much people hate the syntax of R. I primarily work in Python, but I use R once or twice a week... and the syntax seems very clean to me. Can someone give mee an example of what you dislike with R's syntax?

I'm even more surprised to hear complaints about the documentation in R. The help files in R are much more complete and well organized than docstrings in the python libraries we use. Even the web usually lacks anything as useful as what I get from the vignettes function in my R interpreter.

rrosen3262• 14 years ago

(Author here) - I don't have any specific examples, but I was learning R and Python at the same time. I found Python to be very practical and easy to learn. When learning R, I kept getting tripped up. Maybe it isn't that R was harder, but that I had a head start on Python. And as for documentation, R was certainly very complete, but once again, I found it harder. I think because R is written by, and probably for, professional statisticians and mathematicians, it needs to have a different level of rigor than the Python documentation. Anyway, sorry for the lack of specificity.

yarrel• 14 years ago

R does have some weirdness (it took me ages to understand the index and slice notations), but it is very expressive.

I'm not a mathematician, and I've been programming Python for 15 years, but I'd always pick R for its stated problem domain given a choice.

I highly recommend "The Art Of R Programming" for learning R as a programming language. The statistical side of things are then easier to layer on top of that.

zmmmmm• 14 years ago

Personally it's not so much syntax as the confusing data model that gets me in R. So many different but very similar data types - lists, data frames, matrices, tables, vectors - all very similar but slightly different syntax, very frequently converted silently from one to the other when you call functions but resulting in strange quirks that are extremely hard to debug at the other end. The combination of loose data typing and this plethora of similar data types makes it a nightmare to work with at times. On the other hand when you grok it and it works for you ... it's amazing.

dbecker3• 14 years ago

The complaints about R still seem counterintuitive to me. Python has the same data types listed above, and many more.

A list in R is a list in Python. A data frame in R is a date frame in the pandas library. A matrix in R is a matrix in numpy. A vector in R is a 1-dimensional ndarray in numpy.

But Python adds dictionaries, tuples, iterators, sets, and a bunch of other data types that aren't used in R.

R's lists and vectors are relatively similar... but you could say the same thing about numpy's matrix and ndarray. You could probably say the same thing about python's sets, tuples and lists.

To be honest, I'd have said the strength of python is that it has many more data types than R... rather than fewer data types.

peteTorrione• 14 years ago

A nice list of tools. We tried to use Python 3.0, but had the same problems as the author...

And if you are lucky enough to have free-ish access to MATLAB, here's a free, BSD, open-source, github repo'd machine learning toolbox to help you get started:

http://www.newfolderconsulting.com/prt/

Full disclosure: I'm involved.

Radim• 14 years ago

Don't forget gensim (and its tutorials): http://radimrehurek.com/gensim

Plus it's the only one on that list that will scale beyond the "My Data Set" size.

Tools to get started in machine learning

Related Stories