Machine Learning Library for C++

http://image.diku.dk/shark/sphinx_pages/build/html/index.html

by optiminimalist • 13 years ago

76 49 13 years ago

image.diku.dk

JacobiX3• 13 years ago

The library supports some very useful algorithms. Both supervised and unsupervised ones. But I can't use it for our commercial non GPL projects.

aeroevan• 13 years ago

If LGPL works for you, you may want to look into http://www.mlpack.org/

I'm not really sure how they compare since I haven't really used either library, but there does seem to be some overlap.

JacobiX3• 13 years ago

Thanks, it's not as complete as Shark. But it is nonetheless very interesting.

mindcrime• 13 years ago

Could you have the ML part done as a standalone process and not actually linked into your other project code? That is, process some data "out of band" so to speak, and write it out to a file, which is then used by "your stuff"? Or communicate with the ML part over a socket? Either approach would let you use this code without requiring the other parts of your project to be licensed under the GPL.

Just a thought...

JacobiX3• 13 years ago

Yes I think It is possible to use the library as a standalone process. It will introduce some little overhead. More importantly the code will be harder to debug and maintain.

jhartmann6• 13 years ago

Flame Suit On / Rant Mode On

I actually really don't understand why anyone uses GPL for a library. I've been doing open source for a long long time, and love the GPL. I have code in the Linux kernel, and believe free software AND open source software are great solutions to very real problems in software engineering. Having open code just gives people more options, and I firmly believe it will win over time as far as quality is concerned.

I just think only providing libraries to other GPL code is stupid. It just limits the usefulness of the software. LGPL is great here, you get the core changes contributed back to your library from a greater group of people and everyone wins. Limiting a library to GPL means a large population can not use your code, those writing applications that can't be licensed under the GPL. Limiting choice is BAD. The whole reason you should be creating and using free software and OSS is to not weld the hood shut. GPL should be for applications, LGPL just limits choices for libraries. Down with the GPL for libraries!!!

Flame Suit Off / Rant Mode Off

kruhft• 13 years ago

Some people believe in software freedom more than you do.

to3m3• 13 years ago

LPGL isn't vastly better than the GPL as it makes it difficult to link statically and/or release for closed platforms. Its advocates would probably see these as plus points, but I'm not sure that they're going to increase uptake.

mjn• 13 years ago

Limiting choice is BAD. The whole reason you should be creating and using free software and OSS is to not weld the hood shut.

But you sound like you want to limit choice for users. The whole point of free software from the GNU perspective is to keep options open for users, and not allow downstream devs to "weld the hood shut" on derivatives by adding more restrictions on what users can do with those derivatives.

However, there are other reasons people choose it as well. One motivation you sometimes encounter is a view that, if someone's code is used in proprietary, commercial software, they'd like to be paid for it. Hence the dual-licensed model used by libraries like Qt and the Stanford Parser: you can use the GPL version if you're willing to GPL your own app, or you can buy a proprietary license if you aren't. Seems reasonably fair: I give you my code free if you reciprocate and do likewise with your own code, or I sell you a license for cash otherwise.

hackinthebochs• 13 years ago

This looks awesome. I've been itching to try out some ideas I have after having gone through Bishop's book, but I've been hesitant to write the algorithms from scratch. Now I'll have to decide between learning matlab or a library such as this.

c0g2• 13 years ago

Speaking as a PhD student in machine learning-

Implement the algorithm yourself, first, in Python+Numpy. The only reason I feel comfortable with Gaussian Processes and SVMs is due to writing code to solve them manually.

Once you're happy with the basics, and can test your ideas with code you intimately understand, optimise for speed by using a library like this.

misiti37804• 13 years ago

Implementing the SVM from scratch was time consuming - no?

misiti37804• 13 years ago

I do agree with you regardless, just curious

tlarkworthy3• 13 years ago

You should definitely use a scriptable ML library. The process is very iterative and not suited to a compiled language like C++. I use skilearn alot, but also the matlab toolboxes or R are great. At its heart ML is alot of stats, so use something built for maths, not C++. It doesn't really make sense to break out C++ until you know exactly what algorithm and settings you need and your application is real time.

bch2• 13 years ago

If the OP was thinking of writing his own algorithms, and this is a linkable library with that heavy math already implemented, couldn't he write bindings for Python/Lua/Tcl/Ruby and have everything he needs for script-ability, or am I missing something?

pmelendez3• 13 years ago

I am very agreed with this. I had even been thinking in making a DSL (Scheme based probably) oriented to ML instead of a library. I would found that more useful in the exploring phase.

swalsh• 13 years ago

Does anyone know if there's a good ML library written in C# or F#

microtonal• 13 years ago

libsvm and liblinear are nice libraries for training SVM and linear classifiers. There seem to be two ports of liblinear for C# (I haven't tried them):

http://www.csie.ntu.edu.tw/~cjlin/libsvm/

profquail• 13 years ago

I don't know of any written in F#...maybe I'll have to write one :)

You might also ask on the fsharp-opensource mailing list, maybe someone has an F# ML library I don't know about:

https://groups.google.com/forum/?fromgroups#!forum/fsharp-op...

jnazario• 13 years ago

how about numl? http://numl.net/

folks on SO also like WEKA run through IKVM (a Java to .NET converter): http://stackoverflow.com/questions/1624060/machine-learning-...

pilooch3• 13 years ago

Very interesting, but as a daily practitioner I am skeptical.

First, this is a lot of code! As a C++ machine learning programmer, I am impressed as I know the pain (someone explains why, see comment https://news.ycombinator.com/item?id=5613797 ).

Second, it contains a version of Blas and ublas as well as LBFGS and more, much more, coded from scratch as it seems. This seems too much for an ML library, and a lot to maintain.

This makes me skeptical of performances and maintenance of the code, but it would be fairer to try it first.

Still very impressed.

pmelendez3• 13 years ago

May I ask how do you become a C++ ML programmer? It is not an usual position but one I would definitely be interested in pursuing

pilooch3• 13 years ago

Sure, I started as a researcher 14 years ago, then drifted to what I thought was a sweet spot then, half-research / half-programmer. I say 'sweet spot' because many applications did require both the academic and the applied background at the time, so for the sake of thrilling applications, it was worth 'downgrading' to pure engineering work when needed. Now I believe the game has changed a bit, coursera and others are infusing the minds of engineers with highly technical knowledge far more rapidly than before.

Typically I am astonished at the number of implementations of deep learning techniques (Shark does include some, AFAIK).

My past experience is that I had write many AI algorithms myself because I could not find any suitable, free and/or open implementations (or other researchers would not share theirs ;) ).

Schneekatze3• 13 years ago

Hi, shark developer here.

First of all, we are glad that our library is discussed on this board! We are happy for every feedback we can get!

Regarding Performance: we try to get the key algorithms as fast as possible. And for the hardest parts we rely not on ublas, but use optional bindings to ATLAS. Speed was one of the key design criteria. We hope that we achieved that. Clearly this is no guarantee that every algorithm is fast, but in this case: just add a ticket!

Please bear in mind, that Shark is still in beta stage, and we are heavily developing it (I am right now working on the family of multi class SVMs). So for example parallelism using OpenMP is not fully integrated.

jhartmann6• 13 years ago

You should consider using Eigen for linear algebra, I have personally found it much better performance wise than using bindings to ATLAS or other more standard linear algebra solutions. ML algorithms tend to be multistage (think about the update of weights with momentum in a Neural network for example), and the primitives available in ATLAS or a blas library are really too low level. Eigen since it generates code for a whole complicated expression can blow a standard linear algebra library out of the water for a certain class of problems. For others obviously the highly tuned vendor BLAS code would win, but I've seen huge speedups by using Eigen it fits well for complex ML operations.

jhartmann6• 13 years ago

FYI, another bonus on Eigen is if you are using a Matrix-Matrix operation you will get multithreading with OpenMP for free. It isn't the most efficient way (you can do better threading yourself probably), but it gets most of the way there for stuff that will use GEMM operations.

pilooch3• 13 years ago

Amazing work, thanks for sharing. Any hint on how it behaves on big datasets ? Over time I've found that scaling up to real world (and industry) datasets requires going custom (or distributed, a-la-mr / hadoop).

Machine Learning Library for C++

Related Stories