Numenta open-sourced their Cortical Learning Algorithm

http://numenta.org/

by p1esk • 13 years ago

52 20 13 years ago

numenta.org

eli_gottlieb• 13 years ago

Ok, so what does this mean to those of us who don't do deep neuroscience-based machine learning that often?

Basically, what problem (problems?) does this algorithm actually solve?

gdahl3• 13 years ago

Basically none. Numenta has yet to do anything that has impressed any researchers I know. Perhaps someday they will, but I am not counting on it.

oxtopus• 13 years ago

NuPIC was released as free software, in part, because there has been a strong interest in the academic research community. I encourage you to check out the mailing list (http://lists.numenta.org/mailman/listinfo/nupic_lists.nument...) for some recent examples.

chubot• 13 years ago

Have those researchers released any source code?

rdtsc• 13 years ago

That was my main question. How does this compare (either experimentally or theoretically) to already well established approaches.

No references in their paper.

The smells like Wolfram's "New Kind of Science" type work. And they are welcome to prove me wrong, I like AI topics. But it is isolated from other peer work, when AI field is not new. So kind of suspicious.

cr4zy• 13 years ago

I've studied Numenta's CLA's, previously HTM's, as well as established ML approaches fairly in-depth over the past couple of years. Hawkins definitely takes a new approach to neural networks and focuses more on modeling the brain's biology rather than building on known artificial neural network concepts. HTM's did incorporate bayesian nets, but have been abandoned, probably with the departure of Dileep George, who now runs http://vicarious.com/.

However, Hawkins work is similar to Deep Neural Nets in that both are unsupervised neural nets (unlike perceptrons and back-prop) and both aim to represent data as sparse representations. Also, here's a similar question I answered on Quora: http://www.quora.com/Machine-Learning/Does-Numenta-build-on-...

In addition, Numenta does use CLA's to solve real problems for customers: https://www.groksolutions.com/solutions.html

Finally, Hawkins' book, On Intelligence, has at least inspired researchers like Stanford's Andrew Ng at a philosophical level which he talks about IIRC in his online ML course. https://class.coursera.org/ml/class

p1esk• 13 years ago

Here's an independent review of the older version of the algorithm: http://bias.csr.unibo.it/maltoni/HTM_TR_v1.0.pdf

"Systematic experiments on three line-drawing datasets have been carried out to better understand HTM peculiarities and to extensively compare it against other well-know pattern recognition approaches. Our results prove the effectiveness of the new algorithms introduced and that HTM, even if still in its infancy, compares favorably with other existing technologies."

etrautmann• 13 years ago

I'm a neuroscientist and I'm also not clear on exactly what the anticipated use case for this is. The connection to principles in neuroscience is somewhat loose, it appears to be more grounded in machine learning.

volokoumphetico3• 13 years ago

If I use this, could I make something that is better than machine learning using neural networks?

derefr2• 13 years ago

Not better in the general case; most machine-learning algorithms do worse than the naive solution ("throw a Bayes net at it") in most applications, but then have a niche where they excel.

If I recall, Numenta's model is said to excel in the case where the data it's being trained on is similar to the kind the human brain tends to process: sensory data with temporally-local "features" (letters on a page, instruments in a song), where the features form a hierarchy (shapes->letters->words->sentences, frequencies->notes->chords->riffs->melodies, etc.)

tensor• 13 years ago

most machine-learning algorithms do worse than the naive solution ("throw a Bayes net at it") in most applications

This is an extremely misleading statement. There is nothing naive about probabilistic graphical models. It's a massive topic with a huge number of different model variations, training methods, and techniques to avoid overfitting or do feature selection.

More many probably you probably only want a linear classifier, and a discriminative one at that. But if your data is not linearly separable, or has structure, then you'll want something else again.

volokoumphetico3• 13 years ago

Your description of Numenta is very exciting for me as it appears to fit my niche.

For example, can Numenta look at two similar pages from pdf, and be able to deduce and discover the location of title, paragraph, heading, subheading, etc? Those pdf files are different in layout but a human can immediately determine the hierarchy by categorizing entities on that temporally local feature.

tlarkworthy2• 13 years ago

Hmmm, the white paper is not very encouraging:-

What is NOT in this document:

- Although most aspects of the algorithms have been implemented and tested in software, none of the test results are currently included.

-There is no description of how the algorithms can be applied to practical problems. Missing is a description of how you would convert data from a sensor or database into a distributed representation suitable for the algorithms.

So until someone has tried it on something, we don't know how it will perform :s

tlarkworthy2• 13 years ago

BTW I was trying to install it on ubuntu 12.04 (an LTS) and could not get it working. It wanted to change my numpy dist

Shooti• 13 years ago

This Google tech talk provides the missing context: http://www.youtube.com/watch?v=4y43qwS8fl4

Numenta open-sourced their Cortical Learning Algorithm

Related Stories