Maryland research could improve the AI task of sensorimotor representation

https://eng.umd.edu/release/helping-robots-remember-hyperdimensional-computing-theory-could-change-the-way-ai-works

by headalgorithm • 7 years ago

84 29 7 years ago

eng.umd.edu

taneq2• 7 years ago

These HDVs (hyperdimensional binary vectors) sound a lot like the SDRs (sparse distributed representations) that Numenta talks about with their HTMs (hierarchical temporal memory).

Wow, what a lot of TLAs.

andbberger2• 7 years ago

That is probably not a coincidence. Pentti Kanerva, who does a lot of work on hyperdimensional computing, is at the redwood center for theory neuro which Hawkins (numenta) helped found.

evgen• 7 years ago

Kanerva is also the author of the (vastly underrated IMHO) book Sparse Distributed Memory from the late 80s that set the stage for a lot of this and predates Numenta by a couple of decades...

plutonorm3• 7 years ago

Hyperdimensional binary vectors??

What like:

010101010111010101010111011001

So they are encoding perceptions and actions into the same vector space. Cool. But the marketing is just gibberish.

jbotz• 7 years ago

Not really. "hyperdimensional binary vector" may sound rather buzzwordish, but it has a fairly precise definition in this context, and it may indeed be highly significant for the future of neural computing. You can learn more about it with the code (and paper) here: https://github.com/alexanderganderson/bnn_vis

plutonorm3• 7 years ago

Err. Just read the abstract. It's a binary vector. Nothing more. That these high dimensional binary vectors behave similarly enough to real vectors for the purposes of training binary neural networks in no way justifies the name "hyperdenional binary vector". It's a vector of Booleans.

cs7023• 7 years ago

This is a very click-baity headline. I'm tempted to flag it!

Has anyone here seen the actual paper?

vcavallo2• 7 years ago

have you read the content of the link? i wouldn’t flag it

cs7023• 7 years ago

FYI, someone (probably a moderator) changed the headline after I wrote my comment. I'm OK with the new headline.

nabla9• 7 years ago

https://robotics.sciencemag.org/content/4/30/eaaw6736

http://sci-hub.tw/10.1126/scirobotics.aaw6736

p1esk2• 7 years ago

Closed access journal for machine learning in 2019? No thanks

valgor• 7 years ago

The paper is linked in the article if can be bothered to read what you are threatening to flag...

cs7023• 7 years ago

FYI, someone (probably a moderator) changed the headline after I wrote my comment. I'm OK with the new headline.

vcavallo2• 7 years ago

this seems like a pretty significant step towards a sort of “holistic” agent whose past informs its present and future in a more human-relatable way.

gre• 7 years ago

Not posting your machine learning paper on arxiv.org is a red flag.

andbberger2• 7 years ago

What?? arxiv is a _preprint_ server

hsdfhsdal• 7 years ago

What about vixra.org?

p1esk2• 7 years ago

For papers so bad that even arxiv refuses to publish them? Sure, why not...

billconan• 7 years ago

I do not understand the point of this paper. Say, deep reinforcement learning also encodes perception/state into a large vector. Combining perceptions into a large vector doesn’t seem to contribute much?

gnode2• 7 years ago

The article explains so little, so it's hard to know.

Neural networks generally don't learn by themselves, they require some kind of optimisation outside of their operation. It seems to me that they're saying this is more of a cognitive architecture which learns continuously by memorisation.

evrydayhustling• 7 years ago

This is probably a "publishing in the press" translation issue, but the notion that neural networks have no memory is dead wrong. DNNs encode incredibly specific histories of training data in their weights, which is both a privacy/security risk [1] and a phenomena that is actively being utilized in many dnn architectures. Overall, this has a "brand the framework before proving exceptional capabilities" feel.

[1] https://arxiv.org/abs/1802.08232

gnode2• 7 years ago

I don't think that's what they meant. I interpreted it to mean: neural networks are not trained in a continuous online manner.

That said, there are neural network architectures which have state, such as recurrent neural networks, time-delay neural networks, and long short-term memory. However, the state is used for covering problem domains with a temporal nature, rather than for reflective learning. It's typically reset between different inputs.

sgillen2• 7 years ago

There are lots of examples of networks being trained online, I see it all the time in robotic control.

giardini• 7 years ago

Related older easy-to-read article, paper and seminar lecture:

1. "Binary holographic reduced representations for SWI-Prolog" by Jocelyn Ireson-Paine:

http://www.j-paine.org/hrr.html

2. "Dual Role of Analogy in the Design of a Cognitive Computer" by Pentti Kanerva, in Advances in Analogy Research: Integration of Theory and Data from the Cognitive, Computational, and Neural Sciences. Workshop. Sofia, Bulgaria, July 17-20, 1998:

http://faculty.cs.tamu.edu/choe/mirror/kanerva.ANALOGY98-kan...

3. Pentti Kanerva lectures at Stanford on "Computing with High-Dimensional Vectors":

https://www.youtube.com/watch?v=zUCoxhExe0o

4. Additional information, including the slides, for Kanerva's lecture are available at:

http://web.stanford.edu/class/ee380/Abstracts/171025.html

tentboy• 7 years ago

Unfortunately I have very little experience with AI or ML. I did however have the lead contact on this paper (Yiannis Aloimonos) as a professor for computer vision my senior year at Maryland.

He spent most of the lectures rambling about his research, including this, rather than the course topic. I learned the course material on my own but still enjoyed listening to him.

mklyons• 7 years ago

Yiannis is a fantastic professor, probably my favorite during undergrad at Maryland.

Maryland research could improve the AI task of sensorimotor representation

Related Stories