Show HN: Incremental learning of polynomials (and other LIP models) with IRMA

http://buschermoehle.org/andreas/irma.htm

by MrBusch • 6 years ago

24 9 6 years ago

MrBusch4• 6 years ago

I developed a new incremental learning approach (called IRMA) during my PhD in 2014 and haven't touched that research for a few years. But it has always been on the back of my mind as an approach worth following up on.

Now I decided to make it a bit more approachable through an interactive tool that lets you play with a polynomial that learns from incremental examples you provide. I also included some background on how the method works.

Incremental learning (in contrast to batch learning) poses a unique set of problems as the learning algorithm needs to adapt with just a single new example. Compared to the state of the art, IRMA does this through minimizing what it "forgets" about past learned data while adapting to the new example. I chose polynomials as an example as it doesn't work well with the typically used gradient descent but can be learned with IRMA in a much more stable manner.

The same approach has a closed form solution for a variety of other models (that are linear in the parameters, i.e. LIP) and I'd be interested to try and apply it to more models (like neural networks) or other tasks (like classification) as well.

I'm excited about any questions or feedback!

verdverm2• 6 years ago

Interesting, I'll have to dig in some more. I have a similar story with Prioritized Grammar Enumeration (PGE) for Symbolic Regression. It's my PhD work that has been sidelined since 2015 and I've been thinking of resurrecting it.

Nice work!

MrBusch4• 6 years ago

Nice, I have only limited experience with symbolic regression. But, from what I gathered from the abstract of an ACM paper I found, I like the detour from the usual stochastic approach toward a deterministic directed search. Does that imply it could have problems with local optima, though?

saiojd• 6 years ago

Nice work and great presentation.

yorwba• 6 years ago

I'm sure it's a nice visualization, but unfortunately I'm only seeing a white gap between the two sliders and tapping randomly doesn't seem to have any effect. I assume I should be seeing a polynomial graphed even before doing anything.

Tested on Android using both Firefox and Chrome.

MrBusch4• 6 years ago

Sorry, just tested it myself and it seems like the link does not go to the correct https site. The full URL would be https://www.buschermoehle.org/andreas/irma.htm

Might also just be that the web-server is not handling the traffic well. Some refresh's work, others fail to load the javascript.

MrBusch4• 6 years ago

The problem should be fixed now - there was a race condition in the loading order of a javascript library.

jjgreen• 6 years ago

Reminiscent (in spirit) to [1], but there to the specific case of a histogram.

[1] https://ieeexplore.ieee.org/document/6971097

Show HN: Incremental learning of polynomials (and other LIP models) with IRMA

Related Stories