obstbraende

here's a link with reviewer comments https://openreview.net/forum?id=PdauS7wZBfC (praise to openreview!)

You probably need to factor healthcare, rent etc for a fair comparison to, say, the Bay area -- 85k affords a nice upper-upper middle class lifestyle (it scratches the 99th gross income percentile in germany). So 10x sounds like a stretch. What profile would you suggest someone who takes 200-800k € might have? (Tangentially, out of curiosity: Are you in a position to share what the ex-googlers worked on?)

That sounds like a degenerate organisation. But if we're sharing anecdata: My pay in this Berlin startup is very decent, I have respectable equity and it's a friendly & rational environment. I'm scared of US startup work culture, rather.

That's too naive. There's next to nothing known about how activity in individual neurons and their synapses relates to mental contents, in particular "higher level" concepts and thought patterns that you're concerned with in adult learning. In other words, the network dynamics are complex and unknown, and so the suppression / deactivation of certain synapses may just as well be a normal and necessary part of learning as it may be a part of forgetting, or neither. There is currently no contender for a "neuroscience standard model" that would bridge between this kind of neural dynamics and cognitive functions. I hope to live to see one.

do you have more info on that quote? I'd like to read more about that time, but google failed me

it doesn't seem to register key presses on chrome 56 (mac os)

If we want to assign credit for the LSTM to one person, his student Hochreiter is perhaps the better pick

They use evolutionary search to discover spiking neural networks whose response dynamics can solve a control task. This is a fascinating approach, but one that I've only ever seen as a means to do theoretical neuroscience: A way to obtain interesting spiking networks whose dynamics we can study in the hope of developing mathematical tools that will help understand biological networks.

But here, from the claims in the post and the lab website, it sounds as if the goal is in application: Creating better, more efficient controllers. This comes across as a little detached from the applied machine learning literature. At the least, I missed a comparison to reinforcement learning (which has a history of learning to solve this exact task with simpler controller designs and most likely shorter search times) and also to non-bio-inspired recurrent networks.

One more point: Even if I follow along with the claim that 'deep learning' approaches don't have memory (implying recurrent networks aren't included in that label), I want to point out that this particular task setup, with positions/angles as well as their rates of change provided, can be solved by a memoryless controller. It would have done more to highlight the strengths of the recurrent network approach if a partially observable benchmark task had been used, e.g. feeding positions and angles only. Much more difficult high-dimensional tasks e.g. in robotic control are tackled in the (deep) reinforcement learning literature among others.

Can you explain that a bit more? I'm fascinated but don't quite get the rationale behind having not even the authors of the paper know the results. What is their process?

Thanks! When I tried this before, I thought compilation was stuck in an infinite loop and gave up after about a minute. But you're right, it works. Though on my machine, this took two and a half minutes to compile (ten times as long as compiling a small convnet). For 10 recurrence steps, that's weird, right? And the TensorFlow thing above runs instantly.

I think this loop actually still only builds the graph -- what `scan` would do. The computation still happens outside of python. That is, in tensorflow they perhaps don't need `scan` because a loop with repeated assignments "just works"... Let's try this:

It seems like in TensorFlow you can say:

    import tensorflow as tf 
    sess = tf.InteractiveSession() # magic incantation

    state = init_state = tf.Variable(1) # initialise a scalar variable

    states = []
    for step in range(10):
         # this seems to define a graph that updates `state`:
         state = tf.add(state,state)
         states.append(state)

    sess.run(tf.initialize_all_variables())

    print sess.run(states)
    >>> [2, 4, 8, 16, 32, 64, 128, 256, 512, 1024]

Are there any major conceptual differences to Theano? Not that I wouldn't appreciate a more polished, well funded competitor in the same space.

It looks like using TensorFlow from Python will feel quite familiar to a Theano user, starting with the separation of graph building and graph running, but also down into the details of how variables, inputs and 'givens' (called feed dicts in tensorflow) are handled.

one more helpful criticism: the 'epic' music in your video, for me at least, evokes the opposite of a zen-like, concentrated state. it brings up images of action movies, war scenes or documentaries about bridges.

Put more favourably, the purpose of an open access journal is to arrange a critical evaluation of my stuff and to point other researchers to it (curation). To justify paying for this, I think of it like hiring a consultancy and, later, a marketing firm for my research.

Perhaps in a better world, we would disentangle the review and the curation aspects -- I'd pay for the arrangement of peer review of my article, upload it freely to the arxive, and pay separately for unbiased, curated lists of noteworthy articles in my field.

Quantum Bayesianism [0] treats such effects explicitely as Bayesian belief updates of the experimenter. This leads to an interpretation of quantum mechanics free of such paradoxical/counterintuitive statements about reality. I'd like to hear other physicists' opinions on minority interpretations like this.

[0] layman-friendly article / interview: https://www.quantamagazine.org/20150604-quantum-bayesianism-...