siekmanj

"RL: A Deep Reinforcement Learning Framework" seems to have been hallucinated, does not exist.

Video here: https://twitter.com/oregonstatenews/status/15747969243260108...

https://archive.ph/8X76D

True, but it's not trying to be multi-threaded, just concurrent.

Python has actually had concurrency since about 2019: https://docs.python.org/3/library/asyncio.html. Having used it a few times, it seems fairly sane, but tbf my experience with concurrency in other languages is fairly limited.

edit: ray https://github.com/ray-project/ray is also pretty easy to use and powerful for actual parallelism

From some cursory googling:

Afghanistan peak US troops: 102,000 (2011)

Gulf war peak US troops: 700,000 (1990-1991)

Iraq peak US troops: range from 168,000 - 192,000 (2007)

Vietnam peak US troops: 543,000 (1969)

Korea peak US troops: 320,000, unclear what year

edit: obviously, Russia's involvement in Ukraine over the last few days would be by far the biggest operation in Europe since WW2. Just not globally (by a long shot).

This is an amazing outcome. The license cost was a huge barrier to entry and the progress of deep RL generally.

The sim2real approach can work pretty well as long as you are very in tune with where your simulator falls short relative to the real world and take steps to circumvent those shortcomings.

Here's some work my lab did on sim2real for a roughly human-scale bipedal robot (Cassie): https://www.youtube.com/watch?v=MPhEmC6b6XU

We were able to train the robot to climb stairs completely by feel/proprioception without any sort of vision. We trained it in simulation, and then transferred it to the real world without issue.

Since you mentioned it--here's what Oregon State University and the Cassie group is up to these days: https://youtu.be/MPhEmC6b6XU

It's a German phrase which colloquially translated means 'Gotta go bro'

Actually, I think it is conventional neural networks which can only approximate finite state machines. RNNs are (in theory, not so much in practice) Turing complete.

Hey everyone, I am one of the authors of the paper described in this article. We used reinforcement learning and recurrent neural networks to learn a controller which can ascend and descend stairs without any vision-based perception, meaning that it must rely entirely on proprioception to walk. This is the first time (to our knowledge) that a human-sized bipedal robot has been able to climb real-world stairs blind to the world, and we're pretty excited about the results of this research.

Here's a link to the Arxiv submission: https://arxiv.org/abs/2105.08328

And here's the accompanying video: https://youtu.be/MPhEmC6b6XU

And an uninterrupted five-minute video of a test on an outdoor staircase: https://youtu.be/nuhHiKEtaZQ

Happy to answer any questions the HN crowd may have!

I was always very unimpressed with VR until I borrowed a headset to play Half Life Alyx. That game convinced me VR is the inevitable future of gaming. The rest of the industry may not be there yet, but it is an obviously superior experience.

This is mostly true for supervised and unsupervised learning models, but for reinforcement learning the LSTM is king because of the convenient fact that it can be evaluated one time step at a time, instead of just outputting a sequence like a transformer. For things like robotic control, etc, attention-based models are pretty nonsensical.

Python doesn’t really have scope in that way. Variables declared inside an if statement can be accessed outside of the if statement.

Wow. I have been looking for a good resource on implementing self-attention/transformers on my own for the last week - can't wait to read this through.

Unfortunately, looks like KSP 2 will also be using Unity: https://forum.kerbalspaceprogram.com/index.php?/topic/187315...

Interesting, lots of people say the exact same things about Portland. I would guess every booming city is facing this dilemma.

theoldnet.com is currently unable to handle this request.

Not sure if intentional or hilarious irony.

I've been writing a C-based deep learning library for the past year or so, which I've recently been trying to clean up to be a bit more presentable. It was pretty heavily inspired by Darknet, although it admittedly has far fewer features: https://github.com/siekmanj/sieknet

Andrej Karpathy has a pretty good introductory article to RNNs here: http://karpathy.github.io/2015/05/21/rnn-effectiveness/

He has some code which is pretty easy to follow to go along with the article: https://gist.github.com/karpathy/d4dee566867f8291f086

I mean, don’t most researchers use Adam? As far as I know that is a second-order method.

The genetic algorithm currently isn't in a working state unfortunately. I did a big architectural rewrite recently and haven't gotten around to GA stuff - but I'm looking forward to applying some more modern results in neuroevolution from the last two or three years.

shameless self-promotion: https://github.com/siekmanj/sieknet

I don’t think that this comment is relevant to the article.

It's definitely finding a niche in robotic control. My lab just released a paper about running a policy trained in simulation in the real world on a bipedal robot.

Sooo ... the president is legally allowed to turn off my light switches or commandeer my phone, but does the infrastructure for him to do that exist?

That may be true, but that is a far cry from asymptotic

someday we'll achieve a utopia wherein we'll be able to 3d print anything we want, and it won't be possible for governments to exist and we'll all live in libertarian paradise.

At least, that's what I got from him.

We know the brain is doing something - if you don't want to call it computation, then you might as well call it magic.