OpenAI Gym: Toolkit for developing, comparing reinforcement learning algorithms

https://gym.openai.com/

337 18 10 years ago

gym.openai.com

joshmarlow• 10 years ago

Awesome. I can't wait to play with this. I had actually been doing a side project with the same idea (though of course much simpler!). I just got a lot more free time :)

ericjang• 10 years ago

This is really cool - kind of like Kaggle for RL algorithms!

benhamner• 10 years ago

Agree! Down the road, Kaggle will be the Kaggle for RL algorithms ;) This provides a really cool open source environment to build on

jlas• 10 years ago

Other tools for developing / comparing RL algorithms:

* Burlap (from Brown-UMBC) https://github.com/jmacglashan/burlap

* RL Glue http://glue.rl-community.org/wiki/Main_Page

Also looks like some of the challenges come from ALE: https://github.com/mgbellemare/Arcade-Learning-Environment

nrmn• 10 years ago

PLE as well! https://github.com/ntasfi/PyGame-Learning-Environment

(disclaimer I'm the author)

gwern• 10 years ago

Unfortunately RL Glue is not nearly as good as it looks or sounds. I was thinking about using it to solve some problems I have (for example, I was interested in modeling embryo selection for complex traits as a POMPD to see what the optimal approaches might be rather than settling for a simple calculation as I did in http://www.gwern.net/Embryo%20selection ), but the project is abandoned, and while it implements an interface/IPC glue, there are few implementations of any interesting environments or useful agents, so you would wind up writing everything yourself anyway. Once I got the simple tabular agent example working, I went looking at the more complex ones and was quite disappointed.

From the description in OP, it already far exceeds RL Glue's usefulness.

phodo• 10 years ago

Excellent - thank you.

nxzero• 10 years ago

Leaderboards might help grow the community.

MetaMetaApplyHN• 10 years ago

> Evaluations: We've made it easy to upload results to OpenAI Gym. However, we've opted not to create traditional leaderboards. What matters for research isn't your score (it's possible to overfit or hand-craft solutions to particular tasks), but instead the generality of your technique.

https://openai.com/blog/openai-gym-beta/

paulsutter2• 10 years ago

From the link:

we've opted not to create traditional leaderboards. What matters for research isn't your score (it's possible to overfit or hand-craft solutions to particular tasks), but instead the generality of your technique.

(comment copied from MetaMetaApplyHN, seen with showdead)

detaro• 10 years ago

If you go to the permalink of a dead comment, you should see a "vouch" link that should make the comment visible. (I did it now for MetaMetaApplyHNs comment)

wrsh073• 10 years ago

How often will environment versions change? Does a more sophisticated versioning scheme make sense? [like Semantic Versioning?]

I don't really know what it means to have a backward compatible change in an environment, but you know.

Matt3o12_• 10 years ago

Can someone with knowledge in AI explain to me what this framework does compared to others (mainly OS but also proprietary) and if it provides any advances in the field?

wrsh073• 10 years ago

It looks like this system allows easy comparison of reinforcement learning techniques [those techniques can be implemented using any major ML framework].

So it isn't a competitor to Tensorflow, it sort of allows you to measure techniques built using Tensorflow [or Theano etc]. And the metrics are "how general is this technique?" "how effective is this technique?"

Essentially, lots of research says "this technique is effective at this problem," but there is often no way to compare it to other leading techniques in the field. This playground sort of puts everyone on the same playing field.

[and since it's open, it seems you can add your own environments. Like if you want to build a Chess AI using reinforcement learning, you can add a chess environment to this playground]

paulsutter2• 10 years ago

Gym's potential is to be a starting point for a community around OpenAI. There are far more smart people in the world eager to demonstrate their ability, than there are experts directly employed in AI today.

Over 5000 teams around the world submitted solutions to the Netflix prize[1]. OpenAI is a far more important project. They should be able to drive even greater enthusiasm and participation.

[1] http://www.netflixprize.com/leaderboard

wrsh073• 10 years ago

I don't know why it took me this long to realize, but this could be a sort of new-age journal. Published research [on github], reviewed by peers, and reproduced by others. -- gdb@ was that in your mind as you built this?

I really hope they gain traction.

kordless• 10 years ago

And then hook up cryptocurrency payments so the AI can pay for upgrades.

patapizza• 10 years ago

Awesome. I love these open initiatives to tackle big problems all together (like the bAbI tasks for NLU).

OpenAI Gym: Toolkit for developing, comparing reinforcement learning algorithms

Related Stories