HN user

aeoost

65 karma
Posts6
Comments8
View on HN

I'm currently working on an RL project based on an OpenAI Gym environment and have been reviewing the different frameworks available. So far I’ve come across:

- OpenAI Baselines (more a collection of algorithms than a framework)

- Keras-RL (looked ideal but has been abandoned)

- Tensorflow Agents (An 'official'? Tensorflow library, but very basic- only one algo at present)

- rllab (Developed by OpenAI people but seems to be abandoned)

- OpenAI Lab (?)

- TensorForce

My main concerns are: 1. Soundness of the algo implementations. 2. Modularity, ease-of-use, compatibility.

I first looked at Baselines as it seemed to best address the first concern but ran into frustrations when for example the DeepQ implementation didn’t work if my Gym’s action_space was a Tuple space. I am working with a team unfamiliar with RL so want something that is as plug-n-play as possible, like Keras. So far TensorForce looks promising. Can anyone add anything more? Thanks

Try F# my man. In my experience its the best option for a statically typed functional language if you want to actually get stuff done and not just dick around. Also as you're coming from C# it would be the logical choice since the tooling is all there and you're already familiar with it. Visual Studio Code + Ionide plugin is what I use. I really don't understand why F# isn't more popular. Maybe it has something to do with its Microsoft roots making it uncool among the hip functional crowd?

Tensorflow sucks 9 years ago

I got fed up with up with Python and switched to F#. My main issue was the lack of static typing made testing and debugging a major timesink and headache.

It's early days but so far F# it ticks all my boxes and I'm finding it an extremely nice language to write in. It is just as terse as Python (due to type inference) but statically typed, and there is a great plugin Ionide for VSCode which makes for a really polished development environment.

Plan is to use Microsoft's CNTK for ML/DL stuff.

For anyone frustrated with Python's duck typing, I highly recommend you check out F#.