World Models (the long version)

https://adgefficiency.com/world-models/

by climate-code • 6 years ago

48 10 6 years ago

adgefficiency.com

backpropaganda• 6 years ago

World Models is a good paper, but is definitely not the top three most significant RL papers.

Sureai• 6 years ago

Which are the top three mist significant RL papers?

johnmoberg• 6 years ago

Other than the AlphaZero papers and DQN, maybe TRPO? (assuming we're talking about deep RL)

gas9S9zw3P9c• 6 years ago

To be fair, the post does not say that. It says that it's "one of the papers significant to the author" - a subjective ranking.

jefft255• 6 years ago

I loved this paper. It's a good example of a VAE+RNN style model learning for MBRL. It's well exposed. Sure the results aren't the best but it's a well worth read.

PeterStuer2• 6 years ago

Side point on the AWS cost for the project.

Total spend was $3.648 over 11 months. For that amount you can tailor a pretty powerful workstation these days, saving you quite a bit of AWS headaches and overheads (and yes, potentially trading those for others, I know).

Still, this seems like a decent candidate for omitting the cloud as at first sight I do not detect any asks that scream out cloud to me.

Thoughts?

Eridrus• 6 years ago

For a hobbyist, you definitely don't want to use the cloud, buy yourself a good GPU (I got a great deal 2nd hand in eBay) and you'll save a lot of cash.

What you need the cloud for is parallelism. If you want to do any sort of hyperparameter tuning doing it on a single computer is going to leave you waiting for days.

ad404b8a372f2b9• 6 years ago

I can't speak to the author's workflow but I also do ML research and I often need heavy bursts of computation with long stretches of nothing in between. So I might spend a few hundred bucks in a couple of days by using multiple very powerful instances at the same time. Even if the cost ends up being the same as a workstation over a year, when it matters the cloud gives me results faster than a workstation would and this enables fast iteration over ideas.

PeterStuer2• 6 years ago

Bursting is definitely a point. It wasn't clear to me from the data at first glance whether this was used here.

bluetwo• 6 years ago

I did appreciate them detailing these costs.

World Models (the long version)

Related Stories