Accelerating Deep Neuroevolution: Train Atari in Hours on a Single Computer

by mkvorwerck • 8 years ago

155 38 8 years ago

minimaxir• 8 years ago

Modern desktops also have GPUs, however, which are fast at running deep neural networks (DNNs).

That's the rub: deep learning is restricted to CUDA/cuDNN/NVIDIA GPUs, which exclude a large amount of desktops with AMD graphics (e.g. all modern Macs).

Current cloud GPU prices have dropped enough such that this approach may be pretty effective with spot/preemptible instances.

21• 8 years ago

cuDNN/NVIDIA GPUs, which exclude a large amount of desktops with AMD graphics

And whose fault is that?

fluxsauce• 8 years ago

The question isn't about fault, it's about accessibility and availability of commodity hardware. Consider the position of someone who made a financial decision to go with AMD to build a workstation and didn't have the budget to swap out the video card later.

singhrac3• 8 years ago

Does anyone know why AMD has dropped the ball so hard on not supporting Tensorflow/PyTorch? It seems like the kind of work 2-5 talented engineers could do (i.e. push harder on HipM, reimplement CuDNN, etc.), and would seriously impact sales, right? I probably misunderstand either the hardware limitations or business side from AMD's perspective.

currymj3• 8 years ago

They are working on it pretty actively [1], you can see a progress chart on their website. I think some of the PyTorch devs have said they'd welcome AMD compatibility. If you go to the ROCmSoftwarePlatform github page you can see some of their ports, most of which seem to be actively developed.

[1] https://rocm.github.io/dl.html

Nokinside• 8 years ago

You misunderstand the difficulty in the software side.

Developing competitive CUDA equivalent is much harder task than what you assume. There needs to be lots of tooling and need for optimizations in software to make these low level libraries high performance. The amount of testing and benchmarking to make the right choices may take many people working full time. Implementing the functionality is not enough.

AMD has had apis for a long time and it has an has an open source deep learning stack but it's not good enough. AMD also has CUDA to HIP converter but the results are not competitive and miss features.

Both AMD and Intel are probably getting there eventually and Nvidia is cashing the monopoly phase before the prices stop.

fenollp• 8 years ago

Coriander [1] looks promising: "Build NVIDIA® CUDA™ code for OpenCL™ 1.2 devices".

[1] https://github.com/hughperkins/coriander

ironrabbit• 8 years ago

a run that takes 1 hour on 720 cores can be run on the CPUs of a 48-core personal computer in 16 hours

Is calling a 48-core machine a "personal computer" a bit of a stretch, or am I missing something?

segmondy2• 8 years ago

Dunno, I have a 32 core, 128gb ram that I bought for $1000. I have a Xeon phi 60core card that I bought for $200 sitting by it ready go go in when I get the chance. When GPU prices drop, I'm probably going to add 2 1080Ts.

I'm just a regular computer user.

banachtarski• 8 years ago

*eyeroll

You're also one of "those" people

taeric4• 8 years ago

Define "regular computer user." What are you doing with these machines that comes close to using all of the cores?

derefr• 8 years ago

You’re a computer hobbyist, you mean. You have to have done a whole lot of bargain-hunting to set that build up at that price. The “regular computer user” doesn’t bother with that—they just buy what they can afford at their local shop, which usually means a rather crap laptop.

(Mind you, it doesn’t take any more technical knowledgeability than most users to bargain-hunt like you have; but knowing that the product can be of varying quality at a given price, and caring enough to put in the time to get something of good quality, makes you an exception, in the same way that people who e.g. comparison-shop for quality suits or shoes are an exception. It’s a general mentality that most people don’t subscribe to—neither the “buying the best” of the rich, nor the “economizing” of the poor, but rather the optimizing ROI of the economics-minded. And even an economist will only apply this kind of thinking to cases where they actually believe this thing will affect their life enough that the research time-cost is worth it—i.e. when it’s a hobby of theirs. Economics-minded computer hobbyists are a rather more niche set than “regular computer users!”)

mysterydip• 8 years ago

Where'd you get them? I could use an upgrade for that price.

igravious• 8 years ago

It's more than a bit of a stretch I reckon. That's how the all but hermetically sealed SV tech scene warps some engineers sense of what a regular nerd has access to I suppose.

singhrac3• 8 years ago

I think the intended implication is that it's accessible to individual researchers that don't work at Facebook/Brain/DeepMind with gigantic clusters (and the engineers able to maintain those clusters). Unfortunately deep learning is still expensive, but many areas of science are (most biology research, high energy physics, etc.). That's why we have grants.

Symmetry• 8 years ago

I think they're using "personal computer" there only in the narrow sense that a dual socket tower workstation conforms to the PC spec. But then they go on to GPU performance which allows a machine you could reasonably have as your home computer to do this quickly.

currymj3• 8 years ago

i also raised my eyebrows at this. i think "workstation" might have been a better choice of word than "personal computer". but people, especially in the sciences, do actually buy desktop computers with these sorts of specs -- they run like $10k-20k, probably.

haZard_OS• 8 years ago

Closer to the 20k end, I'm afraid. More options are opening up now with GPU processing power on less expensive desktops ( ~ $13k ).

ansible• 8 years ago

If you don't care about your electricity bill, you can buy used Xeon workstations on your favorite auction site.

A couple years ago, we picked up a HP Z800 workstation with two 12-core processors and 24GB of RAM for $400.

The main downside is that if the power supply (proprietary) or motherboard die, there's not easy fix for it other than buying another of the same model.

wpietri2• 8 years ago

Absolutely!

I wanted something to host my private and experimental stuff. A friend of mine who tracks used server auctions pointed me at some servers that were Dell systems clearly custom built for one of the large SV companies. I paid $450 for a box with 72GB of RAM and 16 cores. At that price, I just bought a second one for spares. Which I haven't needed to use in the 4 or 5 years I've been using it.

It has been great.

foobiekr• 8 years ago

WeirdStuff also used to be great for this.

lopmotr• 8 years ago

If you only had 10% of the cores, that hour would become a day, which is still pretty good.

cshenton2• 8 years ago

If anyone’s interested, training a single game is affordable on spot instances.

I have a replication including cloud formation scripts here https://github.com/cshenton/neuroevolution

yazr• 8 years ago

Nice work

Can u elaborate more on the network requirements? How much bandwidth is required? (I presume latency is less of a problem)

cshenton2• 8 years ago

The algorithm only communicates sequences of seeds so the communication overhead is very low. The master server doesn’t even break 1% cpu servicing a few 100 workers.

Accelerating Deep Neuroevolution: Train Atari in Hours on a Single Computer

Related Stories