All the steps needed to create deep dream animations on EC2

https://github.com/JohnMount/CaffeECSExample

by jmount • 11 years ago

72 19 11 years ago

jmount7• 11 years ago

I really wanted to run the deep dream and deep dream animation scripts people have created. I am now re-sharing the instructions I found (with links to original guides). It isn't an automated install. You pretty much have to paste the lines one by one. But I did replace any "edit" steps with append or patch.

To do this on a fresh Ubuntu EC2 g instance there are a lot of steps- but I have tested them and put them all in one place (with links to the original sources and guides). I have CUDA up but not CUDNN as I haven't found how to legitimately download CUDNN without registering on the NVIDIA website.

Again: credit to the actual creators and all the original guide authors.

jerf2• 11 years ago

For better or for worse, with shell, the line between "a list of instructions to be typed in" and "shell script" is very, very thin.

Toss a

    #!/usr/bin/bash

    set -e

on top of that, call it "setup.sh", and give it a runthrough, and you're halfway to it being as "automated" as it needs to be to start with. You're fall more likely to get people start PR'ing a running shell script than dead text files.

(set -e will make the script stop when something returns with an error exit code. Despite temptation, given the size of this script I do advise doing your best to work with that... this is just the sort of script that if you let it blunder ahead blindly, when something doesn't work you'll need an hour just to figure out what didn't work, let alone fix it....)

jmount7• 11 years ago

Some of the steps require user input. Is the "set -e" for dealing with that? Also at least one reboot is needed, the script depends on a lot of external sources being up, and I have seen different behavior on the kernel upgrade steps.

dktbs2• 11 years ago

Thanks for updating and organizing all of this. I got a working EC2 instance with CUDA and CUDNN working yesterday but it took some time. Having all of this is one place is excellent.

deepnet• 11 years ago

Thanks, the Caffe on the PYTHONPATH was the bit I needed.

fennecfoxen• 11 years ago

That's pretty insane. It makes me really appreciate that I get to use package managers and Puppet/Vagrant scripts for most of the stuff I program.

jmount7• 11 years ago

Yeah, I don't normally "do my own infrastructure" (I know, "it shows"). Usually the client or vendor has supplied the build. And I didn't want to commit to a package manager early in the exercise (in case it turned out to be incompatible with something one of the dependent projects did). So I just tried to start from the Caffe instructions and then fix things that broke.

The main components turned out to be: NVIDIA/CUDA, python, Caffe, ffmpeg.

I was very surprised how many steps ffmpeg took.

facetube2• 11 years ago

Would you be willing to publish an AMI of the completed image? Seems like that'd make it really easy to run without all of the package management work.

shreyask• 11 years ago

These steps are cool and very detailed, also check out https://github.com/VISIONAI/clouddream which is containerized so you can try it on local machine as well as on EC2.

jmount7• 11 years ago

Thanks, that might be the better option for a lot of people. I'll add that to the readme.

azinman22• 11 years ago

Or... Just use deepdreamer.io

dktbs2• 11 years ago

That is definitely the more convenient option for running the defaults, but I think this is for a different usecase. If someone wanted faster results and full access to edit the python code and caffe models, this would be the better option.

azinman22• 11 years ago

It's a bit better than the defaults because it generates many at once. But ya if you really wanna hack on it...

amelius• 11 years ago

This is really cool, but honestly, I'm getting a bit tired of seeing all those "eyes". I want to feed it my own training image set. Is that possible?

Alternatively, is there a way to specify I don't want to enhance on certain outputs, like "dogs" and "faces"?

All the steps needed to create deep dream animations on EC2

Related Stories