Deep learning papers reading roadmap

https://github.com/songrotek/Deep-Learning-Papers-Reading-Roadmap

by zerotosixty • 10 years ago

421 25 10 years ago

syphilis22• 10 years ago

Is there a link to the book mentioned in item 1 (1.0 Book) as a PDF? The closest I could find is http://www.deeplearningbook.org/ which says I cannot get a PDF of the book, though clearly one existed at some point.

etiam• 10 years ago

I don't think it's clear that there was ever an official PDF provided. There was some discussion of making a PDF from the HTML in a previous thread dedicated to that book. https://news.ycombinator.com/item?id=11446164

lrwilke• 10 years ago

Is this not it: https://github.com/HFTrader/DeepLearningBook/blob/master/Dee...

syphilis22• 10 years ago

Yes, thank you. I didn't understand that the [pdf] was a separate link from the broken webpage link.

zk000062• 10 years ago

Nice list, but there are too many papers like this and it is easy to get stuck in theory. I would suggest to grab some simple neural network (Darknet is great for that) code and read that first. If something does not make sense, find the theory from papers.

Scea91• 10 years ago

This approach is dangerous in statistics and machine learning. Things often work on your toy examples but it may be for totally different reasons than you think and it can bite you later. Since it looks like it works to you, you will never look into the necessary theory. With the approach 'theory first' you might not get nice results as fast but I would bet that your understanding would be much deeper.

I am not saying that you need to read all the papers before coding a line, but the approach of coding your way into it will give you more shallower knowledge if you don't follow up with the theory.

zk000062• 10 years ago

In fact there is no correct answer to that and it depends on your current state of knowledge and needs for your project. It also changes over time.

I would be careful to claim that with theoretical approach you will always get better understanding.

nkozyra• 10 years ago

Agreed; I'd actually recommend against this. You should know the math and reasoning behind the specific algorithm rather than dissecting it via code.

Eridrus• 10 years ago

These papers are not very theoretical at all. You can get lucky if everything you need has been implemented in your library of choice, but most deep learning papers are highly practical engineering-driven affairs and brushing them off as unnecessary theory is just doing yourself a disservice.

annnnd• 10 years ago

Missing on the list: http://neuralnetworksanddeeplearning.com/

Great book for learning concepts and for getting a generic overview (but goes deep enough that you can jump straight into implementation if you want). I recommend it highly.

amelius• 10 years ago

What are the best video lectures out there, with emphasis on theory (not coding)?

_gfrc• 10 years ago

I can highly recommend Hinton's course on Coursera - https://www.coursera.org/learn/neural-networks

It starts on October 31st, I took it a while ago and learned a lot. The course is gravitating towards the theoretical end of the spectrum. If you want something thats a bit easier to get into, I can recommend the course by Hugo Larochelle:

http://info.usherbrooke.ca/hlarochelle/neural_networks/conte...

pinouchon• 10 years ago

I recommend this course: CS231n Winter 2016 Stanford (Lecturer is Andrej Karpathy) https://www.youtube.com/playlist?list=PLLvH2FwAQhnpj1WEB-jHm...

gamapuna• 10 years ago

In addition to cs231n which someone recommended, I would also suggest cs224d (stanford) and university of waterloo's course by ali ghodsi STAT 946

bosie• 10 years ago

i quite enjoyed nando de freitas oxford lecture: https://www.youtube.com/watch?v=PlhFWT7vAEw

leblancfg2• 10 years ago

# This downloads all the links in that page

# Just save README.md to the folder of your choice

  sed -ne 's/.*\(http[^")]*\).*/\1/p' < README.md | xargs wget -U 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:49.0) Gecko/20100101 Firefox/49.0'

leblancfg2• 10 years ago

Turns out, there are also a bunch of extensionless files in there, too.

  for file in *; do if [ -f $file ] && [ ${file:(-3)} != "pdf" ]; then mv "$file" "${file}.pdf"; fi done

markovbling• 10 years ago

Great, thanks!

I find the problem is actually how to select what content to study given a time constraint like if you had 5 hours or 20 hours or 200 hours - what should you read?

Like an exhaustive list is great but it's an optimization problem - how do I maximise understanding subject to a time constraint - which resources do I select to maximise learning in x hours/days/years?

AlexCoventry• 10 years ago

  > which resources do I select to maximise learning in x 
  > hours/days/years?

Pick a particular open-source implementation of something you're interested in, say the Google image captioning technology, read the papers necessary to understand that, and play with that implementation.

The main problem with the plan implied by the OP's bibliography is that it's not practical at all. You'll end up with a catholic knowledge of the theory and no idea about how to use it.

yalogin• 10 years ago

A bit off topic but what does one need to know/read before doing the Udacity course on autonomous driving?

erikgaas• 10 years ago

You don't really have to know anything. And the current free course does not have anything to do with neural nets. They use monte carlo, kalman filters, particle filters, PID, etc. You should feel comfortable with probability theory and if you know some linear algebra, the kalman filter example is going to make a lot more sense.

ilaksh• 10 years ago

I would skim these to select something basic and then try to experiment with a real system with those documents as a guide/reference.

Try to actually learn one level at a time.

That is my approach right now. I have read a lot but without being 100% on the fundamentals it mostly goes over my head so I am backing up. I plan to try Tensorflow examples also but I expect it will be pretty shaky until my high-level practical surface knowledge can meet in the middle with my fundamental knowledge if I can keep progressing.

jonnys1• 10 years ago

I was SOOO looking fow something like this!

Deep learning papers reading roadmap

Related Stories