MIT lecture series on deep learning in January 2020

https://deeplearning.mit.edu/

by melling • 7 years ago

221 31 7 years ago

deeplearning.mit.edu

chrispeel• 7 years ago

The site loads very slowly. If you're impatient, videos [1] on Youtube from Lex Fridman seem to be a big part of the course.

[1] https://www.youtube.com/playlist?list=PLrAXtmErZgOeiKm4sgNOk...

vo2maxer• 7 years ago

Alternative link: http://introtodeeplearning.com/

ummonk• 7 years ago

Wow, even an MIT subdomain gets the hug of death?

kick• 7 years ago

A lot of MIT subdomains are hosted by students. A MIT subdomain getting enough traffic to bring it down isn't surprising in itself. It's that this one is hosted by the administration itself (seemingly) that makes it notable.

dr_kiszonka• 7 years ago

Look exciting. Realistically, what can one expect to learn from the class? Would I learn any practical skills or is it just an overview?

rckoepke4• 7 years ago

Looks like overview to me. Guys like Francois Chollet aren't going to spend their time doing an in-depth tutorial, and even if he spent the whole 90-minute lecture walking you through the specific thing he invented (Keras), it probably wouldn't be enough time to get a naive programmer up to competency. Each week is a different lecturer.

nelson687• 7 years ago

Is this an online course?

nothis• 7 years ago

I feel like there's about 1000 hours of high quality AI lectures available for free on the internet and while I do believe in a certain amount of selflessness in education, I am skeptical that any of that is providing more than a glimpse of what you need to know to be productive at it. In other words, there's a thousand hours of material out there, which probably takes 10000 hours to actually get into so you might as well put all that effort into properly studying it at a university, getting the knowledge not covered in the lectures alone and a degree to prove it in the end.

If you're just "curious" about AI, a really good half hour lecture should get you up to speed.

ptd• 7 years ago

Why do you have to go to university to properly study machine learning?

thatcat• 7 years ago

Studying it at a university means moving and having a certain background. While i agree that these resources are hard to break down into a curriculum, theres nothing stopping you from copying a university curriculum at home and doing work on your own...

diffeomorphism• 7 years ago

theres nothing stopping you

Yet most people don't. Same for learning an instrument, carpentering etc.. While you could in principle self-study lots of things, study groups, structure, people to talk to and discuss with and even just "we meet every Thursday at noon to ..." are not negligible.

amznthrowaway5• 7 years ago

I feel like there's about 1000 hours of high quality AI lectures available for free on the internet and while I do believe in a certain amount of selflessness in education, I am skeptical that any of that is providing more than a glimpse of what you need to know to be productive at it.

Why? Whether you go to the lectures in a university or watch them online, it makes no difference. I prefer and recommend textbooks over videos though.

In other words, there's a thousand hours of material out there, which probably takes 10000 hours to actually get into so you might as well put all that effort into properly studying it at a university, getting the knowledge not covered in the lectures alone and a degree to prove it in the end.

What knowledge "not covered in lectures alone" or books do you need? Why do you think having a signal of pedigree somehow confers this knowledge onto people.

If you're just "curious" about AI, a really good half hour lecture should get you up to speed.

You don't have to either be an expert or a complete layman. This gatekeeping is ridiculous. I've worked with many phds and most of them are not even close to as competent as folks who are naturally gifted and put in the work to pick up the topics.

buboard2• 7 years ago

ML is a very applied subject. There is entirely too little theory people need to know. In fact the most impressive vision/nlp architectures are indeed uninterpretable alchemy. It would make very little difference to study it at a university. Unless of course you 're going for probability theory.

unishark• 7 years ago

Are you defining "need to know" as in the state of the art methods of today? Problem is the next stage after mature technology is commodity. We'll be getting daily spam from India offering to fulfill all our "A.I. design" needs just as with web design. If all you have to do is follow a blog post or video on how to use a prepackaged framework to get a job done, than everyone else can do it too.

amelius• 7 years ago

For example, it would be refreshing to see a deep-learning course with math at a higher level than high-school level.

GrantZvolsky• 7 years ago

These lectures are for university students as well. Anecdotally, MIT lectures were a vital part of my education while at university; they perfectly complemented our own. Some professors even endorsed them.

waiseristy• 7 years ago

So Lex Fridman curated this page and is the subject of many videos within it. Seems like Missy Cummings and Filip Piekniewski's assumption that MIT has scrubbed him from the site is unfounded.

[1] https://twitter.com/missy_cummings/status/117949700363566285...

[2] https://blog.piekniewski.info/2019/11/18/late2019-the-wizard...

Lex's blocking of dissenting opinions on twitter is still egregious though.

Vervious• 7 years ago

Notably, these are all winter courses, taught over the course of three weeks in January.

They don't have the formality and institutional support of a semester-long course. Often they are taught by students. MIT has a ton of people working in this area, and I'm not sure this particular group of people is representative.

Mugwort• 7 years ago

I'm definitely going to listen to the Lex Fridman lectures simply because I enjoyed his interview with Judea Pearl so much.

codingslave4• 7 years ago

The gap that I see in current machine learning is that everyone is learning how to use the popular models, but no one knows how to construct a new model that solves a new problem. So everyone can download word vectors and use them for what they're good at, but the second you get off the beaten track, almost all machine learning practitioners fall flat. I really dont think this is due to how new the field is, rather that very few people have the mathematical maturity and experience to actually use optimization theory and linear algebra to construct new highly specific language models. There is simply no information available about doing this. You can learn about the underpinnings of matrix factorization and how that relates to word vectors, take that further and read about eigen vectors, but still, its too thin.

M5x7wI3CmbEem10• 7 years ago

wouldn’t this just require a few textbooks in linear/non-linear optimization, mathematical modeling, and real analysis?

codingslave4• 7 years ago

I think its way different than that, those would just be precursors, and in cases like real analysis, superfluous. Instead it would look something like, I have a Universal Sentence Encoder architecture, but its not performing well on my data, aside from tweaking the training set, how can I take this architecture and change it to work better with my individual problem? Assume here that the problem one is trying to solve is very similar to the models original use case. The number of people on the planet that can do this successfully, without wasting months of time messing around with tensorflow is extremely small. But this is where the value is. These massive catch all models only work for the people creating them, just jamming them into any NLP model will always produce sub par and probably unusable results

Buttons840• 7 years ago

What mathematical maturity went into designing the most popular models? What mathematical maturity lead to using ReLU instead of tanh for activation functions, for example?

As far as I know, a lot (most?) advances in the field are just trying new ideas that happen to work. Is this correct?

codingslave4• 7 years ago

Sure, there's lots of trial and error. But consider something like Universal Sentence Encoders (USE) versus facebooks InferSent. USE is superior, mostly because its new, but still under performs infersent in a few areas. This is the kind of thing where actually building a more specialized model for question answering or textual similarity could see huge performance boosts for companies, but nobody is doing it. If they are, its under lock and key. Anyone looking to perform these tasks is mostly just pulling the code, tweaking the data, messing with the heads of the networks, and then calling it a day.

EDIT:

copying and pasting this from another answer:

I think its way different than that, those would just be precursors, and in cases like real analysis, superfluous. Instead it would look something like, I have a Universal Sentence Encoder architecture, but its not performing well on my data, aside from tweaking the training set, how can I take this architecture and change it to work better with my individual problem? The number of people on the planet that can do this successfully, without wasting months of time messing around with tensorflow is extremely small. But this is where the value is. These massive catch all models only work for the people creating them, just jamming them into any NLP model will always produce sub par and probably unusable results

denzil_correa• 7 years ago

One needs to form an experimental design with a ability to detect the challenge, understand the properties and come up with an appropriate computational solution to that challenge. This isn’t just for Machine Learning but for any kind of algorithm you develop.

jononor• 7 years ago

What kind of learning resources to use for this?

MIT lecture series on deep learning in January 2020

Related Stories