optbuild

After having my first Linear Algebra course I came across the online course called Linear Dynamical Systems by Prof Stephen Boyd of Convex Optimisation fame.

Every lecture was so eye opening. I couldn't believe that linear algebra could be taught in such a context with such a variety of application domains.

The lectures are all available online with the assignments : https://ee263.stanford.edu/archive/

Pick one university and do their core CS track with the materials available. Like CMU, Berkeley, MIT, Stanford, etc. Then start building stuff and learning from more advanced CS and math courses/books/papers.

CMU has two open courses on deep learning which are very good.

1. Deep Learning - https://deeplearning.cs.cmu.edu/F23/index.html

2. Deep Learning Systems - https://dlsyscourse.org/

The second course dives much deeper into the internals of the libraries and all.

He is not a Twitter bro though. He has developed software extensively. He is a PL researcher, a professor at BrownU, now working in computing education.

Which book by Nocedal and Wright? Can someone link to it?

A finely written book for beginners. I like how it slowly explains stuff rather than doing in Theorem Proof style short books.

Wow! such a twist. But the advice is spot on!

To clarify: By "cut out" I mean "enjoys doing that activity".

I really want to know about the outcome of reading HTDP with some programming experience, like you, versus taking it as a first course.

Are there any introductory programming or CS courses that you have in mind for someone starting out?

I see many people dive into ML research by tinkering along the way. Although I have no objection to particular tastes, two courses that made modern ML easy for me were:

1. Linear Dynamical Systems by Stephen Boyd [https://ee263.stanford.edu/archive/]

2. Convex Optimisation by Stephen Boyd. [https://see.stanford.edu/Course/EE364A]

Both lecture series and course notes are availabe from the official page of the instructor.

It is very hard to pinpoint a single moment. Rather I would like to list a few books that I feel showed me that mathematics can be beautiful and interesting.

1. Measurement by Paul Lockhart

2. An Infinite Descent into Pure Mathematics by Clive Newstead (available @ https://infinitedescent.xyz/). It taught me basic discrete math and proof writing.

3. Apostol's Calculus Vol 1. It is just so beautiful.

I don't know about mathematicians, but everyone should read How to Design Programs book when they are starting out. Fantastic way of forming great mental models.

Can you elaborate more on the details of your learning?

What book and project did you start with?

Why aren't more universities teaching functional programming first to form good mental models for the students and then show them when to use state and when to not.

Why is everyone teaching Python?

Adding them to the original description.

Both the book and a course:

Behave - Robert Sapolsky

Human Behavioral Biology - https://www.robertsapolskyrocks.com/intro-to-human-behaviora...

As a side effect of the course I also read Chaos by James Gleick and my mind was blown.

Programming Advice -

Try to reverse engineer stuff by surfing through documentations and protocols. There is a youtube channel that does this: https://www.youtube.com/@LowByteProductions

Also start making small hobby projects that don't scale. Just to get the feel of how things work.

What kind of programming and math proficiency is good to have before someone starts to implement papers?

How much does it cost to host? Where do you host it?

I still got more ideas than time.

Not time. Getting bottlenecked by not knowing what can be done or can't be using computer programming.

Recursion is basically the other side of the coin for mathematical induction. I would suggest that you learn a functional programming language and mathematical induction.

You can try out: How to Design Programs at https://htdp.org/ if you want to learn more about how recursion is used in programming and where.

For mathematical induction you make read:

1. What is Mathematics by Courant and Robbins

2. Introduction to Mathematical Thinking by Keith Devlin

I am comfortable with autograd/computation graphs, PyTorch, "classic" neural nets and ones used for vision-type applications, as well as the basics of Transformer networks (I've trained a few smaller ones myself) and RNNs.

Maybe I am a bit off the track. But how do someone reach this state?

Some channels that really explains concepts very well:

1. Veritasium

2. Intro to Mathematical Reasoning: https://www.youtube.com/playlist?list=PLoxJTbDttvt5irjK-gD_b...

3. Human Behavioral Biology - https://www.youtube.com/playlist?list=PL848F2368C90DDC3D