imakecomments

. there’s so much hand holding through required coursework (and homework) that one can’t hardly think deeply about anything actually original.

Take an actual graduate course in pure mathematics, very little hand holding.

there’s basically no time or space for one to explore one’s own thoughts and program unless it specifically matches up with what someone else (namely an advisor) wants to do.

Not necessarily true. Find an adviser that will support YOUR interest (they exist). Sounds like you have some biases against PhD programs without actual experience of being in a good one.

Any PreCalculus text -> Stewart Calculus -> Strang + Axler Linear Algebra -> a Calculus based Statistics & Probability book. Do the problems in the books. Read each chapter. If you do that you'll know Calculus and Linear Algebra better than most. If you want to then move up from there study Real Analysis and higher level math like Measure theory (but this may not be necessary for your purposes).

In my experience, at my school testing is done to pass a professor's designed test cases. Usually you don't know the professor's test cases in advance.

A PhD is paid. Universities near me pay around $30,000+/yr.

Thanks for taking the time to clarify your points. Although, I still disagree on a few views. Even if OP is 30 there isn't an inherent reason college is too late either at the undergraduate or graduate level. If OP had the funds or made specific life choices they could enroll back into a university and study. Some people aren't married/have kids at that age and can work side jobs or what have you to enroll back in school. Not everyone studied CS in their 20s and are satisfied with the careers they had up until that point. I know of individuals that made career changes in their mid-20s/30s. Most of them went back to school in one form or another. I think for those that did that actually set themselves up for a "career upgrade". I've seen students in class that look much older than everyone else taking undergraduate courses. I don't know their story but they aren't treated any differently by the university and I've even heard from professors they prefer older students over younger ones as their graduate students. Also, I've seen a CS PhD student that is 40+.

I don't really agree with this. 30 isn't that old. Also, some 30 year olds still look like they're in their early 20s, so you can't tell their age to begin with.

OP, in a university setting it is easier to find mentors. Look for professors, grad students and in-class TAs if possible.

This is a bit of an overgeneralization. Are all computer science graduates software developers? Are professors with PhD degrees in CS not considered Computer Scientist? Your statement assumes graduates stop at the BS level and get programming jobs.

Most other industries we like to compare ourselves to are leagues above ours in individual merit.

I don't think that rings true to those with PhD degrees doing academic research in CS.

So, unless you think they are lying, your point doesn't stand on limiting low-income students.

You missed the point. Poor kids in general don't go to Math & Science high schools, or high schools that offer college level courses in Calculus I-III, Number Theory, Abstract Algebra, Differential Equations, Linear Algebra, 1-2 semesters worth of programming, Data Structures, etc. These kids don't have have access to private tutors to ensure they ace their SATs, SAT Subjects, ACTs etc. They aren't enrolling in multiple ECs and they don't have teachers with PhDs writing them letters of recommendation. These kids don't have parents that went to colleges and don't have the resources handed to them as the kids that are getting into Brown, Harvard, Princeton, Stanford etc. If you look at the lower level high schools in America, they barely give any of their kids any advantage. They actually put them at a major disadvantage compared to the elite high schools.

It's great elite colleges have these needed based programs. But by virtue of their admissions criteria they are excluding lower income students on a vast scale.

Audit edx's Calculus sequence, taught by MIT. The courses are: Calculus 1A, 1B, 1C. You can watch MIT OCW's Linear Algebra course with Strang and/or enroll in: "Linear Algebra - Foundations to Frontiers" on edx. Use Khan academy for supplemental Calculus & Linear Algebra review. You can get Stewart's Calculus text and read through it/attempt the problem sets. Once you have a solid Calculus/Linear Algebra review you can take a look at: "Statistics 110: Probability" which is found free here: https://projects.iq.harvard.edu/stat110/home.

I'm mostly interested in "modern" mathematical logic then. I'm very interested in learning category theory and its connections to programming language theory. I already know a little bit of category theory, but am open to any good beginner sources. I'm also interested in classic recursion theory and a bit of proof theory with its connections to CS. I don't know many people doing any of this and it doesn't seem that popular in math departments.

I'd like to see a list like this that included the field of mathematical logic. For whatever reason mathematical logic no longer seems to be a "popular" area of research, despite its deep connection to theoretical computer science. But there are distinction in study, as computer scientist tend not to go deeply into computability theory like a traditional mathematician would.

You may be interested in Automated Theorem Proving:

I quote the author Geoff Sutcliffe here:

"Automated Theorem Proving (ATP) deals with the development of computer programs that show that some statement (the conjecture) is a logical consequence of a set of statements (the axioms and hypotheses). ATP systems are used in a wide variety of domains. For examples, a mathematician might prove the conjecture that groups of order two are commutative, from the axioms of group theory; a management consultant might formulate axioms that describe how organizations grow and interact, and from those axioms prove that organizational death rates decrease with age; a hardware developer might validate the design of a circuit by proving a conjecture that describes a circuit's performance, given axioms that describe the circuit itself; or a frustrated teenager might formulate the jumbled faces of a Rubik's cube as a conjecture and prove, from axioms that describe legal changes to the cube's configuration, that the cube can be rearranged to the solution state. All of these are tasks that can be performed by an ATP system, given an appropriate formulation of the problem as axioms, hypotheses, and a conjecture.

The language in which the conjecture, hypotheses, and axioms (generically known as formulae) are written is a logic, often classical 1st order logic, but possibly a non-classical logic and possibly a higher order logic. These languages allow a precise formal statement of the necessary information, which can then be manipulated by an ATP system. This formality is the underlying strength of ATP: there is no ambiguity in the statement of the problem, as is often the case when using a natural language such as English. Users have to describe the problem at hand precisely and accurately, and this process in itself can lead to a clearer understanding of the problem domain. This in turn allows the user to formulate their problem appropriately for submission to an ATP system.

The proofs produced by ATP systems describe how and why the conjecture follows from the axioms and hypotheses, in a manner that can be understood and agreed upon by everyone, even other computer programs. The proof output may not only be a convincing argument that the conjecture is a logical consequence of the axioms and hypotheses, it often also describes a process that may be implemented to solve some problem. For example, in the Rubik's cube example mentioned above, the proof would describe the sequence of moves that need to be made in order to solve the puzzle.

ATP systems are enormously powerful computer programs, capable of solving immensely difficult problems. Because of this extreme capability, their application and operation sometimes needs to be guided by an expert in the domain of application, in order to solve problems in a reasonable amount of time. Thus ATP systems, despite the name, are often used by domain experts in an interactive way. The interaction may be at a very detailed level, where the user guides the inferences made by the system, or at a much higher level where the user determines intermediate lemmas to be proved on the way to the proof of a conjecture. There is often a synergetic relationship between ATP system users and the systems themselves:

The system needs a precise description of the problem written in some logical form, the user is forced to think carefully about the problem in order to produce an appropriate formulation and hence acquires a deeper understanding of the problem, the system attempts to solve the problem, if successful the proof is a useful output, if unsuccessful the user can provide guidance, or try to prove some intermediate result, or examine the formulae to ensure that the problem is correctly described, and so the process iterates.

ATP is thus a technology very suited to situations where a clear thinking domain expert can interact with a powerful tool, to solve interesting and deep problems. Potential ATP users need not be concerned that they need to write an ATP system themselves; there are many ATP systems readily available for use. At the 1st order level some well known and successful systems are Otter, E, SPASS, Vampire, and Waldmeister. Higher order systems include ACL2, Coq, HOL, and Nqthm. For examples of problems written in classical 1st order logic, the TPTP problem library is a useful resource, which also has a simple interface for users to try out many 1st order ATP systems.

Fields where ATP has been successfully used include logic, mathematics, computer science, engineering, and social science; some outstanding successes are described below. There are potentially many more fields where ATP could be used, including biological sciences, medicine, commerce, etc. The technology is waiting for users from such fields."

Link: http://www.cs.miami.edu/~tptp/OverviewOfATP.html

Read "Discrete Mathematics" by Epp. Probably the easiest Discrete Math intro I came across.

No problem.

This is just my opinion and I'm sure it differs from others...

Roughgarden's class is advance and expects mathematical maturity. You may find his course quite fast and rough if you are a beginner.

Sedgwick's class is much easier. He is a bit boring and tries to use "real life" examples (in some instances) from the physical sciences to make the material relatable. This in my opinion detracts from the material. Also, he doesn't always fully explain where he got some of the big ohs here and there.

My advice? Follow MIT's OCW course (it uses CLRS). Supplement it with Algorithms Unlocked, the Khan Academy link in OP and CLRS. If you use those 4 resources and put in the work you'll understand the material.

All 4 sources have Thomas C's DNA touch to it (he is the C in CLRS). So you'll find it consistent when you read from one source to the other. After reading/hearing the same thing about 4 different times in 4 different ways it'll begin to click.

Order of easiness is probably Khan Academy > Algorithms Unlocked > MIT Algorithms Course > CLRS.

Algorithms Unlocked is like "pre-CLRS" and Khan Academy's version is the TL;DR version of Algorithms Unlocked.

Hope this helps.

Below are the links,

https://www.amazon.com/Algorithms-Unlocked-Press-Thomas-Corm...

https://www.amazon.com/Introduction-Algorithms-3rd-MIT-Press...

https://www.khanacademy.org/computing/computer-science/algor...

https://ocw.mit.edu/courses/electrical-engineering-and-compu...

I don't like Sedgewick as much as CLRS. Sedgewick seems to assume knowledge in the physical sciences when explaining examples.

As a follow up to this resource I recommend,

Algorithms unlocked: https://www.amazon.com/Algorithms-Unlocked-Press-Thomas-Corm...

CLRS: https://www.amazon.com/Introduction-Algorithms-3rd-MIT-Press...

Both include the same author as the one in this article (Thomas Cormen).

OP you will want to be comfortable with,

-Multivariate Calculus

-Statistics & Probability (with Calculus)

-Linear Algebra

-Optimizations

as a baseline. You'll need more as you progress.

You want to be comfortable with the math review in: http://www.deeplearningbook.org/

I'd wager to say most people on HN would drop out of an actual graduate level course on ML. These courses (from my experience) are more research oriented and less about "hacking".

Calculus, Linear algebra, Statistics, Probability theory. You won't get far studying "basic" ML without an elementary understanding of those subjects. Unless you blackbox the implementations and hack them, but you'll have trouble understanding why things work/when to use certain techniques vs others or how to tune for optimizations.

There are other examples of mathematical late bloomers. Check out https://en.wikipedia.org/wiki/Stephen_Smale

He almost got kicked out of graduate school. Later went on to win the Fields Medal.

I recommend learning discrete mathematics, then data structures and algorithms.

I cannot stress enough how important mathematical foundations is. It'll make everything else much easier to learn. I haven't read the book but heard good things about: https://www.amazon.com/Discrete-Mathematics-Applications-Sus... as a beginner text.

Coursera has multiple offerings on Data Structures / Algorithms -- find one that works best for you.

For instance:

https://www.coursera.org/learn/introduction-to-algorithms

https://www.coursera.org/specializations/algorithms

https://www.coursera.org/learn/algorithmic-thinking-1

https://www.coursera.org/learn/algorithmic-toolbox

By doing all of those you'll get a good introductory exposure to the topics.

You should also look at a rigorous course offering of Algorithms. MIT has a few online to view.

Some readings for a beginner are:

https://www.khanacademy.org/computing/computer-science/algor...

https://www.amazon.com/Algorithms-Unlocked-Press-Thomas-Corm...

https://www.amazon.com/Introduction-Algorithms-3rd-MIT-Press... (not beginner level but classic)

After all of this you should be fine with diving into interview books. You'll want to whiteboard solutions and be able to do all the difficult problems. Look into sites like leetcode, glassdoor and be able to do the difficult problems posted there.

I would recommend knowledge in Python. If you don't know Python prior to reading this book, then you won't learn it here.

The Linear Algebra presented in the book is easy, but if you aren't familiar with Linear algebra then it would be a benefit to learn some basics before reading.

Familiarity with mathematical notation and mathematical reasoning is also a must. If you don't have a basic understanding of being able to follow mathematical arguments, then some side comments (that aren't explicitly stated) may seem mysterious to you.

This book is very suitable for someone with a background in Python and basic Linear Algebra/mathematical reasoning knowledge.

The point of this book is to understand the math/algorithms and not treat the algorithms as blackbox solutions. You'll learn about processing your data, dimension reductions, etc., etc.

You may have to do some outside studying depending on your background, but the author provides those resources for you in the text. Overall this is a very good book and the author did a good job at writing it.

I don't know exact percentages, but if you have top tier publications, or at the very least have relevant research experience, solid recommendation letters & faculty sponsorship then your chances of being admitted increase dramatically. Getting into top graduate schools isn't easy. GPA wise, usually your junior/senior year is the most important aspect. Good luck!

Not him but in general you need tier 1 publications, recommendation letters, a strong statement of purpose (very clear idea of what you want to do research in) and faculty fit. GRE scores are generally used as a filter. They won't get you in, but they will keep you out.

It's been mentioned elsewhere but I recommend: https://www.youtube.com/playlist?list=PL601FC994BDD963E4

Seems like a few lectures are missing but if you google around you might be able to find them somewhere.

Chapter 1 is completely trivial to anyone with mathematics training. It makes no difference to me if the author expands upon the section or not. It doesn't hurt my demographics of readership because we wouldn't read that section anyways. You know who would read it? The 'practitioner'. Someone that hasn't seen a matrix since high school or freshman year of college.

The interviews the authors give paint the picture this book is for the 'practitioner'. If Chapter 1 is meant for a brief review then don't advertise the book for a complete practitioner/beginner. Either make the book for the practitioner or not. If you do, then don't pretend to serve introductory math in it that the unfamiliarized reader will read and understand. They fail at their purpose there. So either make that chapter useful for the practitioner or leave it out and assume the mathematicians already know it. Maybe put it in an appendix and let us get to the meat quicker. It honestly does not take much time to define what a matrix is, give an example, define matrix multiplication, give examples etc. Same applies with basic definition and examples of derivatives. These are mindless mechanical procedures anyone can learn. It wouldn't take too much extra space to include some thoughtful examples. Maybe I should write an 'introductory group theory' textbook and start discussing geometric group theory 2 pages in if we want to get into not serving an intended audience's purpose.

I like what the author's are doing. I'm on their side, but I'm making suggestions that could serve a wider audience.

Glad I could provide some feedback. I think what you're aiming to do is exceptional. Keep up the good work.

I didn't notice the appendices in the draft. This may help, but the version I reviewed has a brief review of Linear Algebra and Statistics in Chapter 1. The language is written in a way that assumes mathematical familiarity. Putting this level of 'sophistication' in a "practitioners" book early on may turn off a certain demographic of readers. I say 'sophistication' because it's relative to the reader-- to someone with a math degree this section isn't sophisticated (in fact they'd find it completely trivial) and can be skipped. To a software engineer that hasn't taken a Linear Algebra/Statistics class in 10+ years this may appear too much for them. You risk losing the readers entirely or having them skip those sections. Again, this review is not friendly for the beginner and the 'practitioner' title is misleading here.

In my opinion it wouldn't be too difficult or much effort to define what these mathematical objects are and show basic examples with basic computations to solidify the concepts. The notion of gradient descendent & derivates (or partial derivatives) isn't that difficult to understand and could be easily explained in a page or less.

For example when you discuss the Outer Product:

"This is known as the “tensor product” of two input vectors. We take each element of a column vector and multiply it by all of the elements in a row vector creating a new row in the resultant matrix."

It would be nice for the beginner to see an example of this and as stated it wouldn't take much space in the book to provide one. I think these sort of things would differentiate your book from others. If you made it more friendly more 'practitioners' would be willing to read/use it end-to-end.

Regarding your book, have you expanded on the math section? I saw somewhere a draft of the material and the math review seemed to be broken up into short paragraphs. These short paragraphs lacked examples and appeared to assume previous background knowledge in the subject, which seems contradictory to the book's title and aim. For example.. I believe you mentioned somewhere "The Jacobian is a m x n matrix containing the 1st order partial derivatives of vectors with respect to vectors." -- Since I have a math background I can understand what you write. But for someone with little to no math background (e.g. a software practitioner) this may throw them off.

I am hesitant to recommend your book to a true practitioner due to the assumed knowledge presented within the math section. I think a better treatment of mathematics would assume the reader has little to no background but is intelligent enough to learn ground up the specific use cases of the mathematics for the deep learning techniques presented in the book. See: http://www.deeplearningbook.org/ for better treatment of the math review. It seems more thorough and makes less assumptions about the math background of the reader.

I would love to recommend your book to a practitioner but I'm afraid the math section (the version I reviewed) would scare them off/they would get little out of it.