Ask HN: As a professional, does it worth it to learn the math behind NN?

https://news.ycombinator.com/item?id=28843175

6 17 5 years ago

news.ycombinator.com

In 2021, as a (junior) professional working directly with ML/AI, does it worth it to learn what's going on in terms of mathematics behind Neural Networks? Or should we just focus on the pratical side of things, like daily applications using frameworks/libraries and metrics/performance?

PaulHoule3• 5 years ago

How much do you know already?

IWantToRelocate7• 5 years ago

Well, not much. Data isn't the main product where I work for, so I end up just using python libraries for some mundane applications and small datasets (image classification and recommender systems). But I'm the "need to know from scratch" kinda person and I wonder if I should waste my time learning the math behind it or if its useless

PaulHoule3• 5 years ago

What's the first derivative of

   x * cos(x)

?

andyxor• 5 years ago

there is not much math, don't let the academic papers razzle dazzle you, they make things look complicated to get their grants

in practice, as long as you've studied basic calculus and understand how to find a minimum of a function via derivative you're good, there is your "gradient descent" in a nutshell: https://www.mathsisfun.com/calculus/maxima-minima.html

everything else is plug-and-play from existing libraries

you can ask any "data scientist" or "ML engineer" what they do all day, it's a whole lot of copy paste, and tweaking the data and parameters through trial and error until it fits

Edit: Ok , it would also help to understand dimensionality reduction via PCA/SVD at least once, it's available in any linear algebra book: https://en.wikipedia.org/wiki/Singular_value_decomposition , https://en.wikipedia.org/wiki/Principal_component_analysis that's probably the best and most "scientific" part of ML

IWantToRelocate7• 5 years ago

Thank you! appreciate your reply, honestly. i have an engineering degree but it has nothing to do with computers, so i kinda struggle sometimes. I basically want to get fluent with the metrics/performance when applying ML/NN, when to tweak what, how to improve some stuff, what algorithm works best for some of the problems, etc.

aborsy2• 5 years ago

This, in a nutshell.

mindcrime• 5 years ago

My personal feeling is that it's always "worth it" to learn more maths. Not saying it's absolutely required, and I'm a big fan of (generally speaking) prioritizing the pragmatic / practical / hands-on over the purely theoretical. But, even within that belief system, I would say that - all things being equal - any opportunity to learn more maths is worthwhile, at least up to a certain point. Figuring out the details of the "certain point" is left as an exercise for the reader though.

aborsy2• 5 years ago

Frankly, I rarely see this complicated math in academic papers to be useful in practice.

Most of the stuff that is valuable in practice does not require much math.

Many people believe math actually hindered the development of AI.

wnkrshm• 5 years ago

The papers are interesting to people making tools - if you can see what mathematical assumptions are made you can see what limitations a method has and whether it can apply to cases you're interested in.

jstx1• 5 years ago

Since you're working in the field - yes, obviously. It's not even that much math - the building blocks are fairly straightforward linear algebra and calculus.

Here's a nice, fairly detailed, summary - https://explained.ai/matrix-calculus/. But if you haven't taken undergraduate level single/multivariable calculus and linear algebra, I would take them from MIT Opencourseware even if it's just to fight off the impostor syndrome a little bit.

IWantToRelocate7• 5 years ago

thank you! on a quick look, that seems straight to the point, for someone who had calculus and linear algebra in 2006.

IWantToRelocate7• 5 years ago

a printable version of your link, in case someone want it as well: https://arxiv.org/pdf/1802.01528.pdf

ozzythecat• 5 years ago

I recommend learning multi variable calculus and linear algebra. You can take courses online for free. Personally, I didn’t really understand what was going on until I refreshed my math.

It won’t make you an ML expert, but it’s like the difference between knowing how to write a sort function vs. only knowing how to call a sort function. Ideally, you understand both. Not saying you need to understand how every sort function works internally and write a proof for it… but a basic understanding is helpful.

IWantToRelocate7• 5 years ago

thats a great advice and makes total sense. thank you!

Ask HN: As a professional, does it worth it to learn the math behind NN?

Related Stories