Introduction to Autoencoders

https://www.pinecone.io/learn/autoencoders/

by gk1 • 3 years ago

104 11 3 years ago

p1esk2• 3 years ago

Basic autoencoders are conceptually easy, but try to understand how the VAE KL loss was derived. I remember I spent two days reading https://arxiv.org/abs/1906.02691 and I still didn't quite get it.

SnooSux• 3 years ago

Explaining Autoencoders seems to be a rite of passage for ML bloggers. It's conceptually understandable but has broad applications and is a jumping off point into other architectures.

Educational content tends to have a bias towards concepts that are easier to explain rather than what is most useful. But as far as the latter goes, Autoencoders are a good place to start.

uoaei• 3 years ago

Autoencoders are insightful because they move the concept of "intermediate representations" to the foreground. This perspective is useful for understanding the compositional architectures of NNs at any scale.

NNs can be generally described as "complicated feature engineering stacks followed by a relatively simple regression". This description captures the information processing and compression perspectives on NNs, which is realized by autoencoders in a laboratory-ideal way.

theGnuMe• 3 years ago

Can you explain it now? I'll give it a shot now. The minimal KL divergence should be equivalent to the maximum likelihood just another way to express/compute it.

cgs10192• 3 years ago

To grok the ELBO, don't start with VAEs; start with basic variational inference. https://arxiv.org/abs/1601.00670 is good, though maybe increasingly dated...Once you're happy with VI in general, https://arxiv.org/abs/1606.05908 is a nice read on the VAE.

p1esk2• 3 years ago

Thank you! Do you have any recommendations for educational resources at an even more basic level of understanding of variational inference? The first paper you mentioned is "for statisticians", and it seems pretty dense to me on the first glance. My current level is somewhere at the first undergrad pstat course.

liminal• 3 years ago

The author lifted entire sentences without modification from his sources. E.g. "The parameterized distribution in the code of the autoencoder can be randomly sampled to generate discrete values for latent vectors, which can then be forwarded to the decoder, leading to the generation of new data." I thought this seemed unclear given the previous context, so I looked at the source article he linked to [1] and found that exact sentence. This kind of plagiarism is not cool. If you can't explain the concept, don't publish the article.

[1] https://blog.roboflow.com/what-is-an-autoencoder-computer-vi...

gk12• 3 years ago

Hi, I'm responsible for our marketing which includes educational content like this. I'm looking into this urgently. Thank you for flagging it.

gk12• 3 years ago

Update: After a closer look we determined this article is not original and decided to take it down. We're also making some internal changes to prevent this from happening again. Thanks again for flagging this!

leobg• 3 years ago

I’m not surprised. Isn’t the whole concept of Pinecone to spam HN with valuable looking content so you can bait and switch into “our generous free tier” (so it doesn’t matter in the end that you understood nothing, because you can just abstract it away)? I’m not sure what I dislike more. This kind of “content marketing” on HN, or the fact that it apparently works.

Introduction to Autoencoders

Related Stories