HN user
I didn't notice any obvious visual differences, but I'm also not an expert on adversarial examples. The transformer models were similarly susceptible to attacks, but while adversarial examples transferred well within a model class (~40%), they did not across model classes (~5%). These are rough numbers from memory, don't hold me accountable!
Author here. You're absolutely right that "understanding" is a fuzzy word. As you pointed out, part of reason we hold this belief is that the model can generate diverse samples and successfully complete out-of-distribution inputs. But the other part is that the model learns (without labels) useful features for classifying objects. Would be very interesting to test it on a broader set of datasets which measure other 2D image characteristics.
Author here. I ran some early experiments a while ago, and it looked like adversarial examples for convnet classifiers didn't transfer to transformer classifiers and vice versa. Definitely worth looking more into!
I don't think this is true. The ResNet was born at Microsoft, DQN was born at Deepmind, the Transformer was born at Google, and GPT2 was born at OpenAI.
I'm obviously biased since I work at an industry AI lab, but we both have important roles to play.
These analogies need to be taken with a grain of salt. For instance, king - man + woman actually returns king (since woman - man ~= 0). It's only queen when you ban the words king, man, or woman (since the embeddings of queen and king are close).
See Ian's tweet https://twitter.com/goodfellow_ian/status/113352818965167718... or the linked paper https://arxiv.org/pdf/1905.09866.pdf for more information.
The article discusses effects on doctoral students, not masters or undergrads. STEM PhD programs at top schools are a lot less pay-to-play.
Are you serious? This class is intended for students with deep knowledge of CS theory who want to learn more about recent data structures research. 6.006 and 6.046 are much more suitable if you aren't formally trained in computer science.
I took this class in 2010. Most assignments consisted of proving one or two theorems from a recent research paper.