msaroufim

I'm having trouble understanding why the success of pooling would be deemed unfortunate. Max-pooling or average-pooling are essentially learning something about a pair of contiguous features: while max-pooling saves only the most prominent/largest value, average pooling compresses information using the mean. Saying this is unfortunate from Hinton's standpoint amounts to saying that it is very unlikely to see any sort of pooling behavior at the level of neuronal populations. At a physiological and psychological level, what would pooling equate to?

I believe Nadella was referring to a more general sentiment in the machine learning community about our lack of theoretical understanding of deep neural nets. I'd suspect that the network still learns some higher level features that exist across languages which is why it could become better in English after learning German.