underaxon

I may be wrong but I think this is what kernel methods (eg. SVM) do, right? So this looks like a (deep)SVM where the kernels are small NNs.

I've made some experiments that you can check out here:

http://www.clungu.com/Distilling_a_Random_Forest_with_a_sing...

The models aren't downloading correctly. The content of the '*.pt' files says 'Google Drive - Quota exceeded'. I guess too many people have tried downloading the files from your drive.

One solution is to download (and upload to Colab) the models manually in /content/checkpoint/