Understanding Neural Networks Through Deep Visualization

http://yosinski.com/deepvis

by antimora • 11 years ago

139 27 11 years ago

yosinski.com

bpg_92• 11 years ago

I love the fact this is using caffe by Berkeley. Good stuff indeed.

mandor• 11 years ago

A nice video tour of this "deep vizualisation toolbox": https://www.youtube.com/watch?v=AgkfIQ4IGaM&feature=youtu.be

Sven7• 11 years ago

Very cool! This reminds me of galaxyzoo.org

So I guess future corps, will consist of cube farms of employees staring at such screens, training/optimizing proprietary nets.

dnautics2• 11 years ago

Wonder what happens if you use a regularization constraints in fourier space, optimizing the 2DFT of the image, with restraint increasing with frequency...

Houshalter2• 11 years ago

Why would the 2DFT be a good model of natural images?

thanatropism• 11 years ago

Speculatively (pun intended): we see in saccades (rapid eye movements) rather than static analysis of scenery, so we're bound to understand the frequency of, say, edges over the fixed-period saccade rhythm better than we understand distances over a plain 2D-field representation. This is why we're often surprised by perspective in photography (well, also because of stereo vision): as we move our eyes the relative position of lines at different distances moves imperceptibly so we get a clue of 3D space.

When I was younger I took some drawing lessons because I hoped to be an architect, and the first thing we learned was precisely to undo this instinct and see the world as a flat thing -- this is why artists are seen stereotypically as extending their arm and looking at their brush with one eye -- they're using it to measure the distance of points in their visual field as a static field, as contrasted to the dynamic field that can't be put on paper.

dnautics2• 11 years ago

Reducing the amplitude of high frequencies is a way of smoothening the images.

mjw• 11 years ago

When you penalise the L2 norm of the convolution of the image with a filter (like a gradient or edge detector for example) you are effectively doing this. The spectrum of the filter determines how much different frequency components are penalised.

See https://en.wikipedia.org/wiki/Tikhonov_regularization

https://en.wikipedia.org/wiki/Regularization_by_spectral_fil...

I think (although they're a little handwavey about it) that their "Gaussian blur" prior must be of this form. They certainly talk about it penalising high frequency components.

The total variation method they mention is a generalisation of this too.

primus202• 11 years ago

Ok now I want to "hear" one of these trippy neural network things using a speech/language recognition neural network! Or perhaps if there's a music based one? I'd love to hear a computer's auditory hallucinations!!!

Houshalter2• 11 years ago

This isn't quite what you are looking for, but here is one trained on midi files extending let it go: https://ericye16.com/music-rnn/

Here is one trained to predict the next byte in an wav file: https://www.youtube.com/watch?v=eusCZThnQ-U

userbinator• 11 years ago

That YouTube one is a good approximation to what I hear if I leave the radio turned on in the background and playing music, but am not paying attention to it and focusing on other stuff.

shahar2k2• 11 years ago

So if it were possible to read a single neuron in the brain, and optimize input through its' signal would it be similarly possible to extract what triggers that neuron from such a setup?

cafebeen3• 11 years ago

Yes, there's a technique called single-unit recording:

https://en.wikipedia.org/wiki/Single-unit_recording

I think it's pretty rare to see an very selective response though, unless you're looking close to sensory neurons.

nickledave2• 11 years ago

There's definitely selective responses in single unit recordings, even upstream of sensory neurons in "higher" areas; look for articles on the "Jennifer Aniston cell" and on so-called mirror neurons. The question is, how come you can fool a DNN into classifying noise as something with high probability (as they explain in the article), but you can't fool the brain?

chestervonwinch• 11 years ago

http://www.nature.com/nature/journal/v435/n7045/abs/nature03...

edit: woops. I mean to respond to GP.

shahar2k2• 11 years ago

can this be done in a non-invasive way?

FrankenPC• 11 years ago

I wonder if there's a sort of JTAG port into the neural network to do direct scans of the individual neurons.

asanagi• 11 years ago

It's like you're seeing into a machine's imagination. Look at the 8th-layer images of the pitcher, or the gorillas with improved prior, for instance. They're very close to layout sketches an artist might use to block out a painting or photograph before beginning the work.

Unreal. Strong AI is not as far off as we think. 15 years. Maybe 20.

solve• 11 years ago

Looking at those generated images, I realized that many of those training photos seem to apply:

https://en.wikipedia.org/wiki/Rule_of_thirds

joelthelion• 11 years ago

Is this implemented in any of the deep learning libraries? It looks very useful for debugging, but it also looks like a lot of work to implement, so it would make sense to have it directly integrated in the libraries.

Understanding Neural Networks Through Deep Visualization

Related Stories