Making a neural network say “I Don’t Know”: Bayesian NNs using Pyro and PyTorch

https://towardsdatascience.com/making-your-neural-network-say-i-dont-know-bayesian-nns-using-pyro-and-pytorch-b1c24e6ab8cd

by paraschopra • 8 years ago

126 46 8 years ago

towardsdatascience.com

sharemywin2• 8 years ago

I thought you were going adding a class called I don't know.

paraschopra7• 8 years ago

I tried doing that, but I didn't find any pydontknow module on pip.

Your comment reminded me of this xkcd comic https://xkcd.com/353/

svantana• 8 years ago

The nice thing about using likelihood as the training criterion, you don't even need an extra category, you can just combine the outputs with basic decision theory, like so:

  thres = wrong_cost / (wrong_cost+right_gain)
  response = (max(out_probs) > thres) ? argmax(out_probs) : "I don't know"

While the bayesian methods can theoretically perform better for out-of-set inputs, it feels like an oversight to not include this simple method as a baseline. My gut feeling is that this would perform equally well, if not better.

paraschopra7• 8 years ago

This works if you get nicely balanced probabilities out of neural network.

Softmax which is popular way to transform network outputs to probabilities pushes probabilities either to zero or one.

There may be other ways to transform outputs to probabilities where your approach could work but I'm not aware of them.

Also to reply to OP's comments, to have a class like 'I don't know' you need to have a dataset of things that don't look like your training dataset. And you'll always miss examples of things that don't belong to your input class. Bayesian solves it in a nice way by not requiring this infinite not-input training data.

sharemywin2• 8 years ago

found this online(might be useful):

Bayesian Approach: Pros and Cons

 	Pros

Can calculate explicit probabilities for hypotheses.

Can be used for statistically-based learning.

Can in some cases outperform other learning methods.

Prior knowledge can be (incrementally) combined with newer knowledge to better approximate perfect knowledge.

Can make probabilistic predictions.

 	Cons

Require initial knowledge of many probabilities.

Can become computationally intractable.

http://www.ru.is/faculty/thorisson/courses/v2008/gervigreind...

paraschopra7• 8 years ago

Require initial knowledge of many probabilities.

This is not a really big issue. You can always use uninformative priors, which is simply the Bayesian way of saying "I don't know" what the probability distribution looks like upfront. Then the data can help you infer the right shape of probabilities.

Can become computationally intractable.

This is a bummer, but modern methods like variational inference and better sampling techniques are making Bayesian analysis on large-scale data and complex models tractable.

I linked a video the article that I highly recommend watching. Variational Bayes and Beyond: Bayesian Inference for Big Data https://www.youtube.com/watch?v=DYRK0-_K2UU

nightcracker• 8 years ago

Require initial knowledge of many probabilities.

Can you make a two-stage design where the initial probabilities are generated by another machine learning model?

Fragoel2• 8 years ago

I have limited knowledge of machine learning so forgive me if I am wrong. I recall that some classification techniques provide you a confidence score of the classification... so can't you just treat a classification score lower than a certain threshold as "I don't know"?

warent• 8 years ago

This question was answered in the comments section of the article.

Basically Softmax will only give you values close to 1 or close to 0, so there's no room for your own interpretation or threshold.

https://medium.com/@paraschopra/softmax-amplifies-small-diff...

anonymousDan3• 8 years ago

Do you need to do softmax? Why not just look at the underlying amplitudes of the output before applying softmax? Or is it that they won't be normalized?

jeremysalwen• 8 years ago

I think that post is quite misleading. 90 and 100 are extreme values representing extreme confidence. Your model could just as easily learn [0.09, 0.10, 0.05] on the final layer, and the probabilities would then be [0.337, 0.340, 0.323]. And look, now softmax has the opposite problem! The final activation layers differed by a factor of 2, but softmax output almost identical probabilities!

Except this should not be surprising, because softmax is not scale invariant. Values on the unit scale is what you would normally expect, so if you feed it values 100x larger, it's going to be way more confident. If you feed it values 100x smaller, it's going to be way less confident.

conjectures4• 8 years ago

No. Probabilities can be random variables themselves.

Say p is the probability an image is of a cat.

p could be low, e.g. 0.05.

However, perhaps p has the 0.9 credible interval (0.01, 0.1). The width of that interval expresses how confident you are of the point estimate.

The correct translation of "I don't know" is a wide credible interval.

dwohnitmok3• 8 years ago

I disagree with this characterization, especially in a Bayesian context (which credible interval comes from).

Parameters into a model can be modeled as random variables. Probabilities themselves cannot be, unless they are themselves parameters (or happen to coincide with a parameter) into another part of a statistical model. A statistical estimation of an event (in the sense of probability theory) must bottom out at a single number.

Let's take a coin toss as an example. It does not make sense to say I think that the coin has a 0.05 chance of flipping heads with a 0.9 credible interval of (0.01, 0.1). It does not make sense even to have a probability distribution on coming up heads (strictly speaking).

Coming up heads is an event and events do not distributions associated with them, but rather single probabilities.

On the other hand, we can model the outcomes of a coin with a Bernoulli distribution with parameter p where the probability of heads is p and that of tails is 1 - p. Notice here that p is a parameter that just happens to coincide with our probability. It then makes sense to talk of p as a random variable and having a distribution, credible interval, etc.

Note, however, that our estimation of the event of the coin coming up heads is still a single number at the end of the day. Our intermediate calculations might just involve integrating over the space of intermediate values of p, which means that p no longer coincides with the probability of heads (indeed it doesn't really make sense because p is no longer a single number).

But how do we express the intuition of "confidence in my probability estimate?" After all it seems like there's a difference between me saying this coin has a 50% chance of heads when I know it's fair and me saying a 50% chance of heads when you have me an unfamiliar coin I've never inspected (even though reasonable statistical models for either case might both say it's 50%).

In Bayesian statistics this is encoded entirely by the function that updates your posterior probability. If you are very "confident" your posterior probability will remain similar to your prior probability no matter what evidence you see (I know for sure this coin is fair so even if it comes up tails 1000 times I still think the next time will be 50% chance of heads). If you are not very confident your posterior probability might change more dramatically (hmm... I saw a string of tails, I'm pretty sure the next coin flip probably won't be heads again and I'll modify my probability accordingly).

nafizh• 8 years ago

I have to disagree with the top reply. You definitely can interpret low probability as I don't know. Check out this blog post-

http://henripal.github.io/blog/langevin

And these papers-

1. https://arxiv.org/abs/1806.02855 2. http://arxiv.org/abs/1811.11145

jl2718• 8 years ago

In Christopher Bishop’s PRML book, he shows a procedure for getting the full posterior of a neural net using estimates of the Hessian evaluated at the point of inference.

Any idea how sampling compares in performance and computational efficiency?

program_whiz• 8 years ago

SVI still uses sampling under the hood. Any probabalistic programming framework (such as pyro) uses random sampling, there's no other way to estimate the curves of a complex set of interacting probability distributions. Obviously there are some optimizations for simple distributions that are known (e.g. we know what the expected values of simple gaussians will be, and the PDF or CDF of a given sample).

mlevental2• 8 years ago

how effective would this be in deep nets with small data sets?

edit: does anyone know of any papers that talk about this technique formally?

yuy910616• 8 years ago

interesting to see an answer too...usually bays is better with small data sets right?

paraschopra7• 8 years ago

Bayesian techniques are effective with small data sets if you build your model carefully. For example, if you are able to encode the generating process of your dataset, even a _single_ example could be used for effective training. For example, see this http://web.mit.edu/jgross/Public/lake_etal_cogsci2011.pdf

By providing a good generative model, you are shifting the effort from collecting more data to carefully coding your expert knowledge. In my view, this is an exciting paradigm as it allows combination of human knowledge (as generating process and prjors) that can be overcome (in posterior distributions) if enough data is accumulated against it.

rgrieselhuber• 8 years ago

Socrates was on to something.

jameszhao00• 8 years ago

How does this compare to Dropout based methods? http://www.cs.ox.ac.uk/people/yarin.gal/website/blog_3d801aa...

sjg007• 8 years ago

My guess is you use dropout during training.

mlevental2• 8 years ago

sorry for posting twice in this thread but i figure it's a good place to ask: what other methods are there for getting a nn to report "i don't know"

JD557• 8 years ago

My Master thesis supervisor had a fun trick for this, that can be used not only on on NNs, but also on other classifiers: https://arxiv.org/pdf/1011.3177.pdf

The gist is, you train a single classifier that will work as two classifiers (with certain restrictions): one that says "false/not-false" and another that says "true/not-true".

If the output of the classifier is "not-false" and "not-true", then you consider that as a "I don't know".

yboris• 8 years ago

Is this what you mean: For example, in MNIST, your model would output 20 probabilities: (1-yes: 0.8, 1-no: 0.2, 2-yes: 0.1, 2-no: 0.9, etc) where each pair sums to 1?

zenpaul• 8 years ago

Or train the network with unknown or not-a-number data or pseudo data. https://patents.google.com/patent/US5359699?oq=5%2C359%2C699

chrisfosterelli2• 8 years ago

Uh, the lead of the article claims it achieves "97% accuracy on MNIST". If you are removing 12% of the MNIST images at your neural network's discretion, you can't claim the 96% accuracy on the remaining set as your overall accuracy! You're essentially self-selecting for test data the classifier is confident in, so of course that accuracy will be high.

If this was allowed then I'd make an ImageNet classifier which just discards all images except for one I'm really good at classifying, then claim I have 100% accuracy on ImageNet.

paraschopra7• 8 years ago

My main point is not about aggregate accuracy, it's about comparing confidence across classes of datasets.

Edit: the entire point of bayesian approach is that you can make decisions on a loss function where you can make a tradeoff on the (business) cost of making a wrong decision and (business) regret of not making a decision. For real world applications, accuracy is a useless metric. What matters is the net (business) benefit or loss.

chrisfosterelli2• 8 years ago

Your article bolds the following in the introductory paragraph:

My final classifier had accuracy of ~97% on MNIST and it refused to classify white noise and the majority of unrelated (non-MNIST) images.

I'm saying that you are making a misleading claim when you say this. It is categorically untrue that you achieve 97% on MNIST with this approach. Accuracy on a standardized test set makes no sense when you select a subset from the test set for maximal accuracy. I'm not saying that your approach doesn't have value or make sense in some business cases, because I think the overall idea is interesting but these statements are not correct.

dmurray• 8 years ago

Yes, this is buried deeply in the article, but the classifier rejected 12% of the handwritten digits - and did not even have good accuracy when forced to guess on those 12%.

tdgunes• 8 years ago

For people who are interested, there is an upcoming NIPS paper that might be relevant for this problem:

Paper: http://arxiv.org/abs/1806.01768

Demonstration: https://muratsensoy.github.io/uncertainty.html

mmm1111• 8 years ago

the author of this article would do well to admit that he "doesn't know" bayesian inference very well. unfortunately, the usage of pyro is incorrect here. in particular data subsampling is handled incorrectly (there may be more issues). the weights of the network represent a global random variable. consequently, when doing data-subsampling care must be taken that the ELBO that is constructed is scaled correctly. that is not done here. probabilistic programming languages like pyro are a great tool for making bayesian inference easier and more accessible, but they can be produce arbitrarily bad results if the user uses them incorrectly.

jlmaccal• 8 years ago

Could you elaborate on how to do this correctly? This is one of the only examples of using pyro to build a Bayesian NN and it would be really valuable have a correct implementation.

Making a neural network say “I Don’t Know”: Bayesian NNs using Pyro and PyTorch

Related Stories