INTRODUCTION
This tutorial is intended for readers who are interested in applying Bayesian methods to machine learning. Our intention is to teach you how to train your first Bayesian neural network, and provide a Bayesian companion to the well known getting started example in TensorFlow.
So why do we need Bayesian neural networks? Traditionally neural networks are trained to produce a point estimate of some variable of interest. For example, we might train a neural network to produce a prediction of a stock price at a future point in time using historical data. The limitation of a single point estimate is that it does not provide us with any measure of the uncertainty in this prediction. If the network predicts that the stock will increase in value with 95% confidence then we probably have an easy decision to buy, but what if the network has only a 50% confidence? With point estimates we just don't know how uncertain we are. By contrast, Bayesian neural networks enable us to estimate the uncertainty in our predictions using Bayes' rule.