Deep learning grows in popularity and use, but they have two problems:

• Neural networks are highly overparametrized. Neural networks have millions of parameters. This requires large memory, which is detrimental on mobile applications. Also the many parameters make neural networks slow in production. So pruning the parameters will save us memory and computation time.
• Neural networks provide no uncertainty. Current neural networks show only a prediction ("is this a cat or a dog?"), but have no notion of uncertainty. This is detrimental in (medical) diagnoses, quality control or any other critical application. Being Bayesian about our models gives us uncertainty over a prediction. The uncertainty provides much better information on whether we should rely on a prediction.

We discuss experiments on the two popular applications of deep learning: images and time series. For image classification, we train a Bayesian neural network on the CIFAR10 data set. For time series, we train on the 5000 ECG's from UCR. In both experiment, we achieve the same performance as baseline with less than 10% of the weights.

This talk will be practical, so you can get started with Bayesian deep learning yourselves. All code is open source and implemented in Tensorflow. The code consists of simple helper functions that you can add to your own deep learning code.

Slides are here