From the perspective of a PyMC and TFP developer, I will give a developer introduction of PyMC3, and talk about some of our current design pitfall and future direction(s). I hope that with these insights and consideration of how a modern (i.e., depending on some autograd system) probabilistic libraries is designed, it could help user and practitioner to write better probabilistic programs.
With modern automatic differentiation libraries like Tensorflow, Jax, autograd, Pytorch, Theano, and more (insert your favorite autograd library here), writing a Bayesian library (or aspiringly, a Probabilistic programming language) seems could not be easier. So, what are the challenges? In this talk, I will speak about designing a Bayesian computation library using PyMC3 as an example, and share some stories about our (now) two iteration of designing PyMC4, with some anecdotes on comparing different Bayesian libraries, choosing a new computational backend, TF1 to TF2 transition and graph modification. In a way, this talk is NOT a tutorial of how to design a Bayesian library, but the opposite: I will try to convince you not to write one, unless you want to deal with 3 types of shape issues, find 10 alternative ways to rewriting control flows, and learn a lot of tricks to handle edge cases that could quickly goes obscure.