As machine learning becomes more prevalent, ever growing parts of the systems we build are changing from the deterministic to the probabilistic. The accuracy of machine learning applications can quickly deteriorate in the wild without strategies for testing models, instrumenting their behaviour and the ability to introspect and debug incorrect predictions.

This session will take an applied view from my experience of building production machine learning infrastructure at Ravelin. You’ll learn useful practices and tips to help ensure your machine learning systems are robust. We’ll go into:

• Labels and Data - can you trust it? Can you infer them?
• Testing - how do you ensure your model is doing the basics, up to the more complicated examples?
• Auditing and versioning - what's the provenance of your model? What data was it trained on? With which hyper parameters? Can you reproduce it?
• Debugging and introspection when deployed - when you make an awful prediction, can you figure out why that happened and prevent it happening again?

And more, with the aim of helping you sleep a little better at night knowing your model is out there in the wild.