Saturday 13:30–14:15 in Big Room

Diagnosing Machine Learning Models

Lucas Javier Bernardi

Audience level:
Intermediate

Description

A Machine Learning Models is never perfect. If it completely fails, it must be fixed, if it performs well, we want to improve it.
In this talk, several techniques to diagnose the source of errors a model makes will be presented.

Abstract

Your model performs worse than a random model. What do you do? Your model has 99.9% ROC AUC, should you just celebrate? Every time you add a new feature the interpretation of the model's parameters changes compeltely. What 's wrong? Your model has 75% ROC AUC. Should you add more data? More features? Use a more complex model? You are out of new features, no matter what you do, the model performace is the same? What is happening?

Many of these questions apepar once and again when working with Machine Learning,, answering them takes time and has a huge impact on the final outcome of a Machine Learning project. Understanding the current condition of a model is the key to decide what to do next.

In this talk I will describe several techniques to diagnose alogrithms and models, some of them are: Bias and Variance Decomposition Calibration Curves Respons Distribution Chart Residual Plots etc.

Subscribe to Receive PyData Updates

Subscribe

Tickets

Get Now