Thursday 17:10–17:40 in Track 1

Debugging machine learning. Mostly for profit, but with a bit of fun too.

Michał Łopuszyński

Audience level:


Debugging machine learning apps is hard. I feel that this topic is important, however, relatively rarely touched compared to, e.g., latest models or new interesting ML applications. In this talk, I will try try to fill that gap by discussing best practices, recommendations and my own experience on the subject.


Recently, OpenAI examined ten popular reinforcement learning algorithms reimplementation and found that six (!) of them contained significant non-breaking bugs [1]. This illustrates the fact that debugging and testing machine learning software is difficult and requires extra attention.

In particular, machine learning (ML) driven apps exhibit rich spectrum of failure types, specific to the domain. ML systems can be diagnosed:

In this talk, I will touch on each of the above scenario, describing possible pitfalls and recommended engineering practices, as well as providing real-life examples.


Subscribe to Receive PyData Updates


Get Now