Saturday October 30 1:00 PM – Saturday October 30 1:30 PM in Talks I

Computer vision and xAI: explaining a single prediction with visualisations and examples

Sara Tähtinen

Prior knowledge:
No previous knowledge expected


Explainable AI is a broad field of study that aims to explain the results of an AI system in an easy, human readable way. In this talk I will focus on methods that aim to explain one image prediction at the time. First I will discuss visualisations techniques and then using examples to explain the model's result. I will also show results of my own experiments to support the theory.


Assume you have a model that labels images. You show the model a picture of a cat but the model claims that it is a dog. The result is quite surprising and you are left to wonder: why did the model misclassify this image? What made it so confused that it thought that the cat is actually a dog?

This is just one example what could happen when complex AI models are used in practise. There are many cases where you must be sure the model works correctly but you are not totally sure if you trust the model enough. In these cases some kind of explanations would help us but as the models are complex, it is not easy to understand how the model made the decisions. To tackle this kind of dilemma, the field of explainable AI (shortened as xAI) was created: it aims to explain the results of an AI system in an easy, human readable way.

There’s a broad range of different xAI methods and I have given a general literature review of these methods in PyData Helsinki in 2021. In this talk I will focus on more specific area of xAI that aims to explain one image prediction at the time. These kind of explanations can be divided into three different categories: visualisations, example cases and alternative choices. I will be using deep learning models and first introduce different kinds of visualisation techniques, discuss their pros and cons and show some example explanations. Then I will focus on example cases that can be used to support the choice made by AI: with image embeddings (example demo from internet: one can find examples from a training set that the model thinks is similar to our query image. With this technique one can find similar examples of two alternative class predictions and hopefully understand better why the model came into the conclusion it did.