The aim of this talk is to present the peculiarities as well as pros and cons of different types of activation functions used in Neural Networks. During this talk we will go through both theory and practice. Problems from different domains (image recognition, working with text) will be discussed.
The aim of this talk is to present the peculiarities as well as pros and cons of different types of activation functions used in Neural Networks. Activation Functions that we will cover:
The next part of the talk will focus on experimental results. We will present how the training/test errors behave when using different activation functions. We will focus on measures like:
At the end of the presentation we will do an overview of available deep learning frameworks to show which activation functions are implemented in those packages.