During the session I will explain the notion of transfer learning both in context of single task learning and recently very popular multitask models. I will give a brief overview of the state of the art approaches to machine translation (zero shot translation) and image recognition with a focus on transfer learning.

Next I would like to deep dive into the topic of Deep Convolutional Neural Networks (ConvNet) to visualize the information extracted by the network and how it differs depending on how deep into the network we are. After that I would present two approaches on how we could benefit from transfer learning when solving image recognition tasks:

· ConvNet as feature extractor – this approach uses already trained ConvNet and simply changes to top feed-foreward fully connected layers

· ConvNet as start for further learning – this approach also tries to fine tune the weights in the convolutional part of the Neural Net

During the session I would also like to mention the very important idea of Common Representation Learning which is critical when it comes to applying transfer learning to problems that are similar in the idea but use different representation space. Languages are a good example of this, although we say the same thing the input vocabulary sets are usually disjoint. The approaches that I would cover in the session are:

· Canonical Correlation Analysis – an approach that focuses on minimizing the similarity of two different inputs when transformed to common representation

· Autoencoder – an approach that focuses on minimizing the reconstruction error

· A combination of both – a hybrid approach joining advantages of the previous two