In this tutorial we will be learning what Recurrent Neural Networks are and what they are used for. We'll focus our attention on a special and powerful type of RNN called Long Short-Term Memory (LSTM) networks. As part of this tutorial we'll also implement our own LSTM Net to have fun training it to autonomously produce text-based content or even compose music [almost] like a real human! cool, eh?
Long Short-Term Memory (LSTM) Networks are a special and very powerful kind of Recurrent Neural Network (RNN). LSTMs are widely used and they work amazingly well on a large variety of problems, ranging from Sentiment Analysis to Machine Translation, Time Series Prediction, among others. In this tutorial we will study a little bit what's under the LSTM's hood to see how they work from the inside and why they usually work much better than vanilla RNNs. Also we'll be implementing our own LSTM Network using TensorFlow, then we're going to train it so that it can learn to autonomosly compose music and generate text of different kinds.
The class will be divided into the following three segments:
(download the notebooks and other stuff from HERE)
The talk will begin by introducing the concepts behind RNNs, paying special attention on LSTMs. We'll deeply analyze how they work and study some of the math involved on its inner mechanisms.
In this section we'll make a stop along the way to cover the basics of TensorFlow, introducing the concepts like tensors and its properties, computation graph and sessions. This section is intended to help people unfamiliar with TensorFlow to understand the last segment.
Finally we'll implement a LSTM using TensorFlow. In this stage, will train our model to be able to autonomously compose music and generate different types of text-based content. The last 15 minutes of this segment we'll be allocated for attendees to ask questions or just play with the LSTM, for instance, making it produce different types of data.