I'll present how to combine classical computer vision techniques with deep learning methods to build a solution to automate football video analysis. The talk will cover using deep convolutional neural networks for ball and player detection, application of multi-view stereo methods for 3D ball position recovery and using Spatial Transformer Networks to boost accuracy of jersey number recognition classifier. I'll discuss practical problems arising during long shot video analysis of football games. Problems, that make apparently simple tasks, like ball detection, really challenging.