Sunday 10:45 AM–11:30 AM in Room #370B/C (3rd Floor)

Making your code faster: Cython and parallel processing in the Jupyter Notebook

Gustavo A. Patino

Audience level:
Intermediate

Description

As the complexity and scope of applications grow, it is very common to run into slow performance issues. In Python, it is possible to improve the speed of execution with the use of parallel processing and the Cython compiler. The Jupyter Notebook makes the implementation of both of them a relatively simple task, which will be the focus of this session.

Abstract

To improve the speed performance of an application, a requirement that becomes critical as the size of computations and/or databases reach sizes not uncommon nowadays, both code design and programming language issues can be addressed. In Python, scripts are usually dynamically typed and perform one computation at a time, representing two aspects that can be optimized to improve performance. Static typing can be achieved with Cython, a static compiler from Python to C. Parallel processing can be used to take advantage of all the cores present in a machine or a cluster, through the ipyparallel module. While both Cython and ipyparallel can be used in any Python environment, the Jupyter Notebook makes their implementation very simple and is the ideal environment in which to learn the basics about both of these tools. The session will cover the following topics: - Installing the ipyparallel module - Using the Client class to assign tasks to individual cores - Synchronous and asynchronous execution of tasks - Use of the %px magic command - Statically defining variables and functions using Cython - Making modules available to Cython through the use of cimport - Numba as an alternative to Cython