A tour of recent tool developments for understanding and optimising the performance of Python code, covering profiling tools, Numba, and CFFI. The talk will include demonstrations of these tools and their features using motivating examples.
Scaling up the performance of Python applications without having to resort to writing the performance-critical sections in native code can be challenging. This process of performance optimisation has three parts: determining which parts of an application are performance critical; understanding their performance; and acting on that understanding to implement optimisations. This talk is about the latest developments and tools for all these aspects of High Performance Python.
The Python profiler and Kernprof provide some explanation of the performance of Python programs; this talk will demonstrate the Accelerate Profiler, which is Numpy-aware and builds on the standard profiling tools by capturing the flow of arrays throughout program execution. Interactive experimentation and exploration of captured profile information is done in an IPython notebook, simplifying the performance optimisation workflow.
The second part of this talk is about Numba, a tool that compiles user-selected Python functions to native code at runtime, for performance boosts of up to 100x for particular samples of code. Numba is in continuous development, with many additions made over the past year, making it applicable for optimising an ever-increasing selection of performance-critical code. We will introduce and demonstrate applications of recent additions including JIT classes, parallel Multicore/CUDA ufuncs and gufuncs, the @generate_jit decorator (similar to generated functions in Julia) and support for CFFI. The combination of Numba and CFFI is particularly powerful, enabling seamless integration of optimised native code into High Performance Python code.