Friday 14:35–16:10 in Auditorium

Twisted up in a Distributed Tornado - a beginners guide to async frameworks in python

Andrew Stretton

Audience level:
Intermediate

Description

By coming to this tutorial you will learn:

Abstract

At some point in a career in data science, software development or data engineering you will be looking to develop a piece of code that runs alongside another piece of code. For example, you might want to:

There are numerous ways of achieving such concurrency. If creating a project to run on one or a few machines which needs good exception handling then I find tools like Twisted, Tornado and Dask/Distributed ideal for the above tasks.

As a python developer I have always tended to work with Celery and felt intimidated by the different flow control when using async tools like Twisted and Tornado. In this tutorial I will walk through the examples above and talk about how I went from Async novice to Dask/Distributed contributor in 6 months. The tutorial will concentrate mainly on practical use cases. Along the way we will encounter some bumps in the road which will hopefully aid attendees' understanding of the basic dos and don'ts around these tools.

By coming to this tutorial you will learn: