Thursday 4:20 PM–5:00 PM in Radio City (#6604)

Run Numba functions in SQLite: WTF?

Phillip Cloud

Audience level:
Intermediate

Description

This talk focuses on an extremely experimental library called slumba that attemps to bring the convenience and speed of numba to user-defined scalar, aggregate and window functions in a SQLite database.

Abstract

Numba provides Python APIs for creating JIT-compiled functions and classes. SQLite is an in memory database, support for which ships in the Python standard library.

This talk will first survey the available methods for adding user-defined scalar and aggregate functions in the existing Python standard library APIs. After that, a discussion of what needs to be done to add a user defined function using the SQLite C API. Next up we give an extremely brief introduction to numba and the features it provides that we use to implement slumba. An aside into miniast (the library used to generate code) is taken. We then discuss the full set of steps that happens when you use slumba and how you register a function with a running SQLite database.

Finally, we wrap up with a live demo of defining and running a window function using slumba. Time permitting I will show microbenchmarks as well.

Subscribe to Receive PyData Updates

Subscribe