Notebooks have traditionally been a tool for drafting code and avoiding repeated expensive computations while exploring solutions. However, with new tools like nteract's papermill and scrapbook libraries, this technology has been expanded to make a reusable and parameterizable template for execution. We'll look at how to make use of this pattern for Data and ETL processes.
We'll some visual examples and breakdowns of notebooks.
A guide through how a notebook executes and the model it uses to run your code.
Around experimentation and code development.
For production data and operations without full rewrites of Notebook code.
papermill is a library for executing notebooks programmatically.
You'll see some examples in Python and with it's provided CLI.
We'll relate the execution back into original Notebook execution diagrams.
Quick pointer to the extensibility of the library and how to add new functionality.
Failure analysis, Productionalization, Sharing executions...
Making a pipeline with Notebooks.
Good practices Where unittesting doesn't fit
Quick blip about adoption and usage at Netflix.