Friday October 29 6:30 PM – Friday October 29 7:00 PM in Talks I

Serving and Managing Reproducible Conda Environments via Conda-Store

Chris Ostrouchov

Prior knowledge:
No previous knowledge expected

Summary

Conda-Store is an open source tool being developed by Quansight to help manage, build, and serve Conda environments. It enables IT to provide controls over environments, users the flexibility of quickly updating environments, all while serving these environments in a reproducible manner in a pinned YAML, Conda lockfile, Conda-Pack tarball, and Docker images.

Description

End users think in terms of environments not packages. The core philosophy of conda-store is to serve reproducible conda environments in as many ways as possible to users. Conda-store was developed due to a significant need we found in enterprise and hpc architectures. There are many ways to serve environments and each plays an important role. Thus conda-store serves the same environment via a filesystem, lockfile, pinned yaml specification, conda pack archive, and docker image. This logic could easily be extended to also support the creation of VM iso's and singularity containers for HPC environments.

During this talk I will highlight some common problems with environments we have seen while consulting and show how conda-store aims to solve them: - Friction between system administrators and end users in controlled environments where new packages are needed - Enabling a given notebook developed within jupyterlab to be reproducibly run in workflows reliably for years to come - Helping to removing the need for specially crafted docker and singularity containers

This talk will be full of demos along with a site that everyone in the talk can try out. Additionally I will showcase initial work on a jupyterlab extension gator that allows for conda environment management.