Tuesday 5:05 p.m.–5:45 p.m.

Parallel Programming in Python: Speeding up your analysis

Manojit Nandi

Audience level:
Novice

Description

I'll show you how to utilize multi-core, high-memory machines to dramatically accelerate your computations in Python. You'll learn how to determine whether your tasks can be parallelized, general purpose techniques in Python for parallel programming, and specific applications of parallel programming in a machine learning context.

Abstract

  • Motivation: lots of “medium data” use cases that can fit in memory on one large machine
  • Basic conceptual intro to parallelism, general principles and pitfalls
  • Machine learning applications
    • Random Forests, Grid Search, Cross validation
  • Python examples
    • Joblib
    • Clusters in IPython Notebooks
    • Scikit-learn n_jobs (grid search, cross validation, random forest, kmeans)
  • Spawning multiple experiments across machines

Sponsors

Become a sponsor.