Agenda: The talk will describe guidelines and best practices for profiling and tuning PyTorch models.

Target Audience: People using/developing PyTorch models.

Why is this talk important: Users are often unaware of how to profile and tune their model and miss out on important performance wins.

Format: Talk using slides. No hands-on coding. Slides will have code snippets and screenshots.

Outline:

About 15 min: PyTorch Profiling 101

• 2-3 min: Code snippets/screenshots for showing how to setup PyTorch profiler.

• 7-10 min: Walking through output from profiling a PyTorch model on GPU (prepared beforehand).

  • Analyzing execution time
  • Analyzing memory consumption
  • Analyzing stack traces
  • Analyzing data as a flamegraph
  • Analyzing long-running jobs

• About 10 min: Tuning PyTorch Models

  • About 7 min: Category I: Optimizations that are applicable to all models or are very easy to enable. Examples include: Async DataLoaders, Inference mode, Tensor creation etc
  • About 3 min: Category II: Optimizations that are either domain specific (eg not using bias when using batch norm) or require non-trivial work (using fused optimizers).

About 25 min for the talk and 5 min for Q/A in the end