Iteratively building a classifier requires a mix of skill, diagnostic ability and guesswork. I'll lay out a framework that helps you build reliable classifiers with greater confidence and less random guesswork. We'll review different ways to tackle a classification challenge and visual diagnostics that help identify sources of error and missing but exploitable information. Tools demonstrated will include sklearn, YellowBrick, Shapley and pandas_profiling. The approaches we'll discuss will apply equally to regression challenges. Whilst this talk is aimed at Intermediate Data Scientists, people at the start of their career will benefit by having a clear process and by being introduced to new tools.