While sophisticated machine learning approaches such as deep learning and tree ensembles often produce excellent predictive models, they also typically require performing computationally expensive searches over hyperparameter settings (e.g., the number, sizes, and types of hidden layers for a neural network). It is desirable to parallelize this search across multiple machines, but such functionality is not readily available in scikit-learn. However, scikit-learn does provide functionality to parallelize grid search across multiple processes on a single machine, using the joblib package. Also, joblib recently added functionality that allows users to implement custom parallel backends to support parallel computation on their own infrastructure. In this talk, I will describe the implementation of a custom joblib backend for an EC2-based auto-scaling computation engine. I will also describe applying this backend to the problem of grid searching deep learning text classifiers, using TensorFlow models that follow the scikit-learn estimator API.