When doing experimentation at Uber, we sometimes have to expose an entire market to treatment, because network effects between riders and drivers make an A/B test infeasible. In these cases we can construct a synthetic control out of a series of potential control cities to still do causal inference. We discuss the theory and implementation of this approach using a Python package developed at Uber.
When doing experimentation at Uber, we sometimes have to expose an entire market to treatment, because network effects between riders and drivers or legal/regulatory concerns make an A/B test infeasible. In these cases we can construct a synthetic control out of a series of potential control cities to still do causal inference, using a Python package developed at Uber. In our presentation, we discuss the motivation and use cases for this approach in our marketplace and product teams, the theory behind this approach, its implementation in Python at a high level, and a comparison of this method with other alternatives when an A/B test is not possible, such as diff-in-diff or Google's CausalImpact - synthetic control using Bayesian Structural Time Series. Many organizations face similar causal inference challenges, for example, in estimating the effect of media campaigns that have to be rolled out at the city or regional level. A correct application of synthetic control can allow for rigorous decisions based on testing, even in these situations.