Causal Inference, AKA how effective is your new product, policy or feature? Inspired by A\B testing in tech, organizations have turned to randomized testing. However, randomization often fails, leaving us in a biased reality. Join us on our quest to dispel myths about randomized testing and build practical models for effect measurement in business situations, in this Eneco-Heineken joint talk.
In this talk we take you through various approaches to causal inference in an applied business context, as well as highlight a number of statistical pitfalls that people often overlook. We'll start by critically evaluating the way organizations often conduct randomized tests, by taking their cue from success stories in A\B testing or medicine's randomized control trials.
By taking you through two real-life use cases (heat pump savings measurement at Eneco and asset allocation at Heineken), we search for practical econometric and machine learning approaches to measure effect sizes when test-control group randomization is infeasible. These techniques build from the classical literature on linear treatment effect estimation, and extend it with excursions into Bayesian estimation for handling uncertainty and domain expertise and applying tree ensemble methods for estimating individualized treatment effects.