A/B testing is at the core of analytics. We will cover the motivation behind A/B testing, A/B testing at Zalando, pitfalls and learnings, concepts of analysis methods (t-test in particular), Zalando’s open-source Python library: ExpAn, and hot trends of research topics in A/B testing.
In the field of analytics, A/B testing is nowadays of the essence. A/B testing is especially crucial to optimize business processes and user experience -- Microsoft, Airbnb, Twitter, etc are building up huge teams for their experimentation platforms.
In the talk, we will show our experiences of A/B testing at Zalando -- a German e-commerce company. Being the advantage of a large site, we’ve already run hundreds of tests, and we will share our learnings with you.
Moreover, we will introduce our open-source Python library ExpAn. It is developed for the statistical analysis of A/B testing and defines standard data structures for statistical experiments. You can import it easily into your project and analyze your own A/B tests seamlessly. In this talk, we will give a live coding session of using ExpAn.
Depending on the time, we might also briefly mention some hot trends of research topics in A/B testing.