Saturday 15:00–15:45 in Audimax

A/B testing at Zalando: concepts and tools

Shan Huang, Grigory Bordyugov

Audience level:


A/B testing is at the core of analytics. We will cover the motivation behind A/B testing, A/B testing at Zalando, pitfalls and learnings, concepts of analysis methods (t-test in particular), Zalando’s open-source Python library: ExpAn, and hot trends of research topics in A/B testing.


In the field of analytics, A/B testing is nowadays of the essence. A/B testing is especially crucial to optimize business processes and user experience -- Microsoft, Airbnb, Twitter, etc are building up huge teams for their experimentation platforms.

In the talk, we will show our experiences of A/B testing at Zalando -- a German e-commerce company. Being the advantage of a large site, we’ve already run hundreds of tests, and we will share our learnings with you.

Moreover, we will introduce our open-source Python library ExpAn. It is developed for the statistical analysis of A/B testing and defines standard data structures for statistical experiments. You can import it easily into your project and analyze your own A/B tests seamlessly. In this talk, we will give a live coding session of using ExpAn.

Depending on the time, we might also briefly mention some hot trends of research topics in A/B testing.

Subscribe to Receive PyData Updates