Saturday 16:30–17:05 in Megatorium

Continuous online experimentation: 101 A/B tests for personalised news.

Ivo Everts, Matthijs

Audience level:
Intermediate

Description

A consortium of regional news providers in the Netherlands called Regiolab have joined forces to innovate on their news platforms by personalization. There, Dutch startup Primed.IO introduced their AI-infra and teamed up with GoDataDriven for continuous online experimentation with a variety of recommender models, thereby optimizing on metrics such as session duration and retention rate.

Abstract

As news and media are abundantly available nowadays, it is important for news providers to stay relevant by finding their niche and keeping up with technological advancements. This, however, can be a daunting task in the context of today's rapid developments of tech and associated costs. In particular, this is challenging for regional public news providers (RPO's, in Dutch) as they typically do not work with the budgets required for substantial innovation.

To accommodate for this, a consortium of RPO's in the Netherlands called Regiolab joined forces in 2018. It is their goal to remain relevant by attracting more visitors to their online channels, who then should more frequently return because they are happier about the larger amounts of relevant media that they more elaborately consume. In order to reach these goals, the news content ought to be personalized.

Many of the news providers operate on similar frontend and backend modules, making it viable to implement a central personalization platform. The first pilot projects have been implemented only for 'Omroep Gelderland', the RPO which has taken the lead in these developments.

At Omroep Gelderland, Dutch startup Primed.IO has installed their AI infrastructure for collecting web+app events, serving news recommendations and performing other kinds of experiments such as A/B tests on news headlines. This constitutes a platform for continuous deployment and monitoring of experiments, which is implemented on Google Cloud Platform using Kubernetes, Spark, Python, etc.

GoDataDriven is responsible for the data science part in this project. In order to optimize KPI's such as session duration and retention rate, it is investigated if it is better to consider e.g. article clicks or article reads in a collaborative filtering algorithm; using a time decay on the ratings (and what the shape of the decay function should be); the location of the news given that we are dealing with RPO's; visual features relating to the quality and content of associated photographs; blending of any of the above models.

Primed.IO and GoDataDriven team up in this presentation and will elaborate on all the nitty gritty details of continuous online experimentation.

Subscribe to Receive PyData Updates

Subscribe