Unbiased Learning to Rank:
Counterfactual and Online Approaches

Half-day tutorial at The Web Conference 2020.


This tutorial is about Unbiased Learning to Rank, a recent research field that aims to learn unbiased user preferences from biased user interactions. We will provide an overview of the two main families of methods in Unbiased Learning to Rank: Counterfactual Learning to Rank (CLTR) and Online Learning to Rank (OLTR) and their underlying theory.

For LTR practitioners our tutorial gives guidance on how the decision between methods should be made. For the field of Information Retrieval (IR) we aim to provide an essential guide on unbiased LTR to understanding and choosing between methodologies.


A handout of the slides are available here.
You can watch the tutorial here:

If you have any questions we want to hear them!
Tweet us at: @mdr @RolfJagerman or @HarrieOos and use #ltrtutorial
We will address your questions in a follow-up video.


Harrie Oosterhuis Harrie Oosterhuis
University of Amsterdam

Rolf Jagerman Rolf Jagerman
University of Amsterdam

Maarten de Rijke Maarten de Rijke
University of Amsterdam