Ground Truth Uncertainty

Quick links: Pre-Print | Code | Disagreement Workshop Slides | PRECISE Slides


Figure 1: Overview of our proposed framework to measure annotation uncertainty and evaluate AI models while taking ground truth uncertainty into account; see the paper for details.

For safety, AI systems in health undergo thorough evaluations before deployment, validating their predictions against a ground truth that is assumed certain. However, this is actually not the case and the ground truth may be uncertain. Unfortunately, this is largely ignored in standard evaluation of AI models but can have severe consequences such as overestimating the future performance. To avoid this, we measure the effects of ground truth uncertainty, which we assume decomposes into two main components: annotation uncertainty which stems from the lack of reliable annotations, and inherent uncertainty due to limited observational information. This ground truth uncertainty is ignored when estimating the ground truth by deterministically aggregating annotations, e.g., by majority voting or averaging. In contrast, we propose a framework (cf. Figure 1) where aggregation is done using a statistical model. Specifically, we frame aggregation of annotations as posterior inference of so-called plausibilities, representing distributions over classes in a classification setting, subject to a hyper-parameter encoding annotator reliability. Based on this model, we propose a metric for measuring annotation uncertainty and provide uncertainty-adjusted metrics for performance evaluation. We present a case study applying our framework to skin condition classification from images where annotations are provided in the form of differential diagnoses. The deterministic adjudication process called inverse rank normalization (IRN) from previous work ignores ground truth uncertainty in evaluation. Instead, we present two alternative statistical models: a probabilistic version of IRN and a Plackett-Luce-based model. We find that a large portion of the dataset exhibits significant ground truth uncertainty and standard IRN-based evaluation severely over-estimates performance without providing uncertainty estimates.

Download & Citing

The paper is available on ArXiv:

Paper on ArXiv

  title={Evaluating AI systems under uncertain ground truth: a case study in dermatology},
  author={David Stutz and Ali Taylan Cemgil and Abhijit Guha Roy and Tatiana Matejovicova and Melih Barsbey and Patricia Strachan and Mike Schaekermann and Jana von Freyberg and Rajeev Vijay Rikhye and Beverly Freeman and J. Perez Matos and Umesh Telang and Dale R. Webster and Yuan Liu and Greg S Corrado and Yossi Matias and Pushmeet Kohli and Yun Liu and Arnaud Doucet and Alan Karthikesalingam},


The code for this paper can be found on GitHub:

Code on GitHub

It allows to reproduce the results from the paper on a toy dataset and in the future also on the skin condition classification dataset; some of the included components:

  • A toy dataset consisting of multiple overlapping Gaussians that allows to generate ambiguous examples and sample annotations
  • Statistical aggregation models: probabilistic IRN (PrIRN) and Plackett-Luce with partial rankings
  • A Gibbs sampler for the Plackett-Luce model
  • Uncertainty-adjusted metrics: top-k accuracy, top-k set accuracy and average overlap (with partial rankings)