Conformal predictions address diagnostic ambiguity to improve dermatopathology lesion classification

Background: Increased skin biopsy frequency and complexity, coupled with a reduced pathologist workforce, have challenged dermatopathology (D-path) workstreams. PathAssist Derm (PA-D)* is a computer vision-powered digital pathology tool designed to aid D-path case review. Here, we examine the tool's conformal predictions, a dynamic number of possible classifications ('prediction set') based on model confidence.

Methods: PA-D was trained on H&E-stained whole slide images (N=11,543) of 17 D-path classes, using the PLUTO v4 foundation model with additive multiple instance learning for classification. During training, classification scores were calibrated to balance coverage (the likelihood that the 'correct' answer is in the prediction set) and set size. Conformal predictions were applied to construct prediction sets (maximum set size of three). To evaluate prediction sets, a held-out test set (N=519) was manually labeled by 5 pathologists; consensus labels were determined by majority vote.

Results: PA-D accurately distinguished neoplastic, inflammatory, and normal cases in the test set, with 77.9% agreement for the top predicted class. In cases with pathologist consensus (N=407), coverage of the consensus label in the prediction set was 89.8%, with 74.3% accuracy of the top predicted class. In cases with 3-2 consensus (N=56), 64.3% of the minority labels were also present in the prediction set. Model confidence was correlated with the degree of pathologist majority (Pearson r=0.438, p<0.001).

Conclusions: PA-D is a promising digital pathology tool for aiding pathologist review in D-path. PA-D's conformal predictions showed high coverage of consensus pathologist labels despite the large number of classes and disagreement among pathologists, with model uncertainty related to pathologist label ambiguity. Real-world deployments are needed to better understand how conformal prediction sets can affect D-Path workflows. *For Research Use Only. Not for use in diagnostic procedures.

Learning Objectives:

1. Define conformal prediction sets for pathology images
2. Explain how conformal predictions address issues of diagnostic ambiguity in dermatopathology