Human and AI Collaboration to Embrace Diagnostic Diversity and Redefine Ground Truth

Artificial intelligence (AI) is increasingly supporting diagnostic pathology-not by replacing pathologists, but by enhancing their capabilities. A key challenge in this integration is defining 'ground truth,' especially in areas with high interobserver variability and limited biological correlates.Rather than viewing diagnostic disagreement as a failure of human judgment, it should be recognized as a reflection of scientific uncertainty-particularly when experts interpret histologic findings differently. Histologic patterns often lack absolute standards, and while genomic or clinical outcomes provide valuable context, they may yield non-definitive conclusions depending on the methodologies used. AI has the potential to clarify such diagnostic variability rather than suppress it.We previously studied 18 expert pulmonary pathologists who independently evaluated over 5,000 lung adenocarcinoma cases/images. Clustering analysis revealed two diagnostic subgroups (comprising 10 and 5 experts, with 3 excluded). Using each subgroup's consensus as ground truth, we trained two separate AI models, both of which demonstrated similarly strong correlations with patient survival. These two AI models were subsequently applied in practice and evaluated by five pathologists (1 beginner, 2 junior, 2 senior), who assessed 99 cases before and after AI assistance. Diagnostic agreement was measured using Cohen's kappa and analyzed with the Wilcoxon signed-rank test.The two models showed high concordance (&kappa; = 0.926). Agreement between pathologists and the AI significantly improved, particularly for the beginner (&kappa; = 0.547&rarr;0.923, p < .0001) and junior pathologists (p = .0265). Inter-rater agreement among juniors also increased (&kappa; = 0.593&rarr;0.810, p = .0463), and subtype-level consistency improved across all histologic patterns. The mean Cohen's &kappa; increased from 0.316 (95% CI: 0.246-0.387) before AI assistance to 0.489 (95% CI: 0.399-0.579) after AI use.These findings demonstrate that AI enhances diagnostic reproducibility and serves as an educational tool, particularly for less experienced pathologists. Providing outputs from two distinct AI models encouraged active diagnostic reflection rather than passive acceptance.Importantly, many cases may not have a single correct diagnosis due to the current lack of granular biological correlates. This diagnostic ambiguity underscores the need for further refinement. Ground truth should be viewed as a dynamic, evolving construct. AI's role is not to eliminate complexity but to illuminate it-and to help both humans and machines advance together in diagnostic precision.

Learning Objectives: 

1. To investigate the significance and implications of interobserver variability in pathology
2. To understand the challenges in defining ground truth across complex domains of pathology
3. To examine how human AI collaboration can lead to diagnostic innovations beyond the current capabilities of either alone