Domain-General vs. Domain-Specific Pre-trained Models: Binary Patch Grouping for Improved WSI Representation
Introduction: Histopathology is considered the reference standard for making a diagnosis in pathology. This field has undergone a transformation with the introduction of whole slide images (WSIs), i.e. digitized glass slides. Trained histopathologists can diagnose diseases by examining WSIs visually, but this visual inspection is time-consuming and inconsistent. To address these challenges, AI models are being developed to generate slide-level representations of WSIs that summarize the entire slide as a single vector. However, generating expressive and robust slide-level representations is a challenging task, where the quality of the representation depends on the specific process used. The extraction of patch features and the aggregation of these features are the two critical steps in the process. To improve representation quality, we propose integrating an additional Binary Patch Grouping (BPG) step between the two steps. We demonstrate that BPG increases the performance of WSI retrieval, improves WSI classification, and improves the quality of the embeddings associated with WSI.
Methods: We employed pre-trained models including DenseNet-121, KimiaNet, ViT-16/256 (HIPT_4K) and the ViT-S/16 (DINO) as feature extractors. We trained a k-Means algorithm to cluster all the patch feature vectors in a profiling dataset that includes 64,790 patch feature vectors from 716 WSIs into two groups. Using the k-Means on the patch feature vectors from a reference slide and examining the two clusters, we decided on the desired target (BPG) and the non-target (NBPG). We aggregated the outputs after this step into WSI representations using non-parameterized and parameterized pooling and measured their quality by downstream computational pathology tasks.
Results: The UMAP projections of slide-level representations created from patches selected through BPG show a distinct clustering pattern. In contrast, the projections of that without using BPG show a less pronounced clustering pattern. In the comparison of WSI retrieval task performance, the mAP@10 score for the BPG approach was about 4% higher than the baseline approach and about 15% higher than that of the NBPG. In the comparison of WSI classification task performance, the F1-micro score for the BPG approach was about 5% higher than the baseline approach, and 20% higher than that of the NBPG. The pipeline consisted of BPG, DINO and parameterized pooling achieved the highest performance: 0.506±0.014 of mAP@10 and 0.476±0.023 of f1-micro (CV=5, train-test-split=0.5).
Conclusion: Our findings revealed that removing irrelevant patches through BPG improved slide-level representations and downstream computational pathology tasks. Additionally, the pipeline with BPG, domain-general large models and parameterized pooling produced the best-quality slide-level representations as evaluated by these downstream tasks. The simplicity and minimum human intervention of BPG may make it more appealing than complex frameworks. BPG's benefits apply to all feature extractors and aggregation methods, suggesting relevance to computational pathology software design for WSI representation in digital pathology.
Keywords: deep learning, whole slide images, slide-level representation, histology
Objectives:
- Understand the role of slide-level representation in supporting digital pathology.
- Understand current pipelines and approaches to slide-level representation.
- Understand who adding a binary patch grouping (BPG) step may improve slide-level representation agnostic of feature extraction and aggregation methods as a broadly applicable tool in AI-based computational pathology.
Presented by:
Clinton Campbell, MD, PhD
Hematopathologist
William Osler Health System
Dr. Clinton Campbell is a hematopathologist at William Osler Health System and Assistant Clinical Professor of Medicine at McMaster University in Ontario Canada. He has held numerous peer-reviewed grants in artificial intelligence applied to pathology. Dr. Campbell’s research focuses on using machine learning to 1) automate workflows in diagnostic medicine; 2) develop new representations of the information in pathology and 3) enable multimodal representations by linking information between healthcare datasets. Dr. Campbell’s research program is conducted in collaboration with Dr. Hamid Tizhoosh at the Mayo Clinic.
