Andrew Sohn, MD
Johns Hopkins Medicine
Multiple instance contrastive learning for classification and interpretability of pancreatic fine needle aspiration whole slide images.
Background: To date, the majority of deep learning for digital pathology have been utilized for surgical pathology specimens, and for deep learning studies that have been reported for cytopathology, only a tiny handful have investigated fine needle aspiration whole slide imaging (FNA WSI). FNA WSI poses unique challenges in contrast to surgical pathology (i.e., multiple staining for a single case, multiple z-planes captured when digitally scanned, sparsely scattered cells of interest, and generally consists of more noise and various artifacts), which make using deep learning techniques less readily accessible for FNA WSI. We propose a deep learning pipeline which addresses these issues for pancreatic FNA WSI classification.
Methods: We trained a weakly-supervised (diagnostic label for the entire slide) deep neural network classifier via combining contrastive and multiple instance learning, called multiple-instance class-activation contrastive learning (MICACL). First, we thresholded the cells based on the cytology stain (e.g. DiffQuick vs. Pap) and extracted instances (tiles) based on this thresholding. The multiple instance contrastive learner generates class activation maps for the two classes (positive for carcinoma vs. negative for carcinoma) for each instance, which are then used to disentangle the foreground (e.g. cells of interest) from the background (e.g. drying/stain artifact). Specifically, probabilities of the respective classes for each instance are recovered from the class activation map logits, and the instance probabilities above thresholded values for the two classes are sent for contrastive learning. Finally, the class activation map logits for pooled for a slide-level classification. We also adapted and reproduced Mahmood lab's clustering-constrained attention multiple-instance learning (CLAM) network, which has found success in surgical pathology, for our pancreatic FNA WSI dataset and compared it to our proposed network.
Results: On the holdout test dataset over ten folds, CLAM had a test accuracy average of 90.14% and a receiver operating characteristic area under the curve (ROC-AUC) of 0.94095. Our network, MICACL, had an improved test accuracy and ROC-AUC over CLAM, of 94.73% and 0.94725, respectively. Furthermore, the data points originally labeled negative for carcinoma but which were given a high probability of adenocarcinoma by both networks were visualized. Upon analyzing the regions of interests (ROIs) for these cases, both networks revealed human error (i.e., incorrect labeling of the WSI) in a small fraction of the original negative labels.
Conclusion: We have reproduced, adapted, and confirmed that CLAM also works well for the domain of cytopathology WSI. However, due to the sparsity of the cells of interest with abundant background noise and artifact, it doesn't perform optimally for binary classification. Our network, which was built with these two issues in mind, demonstrates superior (+4.59%) performance to the already impressive CLAM. Furthermore, both networks have the ability to 'point' to the region of interest within a FNA WSI that drove the final slide level classification; however, instance probability derivation of MICACL points to ROIs more accurately than that of CLAM's gated-attention. We suggest current FNA screenings can be significantly augmented, and in some cases, even increase precision using deep learning for digital cytopathology.
- Strategic instance clustering when using weakly supervised deep learning for cytopathology (i.e. fine needle aspiration) whole slide imaging is critical.
- Appreciate the differences between surgical pathology and cytopathology whole slide images for deep learning.
- Differing approaches for instance (i.e., tiles) clustering and pooling for multiple instance learning.
Andrew Sohn was born and raised in Queens, NY. He attended Vassar College, where he graduated with a degree in Chemistry, including a senior thesis on computational electrochemistry of carbon nanotubes grown on silicon wafers. Andrew received his M.D. from Sidney Kimmel Medical College at Thomas Jefferson University. During medical school, he completed research fellowships in the Medical Research Scholars Program at the NIH, followed by the HHMI Medical Research Fellows Program at the University of Pennsylvania. For his research, Andrew focused on machine learning and computer vision for histopathology and epigenomics. Andrew is a Physician-Scientist Track resident (AP only) at Johns Hopkins Hospital in the Department of Pathology. Currently, he is completing his NIH T32 Postdoctoral Fellowship between his PGY-2 and PGY-3 in the lab of Alexander Baras, MD/PhD. Andrew's research interests include vision-language deep learning models, MLOps, and differentiable programming for genomics.