Mark Zarella, PhD
Technical Director, Pathology Imaging & Informatics
Drexel University College of Medicine
Peering into the “black box”: a systems-based approach to understanding convolutional neural networks
Background. Deep learning has been responsible for significant growth in computational pathology, in which artificial neural networks are trained to identify and process features with prognostic potential that may be present in histologic images. However, one of the major shortcomings of deep learning is that it produces classifiers that are not easily interpretable; that is, they are considered “black boxes.” As a result, it may be difficult to inspire confidence in novel AI algorithms in pathology. Method. A number of methods have been devised to analyze deep networks and the image features that contribute to a decision although, at present, few have been applied specifically in pathology. We employed methods commonly used in general image classification to characterize deep networks trained on three distinct pathology tasks. We also introduce a novel method that uses established insights from tumor histology to probe network activations by relating synthetic modulations at the input to the activations generated at the output. We independently modulated color/staining, nuclear geometry, and histologic organization to relate these complex feature spaces to network outputs. Results. We found that conventional network visualization methods, when not constrained by known histologic parameters, reveal artificial features with arguably little value to pathologists. By using synthetic modulation of histologic images, we identified output activations strongly dependent on staining and histologic content. Using an occlusion test, we confirmed that the presence and absence of these features, even in a very small portion of the image, can alter classification. Conclusions. By characterizing the features used by a deep network, a more transparent understanding of AI can be achieved. However, these techniques also reveal the impact of what may be considered spurious features. Nevertheless, these insights can have an impact on tuning deep learning approaches in the future, including promoting new data augmentation strategies, regularization techniques, and identifying potential shortcomings in the training data set.
Apply AI to histologic images and understand the features used in classification
Mark Zarella is a Research Assistant Professor at Drexel University and Technical Director of the Pathology Imaging & Informatics service in the Department of Pathology. He received his BS in physics from the University of Massachusetts in 2002 and his PhD in neuroscience from SUNY Upstate Medical University in 2011. His current research interests include whole-slide imaging, in vivo and ex vivo optical microscopy, and computational pathology.