Mayo Atlas - Indexing Tissue, Reports and Molecular Data to Search in Pathology Archives
With post-pandemic emergence of digital pathology, the significance of artificial intelligence (AI) in medicine has been on steady rise. AI models, particularly deep networks, have been proposed for a variety of tasks to assist physicians in triaging, diagnosis, prognosis, and treatment planning of diverse diseases. These tasks include processing and analysis of various data modalities such as text documents (e.g., lab and diagnostic reports), images (radiology and pathology), and molecular data (e.g., RNA sequences) for detection, quantification, identification and categorization of diseases.
The overwhelming majority of AI models are trained in a “supervised” manner to make a decision by classifying data. In contrast to a large body of classifiers, the concept of information matching and retrieval, although quite old, has not yet been implemented in any hospital. In recent months, we have focused on “unsupervised” and “self-supervised” models to design the “Mayo Atlas”, a multi-modal repository of different diseases that can be used for generating computational consensus for the decision-in-focus. In this sense, Mayo Atlas will be an information retrieval platform which is agnostic to disease, organ and modality, a fundamental departure from the mainstream AI in order to implement a physician-centric platform to address the observer variability in medicine. By querying Mayo Atlas, pathologists will be able to search for answers by matching their patient data to the data of indexed patients, cases that have been evidently diagnosed and treated. A computational consensus report provides physicians with explainable clues and majority-based confidence values to help decrease variability in final decision. Mayo Atlas will be multi-modal whereas various modalities will be added depending on the clinical practice for the corresponding disease. Mayo Atlas will be designed by incorporating self-supervised learning of transformer architectures – based on the concept of attention – using novel “ranking loss” to adjust what is learned to the ultimate goal of matching patients, a process that is meant to computationally imitate the “consult” process when physicians rely on each other’s knowledge.
In this talk, we explore the general structure of the Mayo Atlas, and we report the initial results for three cases: skin cancer (risk strtification for cutenous squamous cell carcinoma), liver disease (ASH versus NASH), and breast cancer (epithelial subtypes).
Objectives:
- Learn how to build a pathology atlas (unsupervised AI) - data/patient selection, generating embeddings for all modalities using pre-trained networks or foundation models, index aggregation to make the atlas multi-modal
- Learn how to validate the performance of an atlas - apply "leave-one-patient-out" validation to measure the performance of the atlas using stringent metrics such as "majority vote"
- Learn how to use an atlas (unsupervised AI) - use the empirical evidence to generate a computational report (likely primary diagnosis, visualizations )
Presented by:
Hamid R. Tizhoosh, PhD
Professor
Mayo Clinic
Hamid R. Tizhoosh is the director of Rhazes Lab and a Professor of Biomedical Informatics in the Department of Artificial Intelligence and Informatics, Mayo Clinic, Rochester, MN, USA. Before joining the Mayo Clinic, he worked at the University of Toronto and Waterloo. Since 1996, his research activities encompass artificial intelligence, computer vision and medical imaging. He has developed algorithms for medical image filtering, segmentation and search. As well he has introduced the concept of "Opposition-based Learning". He is the author of two books, several book chapters, and a large body of peer-reviewed journal and conference papers. Dr. Tizhoosh has extensive industrial experience and more than ten years of experience in commercialization and creating and advising start-ups.
