Deep Zoom Images to Visualize Unsupervised Clustering of Slide Tiles

Unsupervised image classification techniques hold significant promise for pathology because they can group similar images without explicit annotations. These methods have been successfully applied in histopathology for tumor classification and identifying patches for training supervised approaches. Typically, the visualization process involves applying dimensionality reduction to embeddings and using unsupervised methods to create clusters, projecting images in two or three dimensions. Each tile is represented by a point, with cluster categories indicated by different colors. While methods exist to view corresponding images for each point or generate separate photomontages for each cluster, they do not allow for correlating the morphology of the tiles with their spatial distribution simultaneously. To address this, we created a deep zoom image with properties equivalent to a whole slide image, allowing visualization of individual tiles at full resolution, distributed in the embedding space. This work aims to demonstrate a proof-of-concept approach for visualizing tile images with spatial context in a deep zoom image. We selected 25 random whole slide images from the TCGA bladder cancer dataset. Tiles were extracted at 512 x 512 pixels and downsampled by a factor of 4, yielding 156,000 tiles. A binary threshold excluded tiles with over 50% white pixels. From this set, a balanced sample of 8,000 tiles across all cases was randomly selected. Embeddings were extracted using a foundational model (Prov-GigaPath), trained on one billion 256 x 256 pathology image tiles from over 170,000 whole slides. Dimensionality was reduced using Principal Component Analysis (PCA), and clustering was performed using the K-means algorithm with multiple cluster numbers (5, 6, 7, 8, and 9). To create the deep zoom image, we calculated the new location of each tile based on its coordinates in the K-means embedding space, generating a large canvas with tiles distributed proportionally to their cluster locations. Once the canvas was ready, each tile was placed in its appropriate position, enabling visualization of individual tiles at full resolution within their spatial context. The resulting deep zoom image spanned 260,000 pixels in each dimension. This spatial representation provided more information than a photomontage without spatial context. For bladder cancer whole slide images, the clusters included invasive cancer with desmoplasia, carcinoma without desmoplasia, urothelium, papillary urothelial carcinoma, stroma, blood, and artifacts such as blurring and cropped edges. Despite some contaminants between classes, the clusters were predominantly morphologically consistent. Impressively, even tiles with color pen marks (green, blue, red) were correctly clustered with tiles of the same class without ink, demonstrating the quality of embeddings created by the foundational model. Evaluating the deep zoom image provided greater context and understanding of how tiles were clustering. This method allows for a more robust evaluation of visual semantic clustering performance by visualizing the spatial distribution of tiles. It can be applied to multiple slides or cases, as well as single slides. The primary computational costs are associated with extracting embeddings from multiple patches and storing large images. Deep zoom images offer significant benefits by providing detailed spatial context, helping to understand the relationships between different clusters and the morphological characteristics of each tile. However, further exploration in various settings is needed to fully determine the method's value and potential applications.

Learning Objectives

1. To describe the advantages of unsupervised learning techniques.            
2. To recognize components of an unsupervised learning pipeline.
3. To understand how deep zoom images can be applied to contexts such as tile clustering visualization in addition to traditional whole slide images.