Multi-Scale Dynamic Sparse Token Multi-Instance Learning for Pathology Image Classification.

Abstract

In many challenging breast cancer pathology images, the proportion of truly informative tumor regions is extremely limited. The disparity between the essential information required for clinical diagnosis (Tumor area less than 10$\%$) and the vast amount of data within Whole Slide Images (WSIs) makes it exceedingly difficult for pathologists to identify subtle lesions. To address the labor-intensive task imposed by this information gap, this paper proposes a dynamic sparse token based multi-instance learning framework. This framework incorporates a dynamic sparse layer into the transformer architecture, gradually adapting to selectively filter key instances beneficial for the task. Furthermore, to tackle complex scenarios in pathology image tasks, we introduce a weakly supervised cross-scale contrastive learning framework. This framework leverages pathology image features at different scales to perform contrastive learning at the bag-level representation to overcome existing challenges in multi-scale feature fusion in pathology image tasks. To validate the effectiveness and transferability of the model, we conducted various single-scale and multi-scale experiments across four cancer datasets and conducted interpretable analyses. Compared to other state-of-the-art methods, our classification model demonstrates superior performance across six evaluation metrics.