Integrating orthogonal supervision for sparse semi-supervised 3D medical image segmentation.

Abstract

Recent advances in semi-supervised learning (SSL) have significantly improved 3D medical image segmentation. However, fully annotating volumetric data slice by slice remains costly and time-consuming. To address this, sparsely annotated SSL has been explored, where only a few representative slices from a limited set of labeled volumes are annotated, while the rest of the 3D data remains unlabeled. Despite this progress, the performance of these methods remains suboptimal, as they typically utilize separate models for each orthgonal view and generate pseudo-labels for the entire volume. This view-specific training introduces incomplete and biased learning. Given that the available supervision is already sparse, we argue that it is more effective to integrate all supervisory signals from orthogonal views into a single, integrated framework. By simultaneously leveraging the complementary information from axial, sagittal, and coronal planes, the model can develop a richer and more holistic understanding of volumetric data distributions, even under limited supervision. Based on this insight, we propose Integrating Orthogonal Supervision (IOS), a strategy that employs a single 3D encoder to learn from all orthogonal planes and construct a comprehensive knowledge representation. Furthermore, we employ a Tri-Decoder Framework to effectively learn from unlabeled slices within sparsely labeled volumes as well as completely unlabeled volumes. A cross-decoder supervision strategy is applied between two decoders to enhance learning from unlabeled slices within the labeled volumes, while the pseudo-labels jointly generated by these decoders are used to guide the training of a third decoder on unlabeled 3D volumes. This design establishes a disentangled learning flow between sparsely labeled and unlabeled data, and the collaboratively generated pseudo-labels from both decoders are more reliable than those produced by a single decoder-thereby improving the overall quality of unlabeled supervision. We validate our framework on three benchmark datasets-LA, Pancreas, and KiTS19-where it consistently outperforms existing state-of-the-art methods under sparse supervision.