Causality-inspired representation learning with spatiotemporal memory for polyp detection in endoscopic videos.

Abstract

Early detection of colorectal polyps is crucial to reduce the morbidity and mortality associated with colorectal cancer. However, during endoscopy, continuous camera motion and the complex clinical environment often degrade key visual cues (e.g., polyp morphology, texture, and boundaries). This leads to cross-frame view shifts, which are characterized by heterogeneous appearances of the same lesion over time, thereby introducing spurious correlations that hinder reliable assessment. To address this, we propose a causality-inspired representation learning framework with spatiotemporal memory for polyp detection in colonoscopy videos (CIRL-Polyp). Specifically, we develop a novel View-Shift-Aware Causal Intervention Module (VACIM) to remove non-causal influences by enforcing prediction invariance under view shift perturbations. To further constrain non-causal factors, we introduce a dual-branch detection framework that processes the original and intervention frame sequences in parallel and enforces prediction consistency across branches, thereby promoting invariance to non-causal variations. In addition, we propose Causal Temporal Consistency Memory (CTCM) to leverage long sequence-dependency features and stabilize causal representations by constructing memory banks across branches and performing temporal consistency-enhanced cross-attention. Comprehensive experiments on two public video datasets and a private dataset demonstrate that CIRL-Polyp outperforms existing methods, which validates the effectiveness and suggests the clinical potential of the proposed framework from a causal perspective.