MLN2SVG: domain-aware spatially variable gene detection using contrastive variational autoencoder and multi-level neighbor search.

Abstract

Spatial transcriptomics (ST) technologies have transformed our ability to examine gene expression within intact tissues, yet accurately identifying spatially variable genes (SVGs) remains challenging due to spatial heterogeneity, data sparsity, and incomplete modeling of domain-level dependencies. To address these limitations, we propose MLN2SVG, a domain-aware framework that integrates contrastive variational autoencoding with a multi-level neighbor (MLN) search algorithm to jointly learn tissue domains and SVGs. MLN2SVG constructs a weighted spatial graph to capture both local and long-range spatial relationships, employing a deep contrastive variational autoencoder to align augmented and original data representations while preserving biological diversity. The MLN algorithm dynamically expands neighborhood connectivity to mitigate sparsity and enhance domain coherence. Across multiple human and mouse ST datasets, including dorsolateral prefrontal cortex, breast cancer, and brain tissues, MLN2SVG consistently outperformed existing methods in clustering accuracy, robustness, and biological interpretability. Notably, in breast cancer tissues, MLN2SVG uncovers fine-grained spatial organization of tertiary lymphoid structures, delineating region-specific immune architectures spanning intratumoral, tumor-edge, and extratumoral compartments. Through the integration of spatial domain discovery and SVG detection, MLN2SVG delivers a robust and biologically interpretable framework for uncovering the molecular and structural complexity of tissue organization.