DNAmBERT: a transformer-based model for non-invasive cancer diagnosis using DNA sequence and methylation data.

Abstract

DNA methylation alterations are early and stable hallmarks of cancer and represent promising biomarkers for non-invasive detection using circulating cell-free DNA (cfDNA). However, current computational approaches often model DNA sequence and methylation features separately and struggle to capture complex read-level methylation architecture in heterogeneous, low-signal liquid biopsy data. Here, we present DNAmBERT, a Transformer-based deep learning framework designed to jointly model DNA sequence context and read-level methylation haplotype structure from cfDNA methylation sequencing data. DNAmBERT integrates k-mer-encoded DNA sequences with methylation haplotype tokens using a unified representation and masked language modelling objective, enabling context-aware learning of sequence-epigenetic dependencies through self-attention. We evaluated DNAmBERT across multiple cfDNA methylation platforms (RRBS, cfRRBS, and cfMethyl-seq) and cancer types, including colorectal cancer, lung adenocarcinoma and hepatocellular carcinoma. In binary classification tasks, the model achieved high performance across platforms (AUC up to 0.99-1.00) and outperformed conventional machine learning and existing deep learning approaches. Aggregation of read-level predictions enabled quantitative tumour probability estimation at the sample level. Beyond binary detection, DNAmBERT supported multi-cancer and stage-aware classification, including early-stage disease, with multiclass AUC values up to 0.99. The framework further demonstrated effective cross-cancer transfer learning, maintaining robust performance under limited data availability. These results indicate that integrated sequence-haplotype representation learning provides an accurate and scalable approach for cfDNA-based multi-cancer detection.