iDNA-DAPHA: a generic framework for methylation prediction via domain-adaptive pretraining and hierarchical attention.

Abstract

Accurately identifying DNA methylation is essential for understanding complex regulatory networks and disease mechanisms. However, the dynamic nature of methylation and species differences make prediction challenging. Existing deep learning methods often overlook the potential of shared features across diverse species' methylation sequences and rely solely on token-to-token attention when modelling long-range dependencies, limiting the model's representation capabilities. To address these limitations, we propose iDNA-DAPHA, an accurate and generic two-stage deep learning framework that leverages domain-adaptive pretraining (DAP) incorporating feature alignment to learn common features across various types of methylation sequences from multiple species, followed by fine-tuning to capture task-specific features. The framework further introduces hierarchical attention (HA) to enhance its representational power. Experimental results demonstrate that iDNA-DAPHA performs better than existing state-of-the-art methods across seventeen benchmark datasets covering three representative DNA methylation types. Ablation studies validate the effectiveness and contributions of DAP and HA. Furthermore, visualization-based analyses reveal that the model can capture conserved sequence patterns and learn discriminative representations. We believe that iDNA-DAPHA will serve as a valuable framework for methylation prediction, especially in scenarios with limited training samples for specific methylation types in certain species.