MDD-MARF: a multimodal depression detection model based on multi-level attention mechanism and residual fusion.

Abstract

Depression is a serious mental disorder that significantly affects patients' work ability and social functioning. With the rapid development of artificial intelligence, researchers have begun to explore automatic depression detection methods based on multimodal data. However, multimodal data are often accompanied by a large amount of noise. Existing methods usually lack sufficient feature screening after extraction and are directly applied to downstream tasks, which may limit the model's generalization ability. In addition, current multimodal fusion strategies still face several challenges. To address these challenges, we propose a novel multimodal depression detection model that integrates three modalities: audio, vision, and text. The model extracts depression-related key features through a multi-level attention mechanism and achieves efficient multimodal feature fusion using skip connections with a residual structure. Experiments conducted on the DAIC-WOZ dataset showed that the proposed method achieved a mean absolute error (MAE) of 3.13 and a root mean square error (RMSE) of 3.59, outperforming existing state-of-the-art models. The generalization ability of the model was further validated on the E-DAIC dataset, demonstrating its effectiveness and robustness. The proposed method provides an efficient and reliable solution for depression detection using multimodal data and multi-level attention mechanisms. The findings highlight the significant value of multimodal learning in the medical field and offer strong support for the development of AI-assisted clinical decision-making systems.