Automatic Radiology Report Generation Based on State-Space Model.

Abstract

The objective of radiology report generation is to alleviate the burden on physicians in drafting reports, thereby improving generation efficiency and reducing patient waiting times. In recent years, there has been a growing emphasis on imaging-based monitoring technologies within healthcare, with a particular focus on the precise detection and interpretation of subtle changes. Likewise, as individual X-rays often exhibit minimal differences, pathologies are frequently concealed within intricate details, making accurate report generation a challenge. To address this issue, we propose a novel method consisting of three key modules: the Self-Attention Mamba Module (Self-Mamba), the Cross-Attention Mamba Module (Cross-Mamba), and the Sparse Mask Loss Function (Sparse-Loss). When processing an X-ray, we use a similar approach to human observation, focusing first on the overall structure and then focusing on identifying possible focal areas. For this purpose, we design Self-Mamba module to extract the features of abnormal areas in X-ray images through global information modeling. The proposed Cross-Mamba module enhances the consistency of medical images and radiology reports by optimizing the ability of cross-modal interaction between the two. Sparse-Loss function is proposed to alleviate the problem of unbalance of positive and negative samples by taking advantage of its sparsity. Experimental results show that our approach outperforms existing models on several metrics and achieves excellent performance on two publicly available datasets, IU-Xray and COV-CTR.