A multi-branch ConvNeXt-MaxViT fusion transformer model for radiographic knee osteoarthritis severity assessment with Grad-CAM++ explainability.

Abstract

Knee Osteoarthritis (KOA) is a progressive degenerative joint disorder and a major cause of disability worldwide. Radiographic grading using the Kellgren-Lawrence (KL) scale remains the standard diagnostic method, but inherent subjectivity and difficulty in detecting subtle structural changes reduce diagnostic reliability. Thus, an automated, robust, and explainable system is clinically essential. This study aims to develop an explainable deep learning model capable of accurately classifying KOA severity (KL-0 to KL-4) by integrating both local and global radiographic features, thereby enhancing transparency for clinical decision support. The proposed model introduces a dual-branch fusion architecture combining ConvNeXt-Tiny (Convoluional Neural Network with Next Generation Design) for local feature extraction and MaxViT-Tiny (Multi-Axis Vision Transformer) for global contextual modeling. Mid-level fusion (concatenation + Multi-Layer Perceptron(MLP)) and late fusion (weighted ensemble) are implemented to strengthen discriminative representation. Model performance is evaluated using Accuracy, Precision, Recall, F1-score, Area Under the Curve (AUC), Confusion Matrix, Receiver Operating Characteristics (ROC), and explainability via Gradient-weighted Class Activation Mapping Plus Plus (Grad-CAM++). The final late-fusion ensemble achieved 96.2 % accuracy, 95.2 % F1-score, 95.4 % precision, 95.1 % recall, and a macro-AUC of 0.981. Class-wise F1-scores were highest for KL-0 (97.1 %) and KL-4 (97.8 %). Five-fold cross-validation confirmed model stability with CI = 96.2 % ± 0.6 %. Grad-CAM++ correctly localized osteophytes, joint-space narrowing, and sclerosis, strengthening interpretability. Future work will integrate multi-modal imaging (X-ray and MRI), incorporate longitudinal progression modeling, and evaluate real-world clinical deployment among radiologists to further improve generalizability, robustness, and practical usability of KOA severity assessment systems.