Inter-modality assessment of medial temporal lobe atrophy in a non-demented population: application of a visual rating scale template across radiologists with varying clinical experience.

Abstract

To assess inter-modality agreement and accuracy for medial temporal lobe atrophy (MTA) ratings across radiologists with varying clinical experience in a non-demented population. Four raters (two junior radiologists and two senior neuroradiologists) rated MTA on CT and MRI scans using Scheltens' MTA scale. Ratings were compared to a consensus rating by two experienced neuroradiologists for estimation of true positive and negative rates (TPR and TNR) and over- and underestimation of MTA. Inter-modality agreement expressed as Cohen's κ (dichotomized data), Cohen's κ_w, and two-way mixed, single measures, consistency ICC (ordinal data) were determined. Adequate agreement was defined as κ/κ_w ≥ 0.80 and ICC ≥ 0.80 (significance level at 95% CI ≥ 0.65). Forty-nine subjects (median age 72 years, 27% abnormal MTA) with cognitive impairment were included. Only junior radiologists achieved adequate agreement expressed as Cohen's κ. All raters achieved adequate agreement expressed as Cohen's κ_w and ICC. True positive rates varied from 69 to 100% and TNR varied from 85 to 100%. No under- or overestimation of MTA was observed. Ratings did not differ between radiologists. We conclude that radiologists with varying experience achieve adequate inter-modality agreement and similar accuracy when Scheltens' MTA scale is used to rate MTA on a non-demented population. However, TPR varied between radiologists which could be attributed to rating style differences. • Radiologists with varying experience achieve adequate inter-modality agreement with similar accuracy when Scheltens' MTA scale is used to rate MTA on a non-demented population. • Differences in rating styles might affect accuracy, this was most evident for senior neuroradiologists, and only junior radiologists achieved adequate agreement on dichotomized (abnormal/normal) ratings. • The use of an MTA scale template might compensate for varying clinical experience which could make it applicable for clinical use.