Does the META score evaluating osteoporotic and metastatic vertebral fractures have enough agreement to be used by orthopaedic surgeons with different levels of training?

Abstract

Differentiating osteoporotic vertebral fractures (OVF) from metastatic vertebral fractures (MVF) is difficult. A magnetic resonance imaging (MRI)-based score (META score) aiming to differentiate OVF and MVF was recently published; however, an independent agreement assessment is required before the score is used. We performed such independent agreement evaluation, including raters with different levels of training. Sixty-four patients with confirmed OVF or MVF were evaluated by six raters (three spine surgeons and three orthopaedic residents) using the META score. We used the intra-class correlation coefficient (ICC) to evaluate inter- and intra-observer agreement and the kappa statistic (κ) to determine the agreement for individual score criteria. We calculated the area under the receiver-operating characteristic curve (AUC) to establish the score accuracy. The inter-observer agreement was poor [ICC = 0.22 (0.12-0.33)]; spine surgeons [ICC = 0.75 (0.66-0.83)] had better agreement than that of residents [ICC = 0.06 (- 0.07 to 0.23)]. The intra-observer agreement was poor [ICC = 0.15 (- 0.04 to 0.30)]; both spine surgeons [ICC = 0.21 (0.05-0.41)] and residents exhibited poor agreement [ICC = - 0.06 (- 0.40 to 0.20)]. The agreement for each specific criterion varied from κ = 0.24 to κ = 0.38. The AUC was 0.57 (0.64 for spine surgeons and 0.51 for residents, p < 0.01). The inter-observer agreement using the META score was adequate for spine surgeons but not for residents; the intra-observer agreement was poor. These results do not support the standard use of the META score to differentiate OVF and MVF. These slides can be retrieved under Electronic Supplementary Material.