Reliability, Usability, and Face Validity of the Capacity Assessment Tool for Orthopedic Surgery (CAT-os) in Low- and Middle-Income Countries (LMICs): A Pilot Study.

Abstract

Trauma injuries account for over 4.5 million deaths annually, with a disproportionately higher impact in low- and middle-income countries (LMICs). Enhancing surgical capacity in these regions is crucial. This pilot study evaluates the reliability, usability, and face validity of the Capacity Assessment Tool for orthopedic surgery (CAT-os), designed to facilitate surgical capacity building in LMICs. The CAT-os was administered to 10 surgeons affiliated with the College of East, Central, and Southern African (COSECSA). We calculated the two-way random-effects model interrater reliability intraclass correlation coefficient (ICC) and assessed usability through the system usability scale (SUS), summarizing feedback with a mean CAT-os SUS score. Interviews and thematic analysis were conducted to assess the CAT-os validity in adequately measuring aspects of orthopedic capacity building in LMICs. Ten surgeons representing seven countries participated in the study. The ICC for two raters evaluating the same local program was 0.971 (95% CI: 0.95-0.98), indicating excellent reliability. The mean SUS score was 70.0 (95% CI: 62.07-77.93), reflecting good usability. Participants' interview transcripts revealed themes consistent with the CAT-os effectiveness in measuring critical aspects of capacity building. The CAT-os demonstrated excellent reliability and good usability in assessing needs for capacity building in orthopedic surgery consistent with stakeholder perspectives on its comprehensiveness capturing domains of capacity building. These results support the CAT-os as a consistent and a practical tool for guiding surgical capacity building in LMICs.