Development and Preliminary Examination of a Fidelity Assessment for the Spinal Cord Injury Movement Index.

Abstract

OBJECTIVE: The study aimed to describe the development and refinement of a fidelity assessment for a novel performance-based item pool, the Spinal Cord Injury Movement Index (SCI-MovIn), and report on its interrater agreement. DESIGN: Guidance for the development of intervention fidelity assessments was adapted to create a framework for the development of an assessment fidelity checklist. An initial draft of the SCI-MovIn fidelity checklist was drafted. The set-up and administration fidelity criteria were exposed to iterative modified Delphi surveys with individuals knowledgeable about the SCI-MovIn. Interrater agreement was then assessed through field-testing of the fidelity assessment by fidelity raters who viewed the same video-recorded SCI-MovIn testing sessions. SETTING: Academic institution in an urban area. PARTICIPANTS: Three individuals participated in the modified Delphi survey (purposeful sample). Three individuals served as fidelity raters for interrater agreement testing. INTERVENTIONS: Not applicable. MAIN OUTCOME MEASURES: The modified Delphi technique was completed iteratively until 100% agreement among respondents was reached that every criterion on the fidelity assessment was relevant, clear, specific, and in alignment with the response scale. Descriptive statistics were used to examine exact percent interrater agreement for the ratings on the SCI-MovIn fidelity assessment across 3 raters. RESULTS: Four survey rounds were iteratively completed until 100% agreement among respondents was reached that fidelity criteria were relevant, clear, specific, and aligned with the fidelity response scale. Across 21 sessions (394 items), raters' total absolute exact percent agreement on scores of fidelity was 79.33%. Nine of 13 criteria had >75% agreement. CONCLUSIONS: The SCI-MovIn fidelity assessment was developed using a framework modified from guidance for intervention fidelity assessments. It was refined over 4 rounds of a modified Delphi process and then exposed to field-testing for rater agreement.