The assessment of physiotherapy practice (APP) is a reliable measure of professional competence of physiotherapy students: a reliability study.

Abstract

What is the inter-rater reliability of the Assessment of Physiotherapy Practice (APP) instrument, and what is the error associated with individual scores? Cross-sectional inter-rater reliability study. Thirty pairs of clinical educators each assessed one student after observing student practice over a 5-week clinical placement. Sixty clinical educators from five Australian universities formed 30 independent pairs of assessors. Each pair completed two independent assessments of one student, providing 60 completed APP assessments and an associated Global Rating Scale score for analysis. Correlational coefficients and measurement error expressed in APP scale units were computed to provide a comprehensive analysis of the likely utility of APP scores and to enable score and change score interpretation. Percentage of agreement between assessors for each item ranged from 56% (Item 19, evidence-based practice) to 83% (Item 20, risk management) and across all items averaged 70% (SD 7). The ICC(2,1) was 0.92 (95% CI 0.84 to 0.96) for the total APP score and 0.72 (95% CI 0.50 to 0.86) for the Global Rating Scale. The standard error of measurement for the total score (scale width 0-80) was 3.2 APP points and the MDC(90) was 7.86 representing 9% of the scale width. Bland-Altman analyses identified no systematic differences between raters. Clinical educators demonstrated a high level of reliability when using the APP instrument to assess physiotherapy students' level of professional competence in workplace-based practice.