Development of the SBIRT checklist for observation in real-time (SCORe).

Abstract

Screening, Brief Intervention and Referral to Treatment (SBIRT) programs have been implemented widely in medical settings, with little attention focused on how well providers adhere to evidence-based service delivery in everyday practice. The purposes of this paper were to: (1) introduce a flexible, relatively simple methodology, the SBIRT Checklist for Observation in Real-time (SCORe), to assess adherence to evidence-based practice and provide preliminary evidence supporting its criterion validity; and (2) illustrate the feasibility and potential utility of the SCORe by analyzing observations of providers within four large-scale SBIRT programs in the United States. Eighteen potential adherence judges were trained to recognize SBIRT service elements presented in realistic taped portrayals constructed to serve as criterion coding standards. Across the four SBIRT programs, 76 providers were observed performing 388 services in three types of medical settings; emergency departments (n = 10), hospital out-patient/ambulatory clinics (n = 16) and hospital in-patient settings (n = 5). Across two exercises, trainees identified 81% of screening and 75% of brief intervention (BI) elements correctly; for the six FRAMES components (Feedback, Responsibility, Advice, Menu of options, Empathy, Self-efficacy), agreement ranged from 69% to 91%. Across programs, 56% of screening, 54% of brief intervention (BI) (81% of FRAMES) and 53% of referral to treatment elements were observed. Programs differed significantly in adherence [screening, P = 0.024; BI, P < 0.001; FRAMES, P < 0.001; referral to treatment (RT), P < 0.001]; medical setting differences were minimal. The Screening, Brief Intervention and Referral to Treatment Checklist for Observation in Real-time provides a flexible method for assessing adherence to evidence-based Screening, Brief Intervention and Referral to Treatment service protocols. Preliminary evidence supports the criterion validity, feasibility and potential utility of the Screening, Brief Intervention and Referral to Treatment Checklist for Observation in Real-time protocol.