Randomised controlled trial to assess the effect of a Just-in-Time training on procedural performance: a proof-of-concept study to address procedural skill decay.

Abstract

A subset of high-risk procedures present significant safety threats due to their (1) infrequent occurrence, (2) execution under time constraints and (3) immediate necessity for patient survival. A Just-in-Time (JIT) intervention could provide real-time bedside guidance to improve high-risk procedural performance and address procedural deficits associated with skill decay. To evaluate the impact of a novel JIT intervention on transvenous pacemaker (TVP) placement during a simulated patient event. This was a prospective, randomised controlled study to determine the effect of a JIT intervention on performance of TVP placement. Subjects included board-certified emergency medicine physicians from two hospitals. The JIT intervention consisted of a portable, bedside computer-based procedural adjunct. The primary outcome was performance during a simulated patient encounter requiring TVP placement, as assessed by trained raters using a technical skills checklist. Secondary outcomes included global performance ratings, time to TVP placement, number of critical omissions and System Usability Scale scores (intervention only). Groups were similar at baseline across all outcomes. Compared with the control group, the intervention group demonstrated statistically significant improvement in the technical checklist score (11.45 vs 23.44, p<0.001, Cohen's <i>d</i> effect size 4.64), the global rating scale (2.27 vs 4.54, p<0.001, Cohen's <i>d</i> effect size 3.76), and a statistically significant reduction in critical omissions (2.23 vs 0.68, p<0.001, Cohen's <i>d</i> effect size -1.86). The difference in time to procedural completion was not statistically significant between conditions (11.15 min vs 12.80 min, p=0.12, Cohen's <i>d</i> effect size 0.65). System Usability Scale scores demonstrated excellent usability. A JIT intervention improved procedure perfromance, suggesting a role for JIT interventions in rarely performed procedures.