Assessing endovascular skills using the Simulator for Testing and Rating Endovascular Skills (STRESS) machine.

Abstract

Endovascular techniques are an integral part of modern-day vascular surgery practice and training. Nevertheless, validated in vitro assessment tools for these skills are scarce. This study describes the development and pilot testing of the Simulator for Testing and Rating Endovascular Skills (STRESS machine). The design was kept straightforward and compact, without the need for contrast or fluoroscopy. A specific technical skill score was designed analogous to the Imperial College Evaluation of Procedural Skill (ICEPS), an assessment score for open surgical skill. This score was combined with an already validated global rating assessment to form the total score (TS). A pilot study was carried out on 18 candidates of varying levels of expertise: novice, intermediate and expert, who were assessed by two independent observers to test inter-observer reliability. Inter-observer reliability was excellent, Cronbach's alpha coefficient of the TS was 0.94 (95% confidence interval: 0.84-0.97). A one-way analysis of variance (ANOVA) showed a significant difference between the novice and expert groups (p<0.001), between the novice and intermediate groups (p<0.01) and between the intermediate and expert groups (p<0.05). The STRESS machine, in combination with the specific technical skill score and global rating assessment, provides a reliable method of discriminating between the novice, intermediate and expert candidates with excellent inter-observer variability.