Evaluation of the prototype Anaesthetic Non-technical Skills for Anaesthetic Practitioners (ANTS-AP) system: a behavioural rating system to assess the non-technical skills used by staff assisting the anaesthetist.

Abstract

This study tested the reliability, validity and usability of a prototype behavioural rating system for the non-technical skills of assistants working with the anaesthetist. Anaesthetic nurses and operating department practitioners (n = 48) used the prototype Anaesthetic Non-technical Skills for Anaesthetic Practitioners (ANTS-AP) system to rate the non-technical skills of anaesthetic assistants in 12 videos of simulated theatre work. Test-retest reliability was assessed with a sub-sample (n = 12). The skill categories assessed were 'situation awareness', 'teamwork and communication' and 'task management'. The internal consistency for the ratings of elements in categories was acceptable (Cronbach's α of 0.78, 0.77 and 0.69, respectively), with more modest inter-rater reliability (intraclass correlations for categories 0.54, 0.70, 0.86), test-retest reliability (intraclass correlations 0.68, 0.58, 0.38) and accuracy (weighted kappa 0.39). Most participants considered the system complete (n = 42, 87%), the wording clear (n = 48, 100%) and the system useful for structuring observation (n = 48, 100%).