Interactive voice response to assess residents' laparoscopic skills: an instrument validation study.

Abstract

The study was undertaken to evaluate the psychometric properties of the Interactive Voice Response (IVR) instrument to assess laparoscopic surgical skills of residents. Surgical competence of obstetric-gynecologic residents at the University of Ottawa (Canada) was evaluated after observed performance of skills. Three questions addressing general domains of surgical skill were rated per surgery using a 5-point Likert scale: overall, 993 surgical ratings were made by 29 residents and 13 faculty preceptors. With use of a generalizability study, a minimum of 12 preceptor ratings (G=0.80) are needed to obtain reliable measures of residents' surgical skills. Item analysis indicated that the three items on the instrument were highly correlated (Cronbach alpha=.92). Elements of construct validity were evident for diagnostic and simple laparoscopic procedures. The IVR instrument can be a valid and reliable measure of residents' surgical skills. The IVR ratings functioned as a global rating scale, capturing overall elements of surgical skills. The ratings on the IVR tool had construct validity with improved performance with increased surgical training and experience.