Is a resident's score on a videotaped objective structured assessment of technical skills affected by revealing the resident's identity?

Abstract

The purpose of this study was to determine whether knowledge of resident's identity affects the evaluation of surgical skills. During an objective structured assessment of technical skills, 33 residents in obstetrics and gynecology who performed 10 surgical skills were videotaped with two digital cameras simultaneously. One camera videotaped "hands only," concealing resident identity; the other camera videotaped from the "waist up," revealing identity. Residents wore opaque gowns and gloves. Four faculty reviewers with previous clinical experience with the residents scored the "hands only" videos first. The scoring was based on the total of a global rating scale and a task-specific checklist. The combined scores for all reviewers was significantly different between the two views (P=.03). When the video tapes were analyzed individually, two reviewers scored the views differently; one reviewer scored higher, whereas the other reviewer scored lower. Bias can occur in evaluation of surgical skills but is dependent on the individual reviewer. Resident identification can alter the score, depending on the evaluator.