Development and Validation of a Shave Biopsy Training Checklist.

Abstract

Residencies train residents in procedures and assess their competency, but existing assessment tools have demonstrated poor reliability and have not been validated. This mixed-methods study validated a shave biopsy checklist with family medicine and dermatology faculty at two academic centers. In each phase of the study, teaching faculty scored a video-recorded simulated procedure using the checklist, and investigators assessed content validity, interrater reliability, and accuracy. In focus groups of nine family medicine and dermatology faculty, 16 of 18 checklist items met or surpassed 80% interrater reliability. Overall checklist reliability was 74%. Focus group surveys initially revealed insufficient content validity. Lowest performing items were removed, and then the follow-up content validity index (0.76) surpassed the required threshold (0.62). Twenty-one of 70 family medicine faculty completed a final survey, which showed a content validity index of 0.63, surpassing the required threshold of 0.42. Twelve of 70 family medicine faculty viewed and scored a simulated video-recorded procedure. Overall interrater reliability was 91% (Cohen's d=1.36). Fourteen of 16 checklist items demonstrated greater than or equal to 90% interrater reliability. Accuracy analysis revealed 67.9% correct responses in focus groups and 84.9% in final testing (simple t test, P<.001, Cohen's d=1.4). This rigorously validated checklist demonstrates appropriate content validity, interrater reliability, and accuracy. Findings support use of this shave biopsy checklist as an objective mastery standard for medical education and as a tool for formative assessment of procedural competency.