Simulator-based assessment of obstetric and gynecological ultrasound skills: validity of three different simulators and feasibility of remote evaluation.

Abstract

The primary objective was to compare the evaluation of trainees' obstetric and gynecological (OBGYN) ultrasound skills by an in-person vs remote examiner. The secondary objectives were to compare the assessment of obstetric ultrasound skills using three types of ultrasound simulator vs traditional examination involving a pregnant volunteer, and to evaluate the perspectives of trainees and examiners on the three ultrasound simulators. This was a prospective study of 12 physicians with minimal experience in OBGYN ultrasound. After completing a structured training course, trainees performed an ultrasound examination at four stations: one involving a mid-trimester pregnant volunteer (obstetric examination only) and three using ultrasound simulators (OBGYN examination), namely an ultrasound simulator app, a high-fidelity mannequin simulator and an optical positioning simulator. At each station, two certified maternal-fetal medicine specialists assessed the scans simultaneously, with one examiner present in the room and the other observing remotely via live video. The assessments were based on the 20-planes approach from the International Society of Ultrasound in Obstetrics and Gynecology Basic Training program and the Objective Structured Assessment of Ultrasound Skills (OSAUS) scale. An anonymous survey was used to gauge the subjective impressions of the maternal-fetal medicine specialists and trainees on the three ultrasound simulators. The image scores from remote vs in-person assessment using the 20-planes approach showed moderate-to-good agreement across three of the four stations, with intraclass correlation coefficient (ICC) values ranging from 0.51 (95% CI, 0.06-0.80) to 0.89 (95% CI, 0.26-0.99). However, remote assessment of OBGYN ultrasound images obtained using the high-fidelity mannequin simulator yielded lower scores compared with in-person assessment, with an ICC of 0.28 (95% CI, -0.32 to 0.72). Additionally, scores on the OSAUS scale obtained remotely differed significantly from those obtained in person for all stations (P < 0.05 for all). Using the 20-planes approach, the obstetric image scores obtained during the traditional examination involving a pregnant volunteer were similar to those obtained using the optical positioning simulator, but not to those from the other two simulators. The OSAUS scores were similar only between the ultrasound simulator app and the high-fidelity mannequin simulator, and between the optical positioning simulator and assessment on a pregnant volunteer, when the assessment was conducted by an in-person examiner. Examiners and trainees generally agreed that ultrasound simulators offer benefit in the training and assessment of OBGYN ultrasound skills. Simulation-based techniques show potential for evaluating OBGYN ultrasound competence. It may be possible for evaluation to be conducted remotely by experts certified in maternal-fetal medicine. While promising as an educational and training tool, the role of simulation as a method for remotely assessing ultrasound skills warrants further investigation. © 2026 International Society of Ultrasound in Obstetrics and Gynecology.