The value of simulation testing for the evaluation of ambient digital scribes: a case report.
case_report · Level V
Where this comes from
- Record sourced from PubMed, PMID 40116912.
- Also identified by DOI 10.1093/jamia/ocaf052 and PMC identifier 12012335.
- No licence information is recorded for this record.
- Because redistribution is not established, this page shows the abstract only. Follow the links below for the full text.
Abstract
The objective of this work is to demonstrate the value of simulation testing for rapidly evaluating artificial intelligence (AI) products. Researcher-physician teams simulated the use of 2 Ambient Digital Scribe (ADS) products by reading scripts of outpatient encounters while using both products, yielding a total of 44 draft notes. Time to edit, perceived amount of effort and editing, and errors in the AI-generated draft notes were analyzed. Ambient Digital Scribe Product A draft notes took significantly longer to edit, had fewer omissions, and more additions and irrelevant or misplaced text errors than ADS Product B. Ambient Digital Scribe Product A was rated as performing better for most encounters. Artificial intelligence-enabled products are being rapidly developed and implemented into practice, outpacing safety concerns. Simulation testing can efficiently identify safety issues. Simulation testing is a crucial first step to take when evaluating AI-enabled technologies.
Medical subject headings
- Artificial Intelligence
- Electronic Health Records
- Documentation
- Computer Simulation