Implementation outcomes of AI documentation support in routine clinical practice: A non-randomized controlled trial.

Abstract

While large language models (LLMs) show promise for reducing clinical documentation burden, existing studies rely on simulated environments or require supplemental audio inputs. This study evaluates an AI documentation copilot integrated into real-world electronic health record (EHR) workflows, measuring its impact on physician efficiency and documentation review burden. We conducted a non-randomized controlled trial with 27 ophthalmology clinicians over 12 months (January 2024-February 2025). Participants self-selected into intervention (n = 11) or control (n = 17) groups after a 4-month baseline. The AI system generated inline suggestions for discharge letter conclusions using only the patient's EHR record as input. The system was based on a Quantized Low-Rank Adaptation (QLoRA) tuned LLM trained on 80,000 institutional records. Primary outcomes included words-per-minute (WPM) rates and the count of required document revisions after checks by the supervising senior physician across 15,615 documents. Intervention group clinicians achieved greater WPM improvements than controls (Δ+7.2 vs. Δ+2.6), while revision frequency decreased within the intervention group (from 1.36 ± 0.14 to 1.26 ± 0.08 per document), with no significant change observed in the control group. Linear mixed-effects modeling confirmed a significant group × exposure interaction for documentation efficiency (p = 0.045), whereas no statistically significant interaction was observed for revision count. This real-world evaluation suggests that AI documentation assistance is associated with improved documentation efficiency under routine clinical conditions, while maintaining documentation standards under established supervision processes. These findings support the cooperative "copilot" design as a viable approach to enhance documentation efficiency without compromising patient safety protocols and indicate its potential to help reduce documentation burden in clinical practice.