Sentence-Level Cross-Referencing Improves Comprehension and Confidence in AI-Generated Patient-Friendly Radiology Reports: Design for Understanding.

Abstract

Determine whether the format of artificial intelligence (AI)-generated patient-friendly reports (PFRs) affects laypeople's comprehension of medical information. In a within-subjects study, 116 participants viewed radiology reports alongside AI-generated PFR versions in three formats: dictionary (term definitions), block (full translation), and a novel sentence-based format (paired source-translation sentences for cross-referencing). Objective and subjective comprehension, confidence, and response time were measured. Relative to the dictionary format, objective comprehension odds were higher for sentence (odds ratio [OR] 2.31 [95% confidence interval 1.36-3.94]) and block (OR 2.23 [1.30-3.80]), and subjective comprehension was higher for sentence (OR 4.63 [2.24-9.57]) and block (OR 2.18 [1.13-4.23]). Response confidence was higher for sentence (OR 3.75 [2.41-5.84]) and block (OR 2.21 [1.45-3.35]) than dictionary, with sentence higher than block (adjust P < .05). Confidence decreased on later trials (OR 0.82 [0.69-0.97]) and for longer reports (OR 0.55 [0.41-0.75]). Response times were faster for sentence (ratio 0.74 [0.66-0.83]) and block (ratio 0.71 [0.64-0.80]) than dictionary; longer reports and higher health-confidence slowed responses. Higher response confidence associated with greater objective (OR 2.75 [1.84-4.13], P < .001) and subjective comprehension (b = 0.59, P < .001). Presentation format is a key, modifiable factor influencing how laypeople interpret AI-generated PFRs. Sentence and block formats showed higher comprehension, confidence, and faster responses than a dictionary format. The novel sentence-based format elicited the highest confidence, suggesting that cross-referencing between AI output and source text fosters trust and supports understanding. These findings highlight the critical role of user interface design in effective AI-generated patient communication.