Comparison of discrepancy rates in resident and faculty interpretations of on-call PE CT and V/Q scans: is one study more reliable during off hours?

Abstract

There has been no comparison of concordance rates for residents' and faculty members' interpretations of pulmonary embolism (PE) CT and ventilation/perfusion (V/Q) studies. If significantly different rates of agreement are demonstrable, this could influence which test is ordered during off hours. The purpose of this study was to evaluate how the performance of residents compared with that of faculty members in interpreting these two modalities. Interobserver agreement between preliminary resident reports and final attending radiologist interpretations was calculated retrospectively for PE CT studies (n = 1,179) and V/Q scans (n = 331) interpreted by on-call residents from April 2007 to October 2008. Discordant cases were reviewed for clinical significance and outcomes at 3 months. Interobserver variability was also evaluated relative to residents' training levels and faculty members' years of experience. Interobserver agreement between faculty members and residents was substantial, at 95% for PE CT (κ = 0.77) and 77.9% for V/Q scans (κ = 0.67). Although changes in interpretation were significantly more common with V/Q scans (22.1% vs 5%; χ(2) < .0001), the rates of clinically significant discrepancy did not differ significantly between the modalities (2.1% for V/Q vs 1.2% for PE CT; χ(2)P = .20). The overall discrepancy rate and the rate of clinically significant discrepancy did not correlate with residents' training levels or faculty members' years of experience. Concordance rates for residents' interpretations of V/Q scans and PE CT studies were high, and discrepancies resulting in changes in patient management were rare for both modalities. Residents' preliminary interpretations of both modalities are reliable and safe for making initial patient management decisions.