EVALUATING POSTERIOR VITREOUS DETACHMENT ANNOTATION CONSISTENCY ON OPTICAL COHERENCE TOMOGRAPHY SCANS IN PATIENTS WITH DISEASE OF THE VITREOMACULAR INTERFACE.

Abstract

To evaluate intergrader variability in posterior vitreous detachment (PVD) classification in patients with epiretinal membrane and macular hole on spectral-domain optical coherence tomography (SD-OCT) and identify challenges in defining a reliable ground truth for artificial intelligence-based tools. A total of 437 horizontal SD-OCT B-scans were retrospectively selected and independently annotated by six experienced ophthalmologists adopting four categories: "full PVD," "partial PVD," "no PVD," and "ungradable." Intergrader agreement was assessed using pairwise Cohen kappa scores. Consensus levels, accuracy, recall, specificity, and grading time were also analyzed using the majority vote as reference. The overall average Cohen kappa was 0.57. Agreement was highest for "partial PVD" (Cohen kappa = 0.70), followed by "full PVD" (Cohen kappa = 0.65), and lowest for "no PVD" (Cohen kappa = 0.14), indicating substantial diagnostic variability. Full consensus was achieved in only 31.1% of OCT scans, whereas 11.4% required adjudication. The sensitivity for "no PVD" was notably low (0.35 ± 0.32), and misclassified OCT scans took significantly longer to grade ( P < 0.001). Our results underscore challenges associated with reliable OCT-based classification of PVD in patients with diseases of the vitreomacular interface, especially for cases with completely attached vitreous. Improving intergrader agreement through consensus grading and advanced imaging modalities will be critical for establishing a solid ground truth to support reliable, artificial intelligence-driven PVD detection systems.