Observer variability when evaluating patient movement from electronic portal images of pelvic radiotherapy fields.

Abstract

A study has been performed to evaluate inter-observer variability when assessing pelvic patient movement using an electronic portal imaging device (EPID). Four patient image sets were used with 3-6 portal images per set. The observer group consisted of nine radiographers with 3-18 months clinical EPID experience. The observers outlined bony landmarks on a digital simulator image and used matching software to evaluate field placement errors (FPEs) on each portal image relative to the reference simulator image. Data were evaluated statistically, using a two-component analysis of variance technique, to quantify both the inter-observer variability in evaluating FPEs and inter-fraction variability in patient position relative to the residuals of the analysis. Intra-observer variability was also estimated using four of the observers carrying out three sets of repeat readings. Eight sets of variance data were analysed, based on FPEs in two orthogonal directions for each of the four patient image sets studied. Initial analysis showed that both inter-observer variation and inter-fraction-patient position variation were statistically significant (P<0.05) in seven of the eight cases evaluated. The averaged root-mean-square (RMS) deviation of the observers from the group mean was 1.1 mm, with a maximum deviation of 5.0 mm recorded for an individual observer. After additional training and re-testing of two of the observers who recorded the largest deviations from the group mean, a subsequent analysis showed the inter-observer variability for the group to be significant in only three of the eight cases, with averaged RMS deviation reduced to 0.5 mm, with a maximum deviation of 2.7 mm. The intra-observer variability was 0.5 mm, averaged over the four observers tested. We have developed a quantitative approach to evaluate inter-observer variability in terms of its statistical significance compared to inter-fraction patient movement. This will assist us in training and assessing observers required to perform this task on a routine basis.