Critical evaluation of the scientific content in clinical practice guidelines.

Abstract

Increasing pressures to provide high-quality evidence-based cancer care have driven the rapid proliferation of clinical practice guidelines (CPGs). The quality and validity of CPGs have been questioned, and adherence to guidelines is relatively low. The purpose of this study was to critically evaluate the development process and scientific content of CPGs. CPGs addressing management of rectal cancer were evaluated. We quantitatively assessed guideline quality with the validated Appraisal of Guidelines Research & Evaluation (AGREE II) instrument. We identified 21 independent processes of care using the nominal group technique. We then compared the evidence base and scientific agreement for the management recommendations for these processes of care. The quality and content of rectal cancer CPGs varied widely. Mean overall AGREE II scores ranged from 27% to 90%. Across the 5 CPGs, average scores were highest for the clarity of presentation domain (85%; range, 58% to 99%) and lowest for the applicability domain (21%; range, 8% to 56%). Randomized controlled trials represented a small proportion of citations (median, 18%; range, 13%-35%), 78% of the recommendations were based on low- or moderate-quality evidence, and the CPGs only had 11 references in common with the highest-rated CPG. There were conflicting recommendations for 13 of the 21 care processes assessed (62%). There is significant variation in CPG development processes and scientific content. With conflicting recommendations between CPGs, there is no reliable resource to guide high-quality evidence-based cancer care. The quality and consistency of CPGs are in need of improvement.