Comparison of AMSTAR 2 and ROBIS Ratings and Inter-Rater Reliability for Each Scale Based on Systematic Reviews/Meta-Analyses.

Abstract

To compare the ratings of quality and risk of bias of systematic reviews/meta-analyses (SRs/MAs) in bariatric research published in 2016 and 2017 using the AMSTAR 2 and ROBIS tools. The methodology is described in detail in our first publication (Obes Rev. 2020; https://doi.org/10.1111/obr.12994). A systematic survey by searching three electronic databases was conducted to identify SRs/MAs assessing any outcome in patients with morbid obesity undergoing or scheduled for bariatric surgery published in 2016 and 2017. Pairs of reviewers independently screened titles and abstracts, assessed full texts, extracted data, and evaluated the quality and risk of bias using the AMSTAR 2 and ROBIS tools. Discrepancies were resolved by a third reviewer. Agreement between reviewers and between the instruments was assessed using Fleiss' kappa and Gwet's statistics. Matching between AMSTAR 2 items (critical and noncritical) and ROBIS signaling questions was also examined. Of the 78 included SRs/MAs, 77 (98.7%) were assessed as critically low methodological quality by AMSTAR 2, while ROBIS classified five reviews (6.4%) as low risk of bias, 61 (78.2%) as high, and 12 (15.4%) as unclear. Both tools demonstrated moderate inter-rater reliability for individual items and overall judgments. The methodological quality of SRs/MAs in bariatric research was predominantly critically low. Calibration exercises are strongly recommended before applying either tool. AMSTAR 2 and ROBIS each address a distinct research question, and tool choice should be guided by the specific study objective. PROSPERO Registration: CRD42017080394.