Rapid Identification of Low-quality Meta-analyses in Plastic Surgery Using the Meta-analysis Critique (MAC) Tool.

Abstract

Meta-analyses are the highest level of evidence within academic health literature. However, they vary significantly in quality and bias. Existing methodological assessment tools for meta-analyses are often time-consuming and difficult for clinicians to use. The "Meta-Analysis Critique" (MAC) Tool, was developed as a simple three-question screening instrument designed to quickly appraise study quality. We analyzed for temporal trends and correlations between the MAC Tool and the gold standard AMSTAR 2. MEDLINE and EMBASE were searched for plastic surgery meta-analyses published between 2011 and 2023, identifying 283 studies. These were independently assessed by two reviewers using both the gold-standard AMSTAR 2 and the novel MAC Tool. While the volume of meta-analyses in plastic surgery significantly increased in the latter half of the decade (n=68 [24%] vs n=215 [76%]), the proportion of high-quality studies did not improve. When compared to AMSTAR 2 score, our novel MAC Tool obtained high specificities when assessing the research question (100.00%, CI: 90.97%-100.00%), appropriateness of the statistical analysis (96.23%, CI: 87.02%-99.54%), and risk of bias (98.97%, CI: 94.39-99.97%), respectively. Taken together, we present a novel tool that simplifies assessment of plastic surgery literature and identifies poor quality meta-analyses but is not intended as a confirmatory quality assessment instrument. It has a high correlation with the gold-standard AMSTAR 2. Future studies should aim to validate this tool and determine its broader reliability in other specialties.