Article type
Abstract
Background
Umbrella reviews (URs) synthesize systematic reviews and meta-analyses and can provide valuable evidence-based health knowledge to inform decision-making. However, methods and reporting of URs vary and have limitations, including suboptimal reporting of the overall certainty of evidence for a given outcome and failure to report absolute estimates of effect.
Objectives
To innovate the guidance for the conduct and reporting of UR as it pertains to (1) absolute estimates of effect and (2) the overall certainty of evidence for each outcome.
Methods
We convened a panel of methodologists with experience in conducting URs and expertise in guideline development to address our research objectives through iterations of discussion. We then used a published UR of saturated fat and patient-important outcomes (Johnston, 2023) to examine the feasibility of these methods.
Results
We illustrate how to calculate the absolute estimates of effect for binary outcomes when conducting a UR. In this example, population-level data (eg, global data) and a meta-analysis of cohort studies were used to estimate the baseline risks for a range of patient-important outcomes. Using these baseline risks, the absolute estimates of effect were calculated based on the relative effect size with GRADEprofiler. For the certainty of evidence, we recommended using the rating as assessed by the authors of the included systematic reviews, as was done in our case study. When the certainty rating in a UR is not credible or not reported at all, we can consider performing a de novo appraisal using GRADE. We discuss the relative merits and limitations of these critical steps and provide guidance for these aspects of conducting and reporting URs.
Conclusions
Reporting the absolute estimates of effect and certainty of evidence when conducting URs can offer clear information for decision-makers. We outline approaches to enhancing the usability and completeness of this information.
Umbrella reviews (URs) synthesize systematic reviews and meta-analyses and can provide valuable evidence-based health knowledge to inform decision-making. However, methods and reporting of URs vary and have limitations, including suboptimal reporting of the overall certainty of evidence for a given outcome and failure to report absolute estimates of effect.
Objectives
To innovate the guidance for the conduct and reporting of UR as it pertains to (1) absolute estimates of effect and (2) the overall certainty of evidence for each outcome.
Methods
We convened a panel of methodologists with experience in conducting URs and expertise in guideline development to address our research objectives through iterations of discussion. We then used a published UR of saturated fat and patient-important outcomes (Johnston, 2023) to examine the feasibility of these methods.
Results
We illustrate how to calculate the absolute estimates of effect for binary outcomes when conducting a UR. In this example, population-level data (eg, global data) and a meta-analysis of cohort studies were used to estimate the baseline risks for a range of patient-important outcomes. Using these baseline risks, the absolute estimates of effect were calculated based on the relative effect size with GRADEprofiler. For the certainty of evidence, we recommended using the rating as assessed by the authors of the included systematic reviews, as was done in our case study. When the certainty rating in a UR is not credible or not reported at all, we can consider performing a de novo appraisal using GRADE. We discuss the relative merits and limitations of these critical steps and provide guidance for these aspects of conducting and reporting URs.
Conclusions
Reporting the absolute estimates of effect and certainty of evidence when conducting URs can offer clear information for decision-makers. We outline approaches to enhancing the usability and completeness of this information.