Article type
Year
Abstract
Introduction:
As part of a systematic review a meta-analysis aims to provide an unbiased summary of data from the literature. However, potentially important studies could be missing because of selective publication and inadequate dissemination of results. If results of missing studies differ systematically from published ones, a meta-analysis will be biased with an inaccurate assessment of the intervention’s effects.
Objectives:
As part of the OPEN project (www.open-project.eu) we conducted a systematic review of methodological research projects (MRPs) that assessed whether inclusion of studies that are not published or published in the grey literature impacts on pooled effect estimates in meta-analyses (quantitative measure) and leads to different conclusions in meta-analyses (qualitative measure).
Methods:
Four bibliographic databases were searched with no limit to publication year or language. MRPs were considered eligible for inclusion if they reviewed a cohort of meta-analyses which 1) compared pooled effect estimates of meta-analyses of health care interventions according to publication status or 2) examined whether the inclusion of unpublished/grey studies impacts the overall finding of a meta-analysis.
Results:
Seven MRPs including 1866 studies (from 1348 published studies) that compared pooled treatment effect estimates between published and unpublished/grey studies were identified. One MRP showed that published studies had larger pooled treatment effects in favour of the intervention than unpublished studies (Ratio of ORs 1.15, 95% CI 1.04-1.28). In the remaining six MRPs pooled effect estimates and overall findings were not changed by the inclusion of unpublished/grey studies.
Conclusion: There is only limited evidence from MRPs for an impact of studies that are not fully published on results of meta-analyses or systematic reviews. However, in several individual examples (e.g., a meta-analysis of reboxetine by Eyding et al 2010 published in BMJ) it could be shown that the inclusion of unpublished studies led to findings that differed from those based on published data.
As part of a systematic review a meta-analysis aims to provide an unbiased summary of data from the literature. However, potentially important studies could be missing because of selective publication and inadequate dissemination of results. If results of missing studies differ systematically from published ones, a meta-analysis will be biased with an inaccurate assessment of the intervention’s effects.
Objectives:
As part of the OPEN project (www.open-project.eu) we conducted a systematic review of methodological research projects (MRPs) that assessed whether inclusion of studies that are not published or published in the grey literature impacts on pooled effect estimates in meta-analyses (quantitative measure) and leads to different conclusions in meta-analyses (qualitative measure).
Methods:
Four bibliographic databases were searched with no limit to publication year or language. MRPs were considered eligible for inclusion if they reviewed a cohort of meta-analyses which 1) compared pooled effect estimates of meta-analyses of health care interventions according to publication status or 2) examined whether the inclusion of unpublished/grey studies impacts the overall finding of a meta-analysis.
Results:
Seven MRPs including 1866 studies (from 1348 published studies) that compared pooled treatment effect estimates between published and unpublished/grey studies were identified. One MRP showed that published studies had larger pooled treatment effects in favour of the intervention than unpublished studies (Ratio of ORs 1.15, 95% CI 1.04-1.28). In the remaining six MRPs pooled effect estimates and overall findings were not changed by the inclusion of unpublished/grey studies.
Conclusion: There is only limited evidence from MRPs for an impact of studies that are not fully published on results of meta-analyses or systematic reviews. However, in several individual examples (e.g., a meta-analysis of reboxetine by Eyding et al 2010 published in BMJ) it could be shown that the inclusion of unpublished studies led to findings that differed from those based on published data.