Article type
Year
Abstract
Background: Publication bias is a pervasive issue affecting scientific research. The failure to publish negative research findings can have a detrimental impact on the robustness of the evidence base that is used to update clinical treatment strategies and policies.
Objectives: To assess the frequency at which publication bias is formally assessed in systematic reviews (SRs) and meta-analyses (MAs) of interventions; and to determine which methods were used, how robust these methods were and how well the results were reported in order to make a series of best practice recommendations.
Methods: We searched KSR Evidence (ksrevidence.com) in October 2019 to identify SRs of immunotherapies for any type of cancer. We pre-specified that SRs had to report at least one MA and have a completed ROBIS risk of bias (RoB) assessment. We applied a rule of thumb that 10 was the minimum number of studies required for a publication bias assessment to be performed, based on Sterne 2011 recommendations. However, in general practice, we acknowledge that there may be occasions when this limit may be changed based on pooled study characteristics (e.g. sample size or heterogeneity).
Results: We identified 87 records and included 45 SRs. 33 of 45 SRs (73%) considered publication bias. 49% (of 33) SRs reported a visual inspection of funnel plots; 33% reported both funnel plots and statistical tests; 9% reported statistical tests alone; 3% reported funnel plots plus sensitivity analyses; 3% reported an informal approach; 3% did not report the approach used. Of the 26 (of 33) SRs that reported results, only 27% reported publication bias assessments for all outcomes; only 27% described the results comprehensively; and only 62% interpreted the results adequately.
15 SRs included
Objectives: To assess the frequency at which publication bias is formally assessed in systematic reviews (SRs) and meta-analyses (MAs) of interventions; and to determine which methods were used, how robust these methods were and how well the results were reported in order to make a series of best practice recommendations.
Methods: We searched KSR Evidence (ksrevidence.com) in October 2019 to identify SRs of immunotherapies for any type of cancer. We pre-specified that SRs had to report at least one MA and have a completed ROBIS risk of bias (RoB) assessment. We applied a rule of thumb that 10 was the minimum number of studies required for a publication bias assessment to be performed, based on Sterne 2011 recommendations. However, in general practice, we acknowledge that there may be occasions when this limit may be changed based on pooled study characteristics (e.g. sample size or heterogeneity).
Results: We identified 87 records and included 45 SRs. 33 of 45 SRs (73%) considered publication bias. 49% (of 33) SRs reported a visual inspection of funnel plots; 33% reported both funnel plots and statistical tests; 9% reported statistical tests alone; 3% reported funnel plots plus sensitivity analyses; 3% reported an informal approach; 3% did not report the approach used. Of the 26 (of 33) SRs that reported results, only 27% reported publication bias assessments for all outcomes; only 27% described the results comprehensively; and only 62% interpreted the results adequately.
15 SRs included