Article type
Year
Abstract
Introduction: Meta-analysts have little control over random errors but substantial control over systematic ones. Including only a portion of all available evidence may introduce systematic errors into the review process and threaten its validity. We set out to examine whether language restricted meta-analysis [i.e. excluding trials published in languages other than English from the quantitative analysis], compared to language inclusive meta-analysis, provide different estimates of the effectiveness of interventions evaluated in randomized trials.
Objective/Methods: We identified and retrieved 79 meta-analyses from several disease areas in which explicit eligibility criteria, regarding trial selection was reported. General characteristics and quality of reporting of all 79 meta-analyses was assessed using a validated instrument. We replicated and repeated [i.e. completing the quantitative analysis by removing those trials published in languages other than English] 18 meta-analyses, involving 211 randomized trials, using published data from the primary studies. To explore the effects of language of publication of the randomized trials on the quantitative results, we completed logistic regression analyses.
Results: Meta-analyses that included all randomized trials regardless of their language of publication, compared to language restricted meta-analysis, shared similar general characteristics. However, the former included more randomized trials and larger sample size compared to the latter. Similarly, the quality of reports of language inclusive meta-analysis [median; interquartile range: 3, 4], compared to language restricted meta-analysis [median =3; interquartile range:3.0, 5.25], was higher. Language restricted meta-analyses [n=l8], compared to language inclusive meta-analyses, did not influence the estimate of benefit of the effectiveness of an intervention [ROR = 0.94; 95%CI: 0.79, 1.13]. This result did not change whether the meta-analyses included one trial [ROR = 0.96; 95%CI: 0.78, 1.18] or more [ROR = 1.04; 95%CI: 0.74, 1.47] published in a language other than English. These results were also robust after a series of sensitivity analyses. We observed an average 65% [OR=0.35; 95%CI: 0.29, 0.41] risk reduction associated with the effectiveness of the interventions across the 211 randomized trials. However, this result was only modified by 6%, on average, regardless of the language of publication of the trials included.
Discussion: We encourage meta-analysts to identify and include all relevant randomized trials regardless of their language of publication. Although the inclusion of trials published in all languages may not influence the estimate of effectiveness of an intervention, they are likely to provide a more precise estimate of the its effectiveness. This is an important clinical and statistical attribute of a meta-analysis.
Objective/Methods: We identified and retrieved 79 meta-analyses from several disease areas in which explicit eligibility criteria, regarding trial selection was reported. General characteristics and quality of reporting of all 79 meta-analyses was assessed using a validated instrument. We replicated and repeated [i.e. completing the quantitative analysis by removing those trials published in languages other than English] 18 meta-analyses, involving 211 randomized trials, using published data from the primary studies. To explore the effects of language of publication of the randomized trials on the quantitative results, we completed logistic regression analyses.
Results: Meta-analyses that included all randomized trials regardless of their language of publication, compared to language restricted meta-analysis, shared similar general characteristics. However, the former included more randomized trials and larger sample size compared to the latter. Similarly, the quality of reports of language inclusive meta-analysis [median; interquartile range: 3, 4], compared to language restricted meta-analysis [median =3; interquartile range:3.0, 5.25], was higher. Language restricted meta-analyses [n=l8], compared to language inclusive meta-analyses, did not influence the estimate of benefit of the effectiveness of an intervention [ROR = 0.94; 95%CI: 0.79, 1.13]. This result did not change whether the meta-analyses included one trial [ROR = 0.96; 95%CI: 0.78, 1.18] or more [ROR = 1.04; 95%CI: 0.74, 1.47] published in a language other than English. These results were also robust after a series of sensitivity analyses. We observed an average 65% [OR=0.35; 95%CI: 0.29, 0.41] risk reduction associated with the effectiveness of the interventions across the 211 randomized trials. However, this result was only modified by 6%, on average, regardless of the language of publication of the trials included.
Discussion: We encourage meta-analysts to identify and include all relevant randomized trials regardless of their language of publication. Although the inclusion of trials published in all languages may not influence the estimate of effectiveness of an intervention, they are likely to provide a more precise estimate of the its effectiveness. This is an important clinical and statistical attribute of a meta-analysis.