Article type
Year
Abstract
Background: Meta-analysis (MA) in Cochrane systematic reviews (SR) of interventions is used to synthesise the effects of interventions from multiple studies to answer healthcare research questions. They can improve precision of effect estimates, investigate factors that can improve or reduce the treatment effect, and allow researchers to answer questions that they might be unable to answer from individual studies. However, these benefits are at risk if the methods of MA are not applied correctly.
Objectives: The aims of this study were a) to assess the application and interpretation of MA methods in newly published Cochrane reviews in 2019 to inform Cochrane guidance and training and b) to compare our findings to similar work carried out in 2014 to see if reporting had improved.
Methods: We evaluated all Cochrane SR published in 2019. Two authors independently extracted data on primary outcome, statistical methods and methods used to assess reporting bias. We adapted methods employed previously by Page and colleagues comparing Cochrane and non-Cochrane SRs (Page 2018).
Results: We have evaluated 35 SRs (March 2020). SRs included a median of 8 studies (IQR: 4-13), and 77% (n=27) of SRs performed a MA. The median number of included participants was 1591 (IQR: 192-1329); the median number of MAs performed was 6 (IQR: 2.5-14). In SRs with a MA, 63% (n=17) of the MAs of the primary outcome was evaluated with a random-effects model, but only justified by the authors in 35% (n=6) of MA. A funnel plot was constructed in 19% of SRs with a MA; of these, the recommended minimum number of studies (n=10) was seen in all funnel plots. The proportion of SRs with an MA and at least 1 subgroup planned or performed was 74% (n=20), while the median number of subgroups actually performed 0 (IQR: 0-2).
Conclusions: In these early results, we found that Cochrane reviews have largely remained unchanged in their reporting of various methods over the past 5 years. As previously found in 2014, approximately 60% of MAs of primary outcomes used a RE model, and 65% of those MAs did not state rationale for its use. Fewer funnel plots are being produced than in 2014 (19% vs 41%), though a larger proportion of those produced have the recommended minimum number of studies included (100% vs 24%). Development of Cochrane guidance and encouragement for review teams to involve statisticians at the planning and drafting stages should continue to be encouraged and is a focal point of the Cochrane Methods Support Unit.
Patient or healthcare consumer involvement: n/a
Objectives: The aims of this study were a) to assess the application and interpretation of MA methods in newly published Cochrane reviews in 2019 to inform Cochrane guidance and training and b) to compare our findings to similar work carried out in 2014 to see if reporting had improved.
Methods: We evaluated all Cochrane SR published in 2019. Two authors independently extracted data on primary outcome, statistical methods and methods used to assess reporting bias. We adapted methods employed previously by Page and colleagues comparing Cochrane and non-Cochrane SRs (Page 2018).
Results: We have evaluated 35 SRs (March 2020). SRs included a median of 8 studies (IQR: 4-13), and 77% (n=27) of SRs performed a MA. The median number of included participants was 1591 (IQR: 192-1329); the median number of MAs performed was 6 (IQR: 2.5-14). In SRs with a MA, 63% (n=17) of the MAs of the primary outcome was evaluated with a random-effects model, but only justified by the authors in 35% (n=6) of MA. A funnel plot was constructed in 19% of SRs with a MA; of these, the recommended minimum number of studies (n=10) was seen in all funnel plots. The proportion of SRs with an MA and at least 1 subgroup planned or performed was 74% (n=20), while the median number of subgroups actually performed 0 (IQR: 0-2).
Conclusions: In these early results, we found that Cochrane reviews have largely remained unchanged in their reporting of various methods over the past 5 years. As previously found in 2014, approximately 60% of MAs of primary outcomes used a RE model, and 65% of those MAs did not state rationale for its use. Fewer funnel plots are being produced than in 2014 (19% vs 41%), though a larger proportion of those produced have the recommended minimum number of studies included (100% vs 24%). Development of Cochrane guidance and encouragement for review teams to involve statisticians at the planning and drafting stages should continue to be encouraged and is a focal point of the Cochrane Methods Support Unit.
Patient or healthcare consumer involvement: n/a