Use of the Cochrane 'Risk of bias' tool in systematic reviews of traditional Chinese medicine

Tags: Poster
Wang L1, Fei Y1, Zhang K1, Feng S1, Liang N1, Yang G1, Liu J1
1Centre for Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine, China

Background: The Cochrane 'Risk of bias' (RoB) tool, which was published in the Cochrane Handbook for Systematic Reviews of Interventions in 2008, has been widely embraced by the systematic review (SR) community to assess the methodological quality of randomized controlled trials (RCTs) in SRs.

Objectives: To evaluate the use of the RoB tool in Cochrane SRs of traditional Chinese medicine (TCM).

Methods: We searched for intervention SRs on TCM published in the Cochrane Library from January 2009 to April 2015. Tool adopted to assess methodological quality of the RCTs were abstracted and analyzed by reporting quality of each item and by time sequent.

Results: We identified 83 SRs of TCM, which included 1143 RCTs or quasi-RCTs and 124,800 participants. In the Methods section of the SRs the following tools were used to assess the methodological quality of the included studies: the Cochrane RoB tool (71/83), other criteria (4/83; three in 2010, one in 2013) included Jadad etc., self-defined criteria (3/83; two in 2009, one in 2012), and the remaining five SRs (two in 2009, three in 2012-2015; four were 'empty' reviews) did not mention quality assessment. For the 71 SRs that used the Cochrane RoB tool, nine SRs were 'empty' reviews. In the remaining 62 SRs, 75.81% (47/62) reported all the items in the RoB table, 24.19% (15/62) (9 in 2009-2011, 6 in 2012-2014) failed to report all items: other bias (11 SRs), selective reporting (five SRs), incomplete outcome data (two SRs), blinding (one SR), and sequence generation (one SR) were often omitted. Criteria for judging 'other bias' were not reported in 37 SRs.

Conclusions: Most (85.54%) SRs used the RoB table, but 24.19% of SRs did not report/adopt all the items. Some (8.43%) SRs used other criteria or self-defined criteria, as their protocols were published before 2009. Other (6.02%) SRs did not report information about methodological quality evaluation; in fact, it should be defined in the Methods section of SR protocols whether or not the SRs include trials. SRs published in 2009-2011 need to be updated in their methods of assessing methodological quality of included trials.