Article type
Year
Abstract
Background: Non-randomized studies (NRS) are increasingly used as evidence for effectiveness of interventions in Cochrane Reviews. How and why NRS are included has not been evaluated to date.
Objectives: to conduct an overview of practice of including NRS within the CC.
Methods: We included all Cochrane Reviews that included one or more NRS. We conducted study selection and data collection in duplicate. We interpreted and categorized, via an iterative process, the reasons for including NRS, the way Risk of Bias was assessed and how this was influenced by other review characteristics.
Results: We included 202 reviews. All but one review included RCTs as well as NRS. NRS types that were specified were quasi-RCTs (45%), CCT (55%), controlled-before after studies (60%), interrupted time-series (52%), cohort-studies (37%), case-control studies (26%) and various other. Fifty percent of the reviews did not cite a reason for including NRS. For those that did, reasons could be divided into: RCTs are wanted but not available and RCTs are not needed because NRS provide good evidence. Unavailability was underpinned with: lack of RCTs based on a pilot search, randomization not feasible because of complex intervention, setting or aggregate level and randomization not ethical because of a vulnerable population. The claim that RCTs are not needed was supported by: restriction to RCTs is too dogmatic, NRS maximize the available data, NRS are needed to compare with RCTs and infrequent and long-term outcomes can only be studied in NRS. Risk of bias was based on EPOC-criteria in 38% the reviews. The rest used a variety of checklists and self-constructed tools.
Conclusions: Most reviews don’t specify reasons for including NRS. Where they do, there is a wide variation in reasons that are not always supported by valid arguments. Risk of bias assessment of NRS also varies and needs urgent improvement.
Objectives: to conduct an overview of practice of including NRS within the CC.
Methods: We included all Cochrane Reviews that included one or more NRS. We conducted study selection and data collection in duplicate. We interpreted and categorized, via an iterative process, the reasons for including NRS, the way Risk of Bias was assessed and how this was influenced by other review characteristics.
Results: We included 202 reviews. All but one review included RCTs as well as NRS. NRS types that were specified were quasi-RCTs (45%), CCT (55%), controlled-before after studies (60%), interrupted time-series (52%), cohort-studies (37%), case-control studies (26%) and various other. Fifty percent of the reviews did not cite a reason for including NRS. For those that did, reasons could be divided into: RCTs are wanted but not available and RCTs are not needed because NRS provide good evidence. Unavailability was underpinned with: lack of RCTs based on a pilot search, randomization not feasible because of complex intervention, setting or aggregate level and randomization not ethical because of a vulnerable population. The claim that RCTs are not needed was supported by: restriction to RCTs is too dogmatic, NRS maximize the available data, NRS are needed to compare with RCTs and infrequent and long-term outcomes can only be studied in NRS. Risk of bias was based on EPOC-criteria in 38% the reviews. The rest used a variety of checklists and self-constructed tools.
Conclusions: Most reviews don’t specify reasons for including NRS. Where they do, there is a wide variation in reasons that are not always supported by valid arguments. Risk of bias assessment of NRS also varies and needs urgent improvement.