Searching for and including non-randomised trials in Cochrane reviews: is it worth the effort?

Article type
Authors
Glenton C1, Scheel I2, Lewin S3, Swingler G4, Munabi-Babigumira S5, Johansen M5
1Department of Global Health and Welfare, SINTEF,Oslo, Norway
2Department of Global Health and Welfare, SINTEF,Oslo, Norway;
3Medical Research Council of South Africa, Cape Town, South Africa
4School of Child and Adolescent Health, Red Cross War Memorial Children s Hospital, Cape Town, South Africa
5Norwegian Knowledge Centre for the Health Services, Oslo, Norway
Abstract
Background: In a recent Cochrane review we evaluated the effectiveness of lay health workers for maternal and child health. For some areas, including vaccination, we identified few randomised trials (RCTs) from low- and middle-income countries (LMICs), limiting the applicability of this evidence for these settings. Furthermore, it is often suggested that substantial evidence from non-randomised studies is available and should be included in effect reviews. We therefore carried out a second review where non-randomised studies were included. Objective: To assess whether including non-randomised studies impacted on the evidence obtained and its usefulness for decision making. Method: We compared the number of studies found in each review and the risk of bias of the RCTs and non-randomised studies. We assessed whether the inclusion of non-randomised studies would have changed our conclusions regarding the intervention effects. Results: In addition to the seven RCTs included in the Cochrane review, we included an additional two controlled before-after studies (CBAs), both from LMICs, and one interrupted time series study from the USA. One of the CBAs did not include sufficient data to easily assess outcomes. For the other two studies, the results were similar overall to those found in the Cochrane review, generally indicating that lay health workers improved the extent to which immunisation was up to date, compared to usual services. The non-randomised studies had a higher risk of bias than the RCTs. Conclusions: Despite including non-randomised studies, only three such studies were found. These studies contributed to our understanding of the range of interventions that might be implemented. However, their findings did not provide significant additional evidence regarding the effectiveness of lay health workers for vaccination and did not change the Cochrane review conclusions. For this question at least, substantial additional evidence on effectiveness was not found from non-randomised studies.