Systematic reviews of the effectiveness of quality improvement strategies and programmes: Some methodological issues

Pantoja T1, Mayhew A2, Grimshaw J2
1Departamento Medicina Familiar, Pontificia Universidad Católica de Chile, Chile; Cochrane Effective Practice and Organisation of Care Group, Canada, 2Cochrane Effective Practice and Organisation of Care Group, Canada

Background: Well conducted systematic reviews are increasingly seen as providing the best evidence to guide choice of quality improvement strategies in healthcare. But in contrast with reviews of pharmacological interventions, a number of methodological challenges should be addressed by reviewers undertaking reviews in this area.

Objective: To discuss methodological issues relevant to the conduct of reviews of quality improvement strategies and programmes.

Methods: We built on our experiences in the Cochrane Effective Practice and Organisation of Care (EPOC) Group and on a focused review of the methodological literature in the area.

Results: We have selected the following issues for a further analysis: i) Formulating a relevant question and inclusion criteria: study designs, populations and outcomes; the use of non-randomised evidence for effectiveness questions; the lack of an accepted taxonomy for quality improvement strategies; and the broadness/narrowness of the review question (the 'lumping versus splitting’ debate). ii) Identifying and screening evidence sources: published versus nonpublished literature; search strategy considerations. iii) Methods of synthesis/analysis: appropriateness of meta-analysis; other synthesis approaches (range of effect sizes); dealing with expected heterogeneity; identifying effect modifiers.

Conclusions: We have discussed issues that authors will face when conducting reviews of the effectiveness of quality improvement strategies. Additionally we have presented some of the current methodological developments in order to contribute to more sound and informative reviews in this field.