A new online course on Campbell Systematic Review and Meta-Analysis Methods

Article type
Valentine J1, Littell J2, Young S3, Pigott T4, Premji Z5, Engelbert M6
1University of Louisville
2Bryn Mawr College
3Carnegie Mellon University
4Georgia State University
5University of Victoria
6International Initiative for Impact Evaluation (3ie)
Many systematic reviews are conducted by research teams with deep domain knowledge but little experience or expertise in evidence synthesis methods. One goal of the Campbell Collaboration is to provide training for the conduct of high-quality systematic reviews and meta-analyses in the social sciences. There is a need to scale up this training and to develop openly available instructional materials for online learning with a global reach.

We will describe a new online course created by members of the Campbell Training Group for authors of Campbell systematic reviews. The goal of this course is to prepare people to conduct and report rigorous and relevant systematic reviews and meta-analyses to inform policy, practice, and future research.

Course development and structure
The course follows guidelines and standards developed by Campbell and is based on empirical evidence about how to produce the most comprehensive and accurate reviews of research. Developed within the Open Learning Initiative (OLI), an open educational platform based on learning science principles, this course is designed for use in both independent (self-directed) and classroom learning environments. The course begins with an overview of the logic and methods of systematic reviews and meta-analysis. It then follows the steps in the production of systematic reviews of quantitative studies, including the following: problem formulation, searching the literature, screening potentially eligible studies, data extraction and coding, critical appraisal, effect sizes, and meta-analysis. The course includes frequent assessments and feedback tied to learning objectives at each step, providing course designers and instructors with data to drive course improvement and targeted in-class curriculum.

Conclusions and implications:
Beyond the production of Campbell reviews, we encourage wider applications of the knowledge and skills covered in this course, with a focus on systematic reviews and meta-analysis in the social sciences. With the OLI platform, instructors can request implementation of the course for a student cohort integrated with a learning management system such as Canvas, thus serving as a free and open online course textbook. The course has the potential to improve the rigor of systematic reviews and adherence to accepted standards and guidelines.

Patient, public, and/or healthcare consumer involvement: