Routine piloting in systematic reviews to improve usability: a case study

Article type
Year
Authors
Long L1, Abraham C1
1University of Exeter Medical School, UK
Abstract
Background: In 2014, Linda Long proposed a method for the routine piloting of a systematic review through to evidence synthesis stage using data from a sample of included papers in order to improve efficiency and validity of the full review.

Objectives: This paper describes and evaluates the method in a systematic review of low intensity interventions to prevent sexually transmitted infections in young people and men who have sex with men (MSM).

Methods: Seven databases were searched up to October 2014; 23 'young people' RCTs and 10 MSM RCTs were identified for inclusion in the review. A sample of five young people studies was piloted through to evidence synthesis, undergoing data extraction and quality appraisal using the Cochrane ‘Risk of bias’ tool. 'Summary of findings' (SoF) tables describing participants, intervention delivery and study outcomes were created. Following the mini-synthesis, the data extraction form and SoF tables were circulated to team members in all participating research institutions. Feedback was sought on their usefulness to inform the next stage of the project and amendments invited. MSM papers were not subject to the piloting method and were processed as usual.

Results: Following the mini-synthesis, a number of criteria in the data extraction form and SoF tables were identified as needing modification. After amendments, relevant data from the remaining young people RCTs were efficiently extracted in one phase. All 10 MSM RCTs had data extracted and their quality appraised. However, after modification of the data extraction form and SoF tables, a second phase of data extraction from all 10 studies had to be performed.

Conclusions: Routine piloting in this systematic review facilitated a 'bespoke' review, with time saved through efficient data extraction. In addition, the mini synthesis provided a potential version of the full review that could be discussed and agreed by all stakeholders at an early stage of the review process. This supported review project management, improved efficiency, and ensured optimal usability by researchers involved in the next stage of the research program.