Article type
Year
Abstract
Background: Journal editors and reviewers are often concerned that results of systematic reviews may be out-of-date, and thereby provide misleading information to decision makers. Searches conducted a year or more previously can result in a journal rejecting a review, or in a requirement that searches are updated. However, as review methods become increasingly complex, synthesis requires additional time placing many reviews at risk of rejection by publishers.
Objective: To consider whether the focus on rapid publication might hinder production of the most rigorous or ambitious reviews.
Method: We focused on recent developments in review designs and methods to consider whether current journal guidelines may be at odds with newer review approaches intended to increase review quality.
Results: A variety of factors may improve the quality of systematic reviews whilst also increasing the time needed to undertake them. Several recommended review processes increase the number of steps required in a review. These include: methods to enhance rigor and transparency in presentation of review findings such as GRADE and CerQUAL; methods to enhance transparency relating to intervention components such as the TiDIER checklist; and efforts to enhance patient and public involvement (PPI) in the post-search phases of reviews. Several factors increase the volume of literature being synthesized. As the extent of available research literature continues to grow exponentially this can result in larger numbers of studies to screen for inclusion in each review. Similarly, the scope of many reviews is broader than the PICO approach in which a single intervention, outcome and population are specified. EPPI-Centre's work with policy-makers has shown that policy concerns often require reviews that cover a range of potentially useful interventions to enable consideration of where best to invest resources. Other factors relate to the extent of analysis required to ensure utility for decision-makers. Review approaches designed to understand the implications of intervention and contextual complexity, such as mixed-method research synthesis or realist synthesis, often involve multiple synthesis stages. A related point is that these reviews often require a high level of conceptual development and interpretation which also increases the time required for synthesis. Hi-tech solutions for searching and screening may reduce the time burden for parts of the review process in the near future, but this is unlikely to significantly impact on reviews that require iterative searching or high levels of interpretation.
Conclusion: Publishers' focus on timely publication of reviews poses a risk that research teams will favour questions that are easier to review quickly and avoid questions that require a broader scope or deeper analysis, ultimately leading to production of reviews that are less useful to decision-makers.
Patient or healthcare consumer involvement: PPI is highly relevant. To avoid tokenism ample time is needed to engage in PPI at various stages of a review.
Objective: To consider whether the focus on rapid publication might hinder production of the most rigorous or ambitious reviews.
Method: We focused on recent developments in review designs and methods to consider whether current journal guidelines may be at odds with newer review approaches intended to increase review quality.
Results: A variety of factors may improve the quality of systematic reviews whilst also increasing the time needed to undertake them. Several recommended review processes increase the number of steps required in a review. These include: methods to enhance rigor and transparency in presentation of review findings such as GRADE and CerQUAL; methods to enhance transparency relating to intervention components such as the TiDIER checklist; and efforts to enhance patient and public involvement (PPI) in the post-search phases of reviews. Several factors increase the volume of literature being synthesized. As the extent of available research literature continues to grow exponentially this can result in larger numbers of studies to screen for inclusion in each review. Similarly, the scope of many reviews is broader than the PICO approach in which a single intervention, outcome and population are specified. EPPI-Centre's work with policy-makers has shown that policy concerns often require reviews that cover a range of potentially useful interventions to enable consideration of where best to invest resources. Other factors relate to the extent of analysis required to ensure utility for decision-makers. Review approaches designed to understand the implications of intervention and contextual complexity, such as mixed-method research synthesis or realist synthesis, often involve multiple synthesis stages. A related point is that these reviews often require a high level of conceptual development and interpretation which also increases the time required for synthesis. Hi-tech solutions for searching and screening may reduce the time burden for parts of the review process in the near future, but this is unlikely to significantly impact on reviews that require iterative searching or high levels of interpretation.
Conclusion: Publishers' focus on timely publication of reviews poses a risk that research teams will favour questions that are easier to review quickly and avoid questions that require a broader scope or deeper analysis, ultimately leading to production of reviews that are less useful to decision-makers.
Patient or healthcare consumer involvement: PPI is highly relevant. To avoid tokenism ample time is needed to engage in PPI at various stages of a review.