Article type
Year
Abstract
(Please note Part 1)
Background: this poster refers to the submitted abstract “Rapid, rapid, rapid…how reliable is a single screener? A crowd-based, randomized methods study (Part 1)”, which presents a randomized methods study to assess the validity of single-reviewer compared with dual-reviewer abstract screening. A crucial step in any systematic review is the screening of abstracts. To reduce the risk of excluding relevant studies, international methodological guidance recommends that at least two investigators independently screen each abstract. By contrast, the Cochrane Handbook for Systematic Reviews of Interventions 6.0 explicitly states that it is “acceptable that initial screening of titles and abstracts is undertaken by one person”.
To date, however, the evidence base about the magnitude of bias that single-reviewer screening introduces compared with dual-reviewer screening is scarce.
Objectives: the objective of this poster is to summarize briefly results of our methods study and to acknowledge participants of the crowd screening who play a crucial role in this methods project.
Methods: we will use the Cochrane Crowd platform to recruit participants for our study. Two published systematic reviews, one on a pharmacological topic, one on a public health topic will serve as reference standards. Cochrane Crowd randomly assigns participants to 100 randomly selected abstracts. At least 10 participants will screen each abstract. The included or excluded studies of the two systematic reviews serve as the reference standards. The primary outcome of our study is the proportion of correct decisions of single screeners compared to the decisions of two screeners. We will calculate the average proportion of correct and incorrect decisions (i.e. sensitivities and specificities) over all raters with the corresponding 95% confidence intervals.
Results: results will be available by the time of the Cochrane Colloquium. Because our study depends on the voluntary work of participants of Cochrane Crowd, we would like to acknowledge their important contributions in this poster by presenting their names and the countries they come from. Participants of Cochrane Crowd anonymously donate time to support Cochrane and the advancement of methods of systematic reviews. This poster should be a token of appreciation to their commitment and their important contributions.
Conclusions: we cannot draw conclusions yet. Results will provide important information about the trade-off in validity that the more rapid single-reviewer screening has compared with the more time-intensive dual-reviewer screening.
Patient or healthcare consumer involvement: rapid reviews generally involve patients and healthcare providers in the review process in order to focus on patient-relevant health outcomes. For well-informed decision making, rapid reviews present the synthesized evidence conclusively in a simple and concise way.
Background: this poster refers to the submitted abstract “Rapid, rapid, rapid…how reliable is a single screener? A crowd-based, randomized methods study (Part 1)”, which presents a randomized methods study to assess the validity of single-reviewer compared with dual-reviewer abstract screening. A crucial step in any systematic review is the screening of abstracts. To reduce the risk of excluding relevant studies, international methodological guidance recommends that at least two investigators independently screen each abstract. By contrast, the Cochrane Handbook for Systematic Reviews of Interventions 6.0 explicitly states that it is “acceptable that initial screening of titles and abstracts is undertaken by one person”.
To date, however, the evidence base about the magnitude of bias that single-reviewer screening introduces compared with dual-reviewer screening is scarce.
Objectives: the objective of this poster is to summarize briefly results of our methods study and to acknowledge participants of the crowd screening who play a crucial role in this methods project.
Methods: we will use the Cochrane Crowd platform to recruit participants for our study. Two published systematic reviews, one on a pharmacological topic, one on a public health topic will serve as reference standards. Cochrane Crowd randomly assigns participants to 100 randomly selected abstracts. At least 10 participants will screen each abstract. The included or excluded studies of the two systematic reviews serve as the reference standards. The primary outcome of our study is the proportion of correct decisions of single screeners compared to the decisions of two screeners. We will calculate the average proportion of correct and incorrect decisions (i.e. sensitivities and specificities) over all raters with the corresponding 95% confidence intervals.
Results: results will be available by the time of the Cochrane Colloquium. Because our study depends on the voluntary work of participants of Cochrane Crowd, we would like to acknowledge their important contributions in this poster by presenting their names and the countries they come from. Participants of Cochrane Crowd anonymously donate time to support Cochrane and the advancement of methods of systematic reviews. This poster should be a token of appreciation to their commitment and their important contributions.
Conclusions: we cannot draw conclusions yet. Results will provide important information about the trade-off in validity that the more rapid single-reviewer screening has compared with the more time-intensive dual-reviewer screening.
Patient or healthcare consumer involvement: rapid reviews generally involve patients and healthcare providers in the review process in order to focus on patient-relevant health outcomes. For well-informed decision making, rapid reviews present the synthesized evidence conclusively in a simple and concise way.
PDF