Article type
Year
Abstract
Background:
The credibility of evidence syntheses can be compromised by reporting biases. These include 'publication bias', when the probability that a study is published depends on its findings, and 'outcome reporting bias', when findings that are statistically non-significant are less likely to be reported. Reporting biases imply bias in evidence syntheses because the missing results differ systematically from the available results. Existing tools for assessing the risk of reporting biases in evidence syntheses are limited in terms of their scope, guidance for reaching risk of bias judgements and measurement properties.
Objectives:
To introduce ROB-ME, a comprehensive new tool for assessing the risk of reporting biases in evidence syntheses, and to provide participants with the opportunity to apply ROB-ME.
Description:
The workshop will be split into two parts.
1) Introduction to ROB-ME:
We will provide a brief overview of the development and structure of ROB-ME. Key components of the tool include:
- preliminary consideration of which outcomes were pre-specified (e.g. in trial registries, protocols) in each study identified, and whether results are fully available, partially available or not available for outcomes of interest;
- specification of a particular synthesis to be assessed for risk of bias;
- use of signalling questions to inform risk of bias judgements (such as how many studies are definitely missing from the synthesis because results are not available or are partially available, and whether other studies are likely to be missing because of non-comprehensive searching);
- algorithms to map responses to signalling questions to judgements about risk of bias.
2) Applying ROB-ME:
Participants will apply ROB-ME to an example review in small groups. Each group will assess the example review for a particular component of the ROB-ME tool. In a keynote session we will discuss the results of each group's assessment and any issues that arose during the assessment process.
The workshop will conclude with a facilitated, structured discussion focusing on the implications of using the tool alongside other risk of bias tools (e.g. RoB 2.0), and further development needs for guidance and software.
The credibility of evidence syntheses can be compromised by reporting biases. These include 'publication bias', when the probability that a study is published depends on its findings, and 'outcome reporting bias', when findings that are statistically non-significant are less likely to be reported. Reporting biases imply bias in evidence syntheses because the missing results differ systematically from the available results. Existing tools for assessing the risk of reporting biases in evidence syntheses are limited in terms of their scope, guidance for reaching risk of bias judgements and measurement properties.
Objectives:
To introduce ROB-ME, a comprehensive new tool for assessing the risk of reporting biases in evidence syntheses, and to provide participants with the opportunity to apply ROB-ME.
Description:
The workshop will be split into two parts.
1) Introduction to ROB-ME:
We will provide a brief overview of the development and structure of ROB-ME. Key components of the tool include:
- preliminary consideration of which outcomes were pre-specified (e.g. in trial registries, protocols) in each study identified, and whether results are fully available, partially available or not available for outcomes of interest;
- specification of a particular synthesis to be assessed for risk of bias;
- use of signalling questions to inform risk of bias judgements (such as how many studies are definitely missing from the synthesis because results are not available or are partially available, and whether other studies are likely to be missing because of non-comprehensive searching);
- algorithms to map responses to signalling questions to judgements about risk of bias.
2) Applying ROB-ME:
Participants will apply ROB-ME to an example review in small groups. Each group will assess the example review for a particular component of the ROB-ME tool. In a keynote session we will discuss the results of each group's assessment and any issues that arose during the assessment process.
The workshop will conclude with a facilitated, structured discussion focusing on the implications of using the tool alongside other risk of bias tools (e.g. RoB 2.0), and further development needs for guidance and software.