Article type
Abstract
Background
There is evidence that around 50% of all studies remain unpublished and the resulting bias due to missing evidence is observed to impact on various research fields and topics. Approaches for assessing bias due to missing evidence are advancing as reflected in the Cochrane Handbook and, partially, the GRADE guidance. It remains unclear to what degree the systematic non-dissemination of research is reflected in Cochrane reviews. This study wants to find out about the current assessments of the GRADE domain “publication bias” and subsequent decisions about GRADEing.
Objectives
Our main aims are:
1.To estimate how frequent rating down for the GRADE domain “publication bias” occurs;
2.To generate an overview of the assessments of “publication bias”;
3.To obtain an understanding about reasons for downgrading the certainty of the evidence for “publication bias”.
Methods
This is a cross-sectional study. We screened all Cochrane effectiveness reviews published between January 2021 and October 2023. With the help of text mining and automated data extraction, two researchers selected all reviews that downgraded at least one outcome for publication bias. We collected data in a tailored and piloted MS Excel sheet covering intervention characteristics, number of included studies and relevant outcome characteristics (e.g., n included studies, effect estimate, methods to assess reporting biases, and the authors’ rating of publication bias, reasons for rating down and the final certainty of evidence). Selected data was extracted automatically, other data was extracted by one reviewer and cross-checked by a second reviewer. We used descriptive statistical analyses for quantitative data and a deductive qualitative approach for analysing qualitative data.
Results
Characteristics of our sample are reported in table 1. Most included studies were RCTs. We are currently evaluating a) characteristics of the included reviews and b) characteristics of individual outcomes. We will present a comprehensive overview of our findings during the Global Evidence Summit.
Conclusions
We will draw and present conclusions based on our findings.
Relevance and importance to patients
Our findings potentially inform the production of robust guidance for decision-making.
Patient and public involvement
Not applicable.
There is evidence that around 50% of all studies remain unpublished and the resulting bias due to missing evidence is observed to impact on various research fields and topics. Approaches for assessing bias due to missing evidence are advancing as reflected in the Cochrane Handbook and, partially, the GRADE guidance. It remains unclear to what degree the systematic non-dissemination of research is reflected in Cochrane reviews. This study wants to find out about the current assessments of the GRADE domain “publication bias” and subsequent decisions about GRADEing.
Objectives
Our main aims are:
1.To estimate how frequent rating down for the GRADE domain “publication bias” occurs;
2.To generate an overview of the assessments of “publication bias”;
3.To obtain an understanding about reasons for downgrading the certainty of the evidence for “publication bias”.
Methods
This is a cross-sectional study. We screened all Cochrane effectiveness reviews published between January 2021 and October 2023. With the help of text mining and automated data extraction, two researchers selected all reviews that downgraded at least one outcome for publication bias. We collected data in a tailored and piloted MS Excel sheet covering intervention characteristics, number of included studies and relevant outcome characteristics (e.g., n included studies, effect estimate, methods to assess reporting biases, and the authors’ rating of publication bias, reasons for rating down and the final certainty of evidence). Selected data was extracted automatically, other data was extracted by one reviewer and cross-checked by a second reviewer. We used descriptive statistical analyses for quantitative data and a deductive qualitative approach for analysing qualitative data.
Results
Characteristics of our sample are reported in table 1. Most included studies were RCTs. We are currently evaluating a) characteristics of the included reviews and b) characteristics of individual outcomes. We will present a comprehensive overview of our findings during the Global Evidence Summit.
Conclusions
We will draw and present conclusions based on our findings.
Relevance and importance to patients
Our findings potentially inform the production of robust guidance for decision-making.
Patient and public involvement
Not applicable.