Background: Methodological deficiencies are known to affect the results of randomized controlled trials (RCTs). Differences in the risk of bias may indicate that the effect estimates of some trials are more biased than others.

Objectives: This work concentrates on reviews within emergency and critical care medicine and estimates the extent of bias on mortality. RCTs in patients with acute and partially rare disease are typically small and heterogeneous because of organizational, ethical and judicial problems. High risk of bias can result from long periods of recruitment with changing standard treatment or problems in funding. Risk of bias was assessed using the Cochrane Collaboration’s tool and six additional indication-specific quality measures: early stopping, pre-intervention, conflict of interest, baseline imbalances, cross-over trials, and insufficient length of follow-up.

Methods: Subgroup analyses, logistic regression models, and fixed and random effect models were used to investigate whether quality measures explain any of the heterogeneity of odds ratios in 73 randomized trials from eight meta-analyses of Cochrane reviews of patients with sepsis, or septic or cardiogenic shock.

Results: The percentage of trials with low risk of bias varied between 23% in early stopping and 81% in addressing incomplete outcome data. Overall, odds ratios were underestimated in trials with inadequate allocation concealment, selective reporting, or if a contra-active or similar supporting pre-randomization intervention could have influenced the effect of the randomized intervention or cross-over could not be ruled out. Odds ratios were overestimated in RCTs with incomplete reporting of mortality. Full data will be presented at the colloquium.

Conclusions: Investigated sources of bias can lead to exaggerated or reduced estimates of the intervention effect in this medical area. These associations will support the assessment of risk of bias in Cochrane reviews on interventions in intensive care and emergency medicine.