Background: In general, systematic reviews focus on estimates of relative effect when presenting binary outcomes. Ratings of confidence in estimates (quality of evidence) therefore reflect confidence in relative effect estimates. Treatment decisions, however, must be based on trade-off in absolute effects, often calculated by applying relative risk to estimates of control group risk.

Objectives: To explore how uncertainty in control group risk influences confidence in estimates of effect in the context of evidence summaries prepared for clinical practice guidelines.

Methods: We surveyed the evidence summaries prepared for the ninth iteration of the American College of Chest Physicians Antithrombotic Guidelines looking specifically at sources of estimates of control group risk and possible sources of uncertainty in control group risk.

Results: We found numerous instances when uncertainties in estimates of control group risk potentially compromised confidence in estimates of absolute treatment effect. Examples included the following. Risk of bias in estimates of symptomatic thrombosis risk arose in randomized trials that used screening venography to diagnose asymptomatic thrombosis, and in observational studies that failed to report antithrombotic prophylaxis. Risk of bias also increased in observational studies that used billing codes to determine thrombotic outcome diagnoses. Small sample size in observational studies resulted in imprecision of estimates of control group risk. Reliance on estimates from randomized trials raised issues of directness or applicability to clinical populations. Inconsistency in results from observational studies sometimes undermined confidence in estimates of control group risk.

Conclusions: Limitations in the quality of evidence regarding estimates of control group risk is an under-appreciated source of decreased confidence in estimates of absolute treatment effects in evidence summaries from systematic reviews.