Article type
Year
Abstract
Background: Publication bias (preferentially publishing positive findings) is well recognized in the translation of evidence from research into the primary literature and is of concern because it may lead to incorrect conclusions about effectiveness of interventions. It is possible that publication bias also exists in the translation of evidence from the primary literature to the secondary literature (evidence based summaries of the primary literature) with the same effect.
Objectives: Primary: To investigate the existence of publication bias in the secondary literature (using ACP Journal Club as an example of the secondary literature and the Medline database as representative of the primary literature) summarizing randomized controlled trials (RCT's) of therapy. Secondary: To identify other selection biases in ACP Journal Club and identify trends in the quality of reported/summarized RCT's over time.
Methods: A cross sectional survey of randomly selected RCT's of therapy published between 1994 and 2002 in English in the Medline database and all summaries of therapy RCT's published by ACP Journal Club between the same dates. The main outcome measure was the rate of positive trials from among trials aiming to find a difference between groups. Also studied were: aim, sample size, blinding, intervention types, sites, disease field.
Results: 831 trials from the Medline database and 823 summaries of trials from ACP Journal Club met the inclusion criteria. Compared to trials catalogued in the Medline database, ACP Journal Club preferentially summarized trials with a positive outcome (p <.001). This bias remained after controlling for other selection biases seen in ACP Journal Club such as preferentially summarizing trials that were multi-centered, had large sample size, no active treatment control, blinding and in certain disease fields (Adjusted Odds Ratio 2.8, 95% Confidence Interval 2.02-3.93). There has been no change in trial characteristics over time in the Medline database. ACP Journal Club, however, has increased the prevalence of blinding, no active treatment controls and negative trials.
Conclusions: ACP Journal Club preferentially summarizes RCT's with positive outcomes, possibly leading clinicians to overestimate the effectiveness of therapy. Efforts should be made to reduce this bias and to search for similar bias in other secondary literature.
Objectives: Primary: To investigate the existence of publication bias in the secondary literature (using ACP Journal Club as an example of the secondary literature and the Medline database as representative of the primary literature) summarizing randomized controlled trials (RCT's) of therapy. Secondary: To identify other selection biases in ACP Journal Club and identify trends in the quality of reported/summarized RCT's over time.
Methods: A cross sectional survey of randomly selected RCT's of therapy published between 1994 and 2002 in English in the Medline database and all summaries of therapy RCT's published by ACP Journal Club between the same dates. The main outcome measure was the rate of positive trials from among trials aiming to find a difference between groups. Also studied were: aim, sample size, blinding, intervention types, sites, disease field.
Results: 831 trials from the Medline database and 823 summaries of trials from ACP Journal Club met the inclusion criteria. Compared to trials catalogued in the Medline database, ACP Journal Club preferentially summarized trials with a positive outcome (p <.001). This bias remained after controlling for other selection biases seen in ACP Journal Club such as preferentially summarizing trials that were multi-centered, had large sample size, no active treatment control, blinding and in certain disease fields (Adjusted Odds Ratio 2.8, 95% Confidence Interval 2.02-3.93). There has been no change in trial characteristics over time in the Medline database. ACP Journal Club, however, has increased the prevalence of blinding, no active treatment controls and negative trials.
Conclusions: ACP Journal Club preferentially summarizes RCT's with positive outcomes, possibly leading clinicians to overestimate the effectiveness of therapy. Efforts should be made to reduce this bias and to search for similar bias in other secondary literature.
PDF