Article type
Year
Abstract
Background: In part to address concerns about reporting bias, many journals and funders require trials to be registered prospectively. However, bias may occur if outcomes are not 'fully defined' using the five elements recommended by ClinicalTrials.gov: domain, measure, metric, method of aggregation, and time.
Objective: To compare outcomes in a usually hidden source, Clinical Study Reports (CSRs), with outcomes in public data sources.
Methods: Eligible reports described randomized trials of gabapentin for neuropathic pain or quetiapine for bipolar depression; we prespecified 5 and 8 outcome domains for gabapentin and quetiapine, respectively. We searched for reports of trials in conference proceedings, trial registers, bibliographic databases, and reference lists electronically and by hand, and obtained CSRs through unsealed litigation files. Two people performed screening and data extraction, resolving differences by discussion. For each data source, we counted the number of outcomes that were fully defined. We assessed if results could be included in a meta-analysis (i.e. reported a point estimate and measure of variability).
Results: We identified 21 eligible gabapentin trials and 7 quetiapine trials; 6/21 and 2/7 had associated CSRs, respectively. Five of 6 (83%) and 2/2 (100%) of the trials with CSRs also had associated journal articles and/or conference abstracts. CSRs included many more outcomes than journal articles and conference abstracts. CSRs did not usually provide information about additional domains compared with other sources. Instead, the additional outcomes differed in 1 or more of the other 4 elements. Almost all of the fully defined outcomes in CSRs and most in journal articles about one trial were analyzable. Few outcomes in journal articles about multiple trials and conference abstracts were analyzable.
Conclusions: Even when outcome domains are prespecified, selective outcome reporting may be related to variations in the other 4 elements. When outcomes are not fully defined a priori, the multiplicity may provide trialists and systematic reviewers with opportunities for post hoc analytic decisions and cherry picking of outcomes.
Objective: To compare outcomes in a usually hidden source, Clinical Study Reports (CSRs), with outcomes in public data sources.
Methods: Eligible reports described randomized trials of gabapentin for neuropathic pain or quetiapine for bipolar depression; we prespecified 5 and 8 outcome domains for gabapentin and quetiapine, respectively. We searched for reports of trials in conference proceedings, trial registers, bibliographic databases, and reference lists electronically and by hand, and obtained CSRs through unsealed litigation files. Two people performed screening and data extraction, resolving differences by discussion. For each data source, we counted the number of outcomes that were fully defined. We assessed if results could be included in a meta-analysis (i.e. reported a point estimate and measure of variability).
Results: We identified 21 eligible gabapentin trials and 7 quetiapine trials; 6/21 and 2/7 had associated CSRs, respectively. Five of 6 (83%) and 2/2 (100%) of the trials with CSRs also had associated journal articles and/or conference abstracts. CSRs included many more outcomes than journal articles and conference abstracts. CSRs did not usually provide information about additional domains compared with other sources. Instead, the additional outcomes differed in 1 or more of the other 4 elements. Almost all of the fully defined outcomes in CSRs and most in journal articles about one trial were analyzable. Few outcomes in journal articles about multiple trials and conference abstracts were analyzable.
Conclusions: Even when outcome domains are prespecified, selective outcome reporting may be related to variations in the other 4 elements. When outcomes are not fully defined a priori, the multiplicity may provide trialists and systematic reviewers with opportunities for post hoc analytic decisions and cherry picking of outcomes.