Article type
Year
Abstract
Background: Biased results from poorly designed and reported RCTs can mislead decision making. Whether publications accurately reflect the actual methodological quality of randomized controlled trials (RCTs) has not been comprehensively evaluated. Additionally, it has been hypothesized that trial sample size is associated with better methodological quality.
Objective: To compare the reported methodological quality of a RCT as reflected in publications with actual methodological quality as depicted in the protocols, and assess association of RCT sample size with published versus actual methodological quality.
Methods: All consecutive published phase III RCTs conducted by 8 National Cancer Institute sponsored Cooperative Groups (NCICOG) until year 2006 were eligible for inclusion. Data on methodological quality domains relevant to minimizing bias and random error were extracted from protocols and publications for each study.
Results: A total of 429 RCTs met the inclusion criteria. Overall reporting of elements of methodological quality domains relevant to minimizing bias (Figure 1A) and random error (Figure 1B) was poor and did not reflect the actual high methodological quality of RCTs. Similarly, the results showed a positive correlation between RCT sample size and reported methodological quality domains of adequacy of blinding procedures, ITT analysis and the choice of the comparator (pvalue<0.001). However, this correlation disappeared when actual methodological quality was considered.
Conclusion: This largest study to-date comparing published versus actual methodological quality of RCTs shows that poor quality of reporting by NCICOGs does not reflect the actual high methodological quality. Similarly, we found no association between RCT sample size and its actual methodological quality. The positive correlation between methodological quality of reporting and RCT sample size is misleading. Our findings underline the need for adherence to the CONSORT statement by authors and journal editors for transparent evaluation of RCTs and highlight the importance of publication of RCT protocols in the public domain.
Objective: To compare the reported methodological quality of a RCT as reflected in publications with actual methodological quality as depicted in the protocols, and assess association of RCT sample size with published versus actual methodological quality.
Methods: All consecutive published phase III RCTs conducted by 8 National Cancer Institute sponsored Cooperative Groups (NCICOG) until year 2006 were eligible for inclusion. Data on methodological quality domains relevant to minimizing bias and random error were extracted from protocols and publications for each study.
Results: A total of 429 RCTs met the inclusion criteria. Overall reporting of elements of methodological quality domains relevant to minimizing bias (Figure 1A) and random error (Figure 1B) was poor and did not reflect the actual high methodological quality of RCTs. Similarly, the results showed a positive correlation between RCT sample size and reported methodological quality domains of adequacy of blinding procedures, ITT analysis and the choice of the comparator (pvalue<0.001). However, this correlation disappeared when actual methodological quality was considered.
Conclusion: This largest study to-date comparing published versus actual methodological quality of RCTs shows that poor quality of reporting by NCICOGs does not reflect the actual high methodological quality. Similarly, we found no association between RCT sample size and its actual methodological quality. The positive correlation between methodological quality of reporting and RCT sample size is misleading. Our findings underline the need for adherence to the CONSORT statement by authors and journal editors for transparent evaluation of RCTs and highlight the importance of publication of RCT protocols in the public domain.
Images
