Article type
Year
Abstract
Background: Reanalyses of randomized controlled trial (RCT) data may help the scientific community assess the validity of reported trial results, but it may also cause confusion.
Objectives: To identify reanalyses of RCT data; characterize methodological and other differences between the original trial and re-analysis; evaluate the independence of authors performing the re-analyses; and compare re-analysis findings to those from the original trial to assess whether the re-analysis modified interpretations from the original paper about the kinds or number of patients who should be treated.
Methods: We completed an electronic search of MEDLINE from inception to 9 March 2014 to identify all published studies that completed a re-analysis of individual patient data from previously published RCTs addressing the same hypothesis as the original RCT. Four data extractors independently screened papers and extracted data. We assessed for changes in direction and magnitude of treatment effect, statistical significance, and interpretation about who should be treated.
Results: We identified 37 eligible reanalyses in 36 published papers, five of which were performed by entirely independent authors, (two based on publicly available data, two on data that were provided on request, the availability of data was not clear for one). Reanalyses differed most commonly in statistical or analytical approaches (n = 18) and in definitions or measurements of the outcome of interest (n = 12). Four reanalyses changed the direction and two changed the magnitude of treatment effect, while four reanalyses led to changes in statistical significance of findings. Thirteen reanalyses (35%) led to interpretations that differed from those of the original paper: three (8%) could be interpreted as showing that different patients should be treated, one (3%) that fewer patients should be treated, and nine (24%) that more patients should be treated.
Conclusions: A small number of reanalyses of RCTs have been published to-date and original study conclusions were altered in 35%.
Objectives: To identify reanalyses of RCT data; characterize methodological and other differences between the original trial and re-analysis; evaluate the independence of authors performing the re-analyses; and compare re-analysis findings to those from the original trial to assess whether the re-analysis modified interpretations from the original paper about the kinds or number of patients who should be treated.
Methods: We completed an electronic search of MEDLINE from inception to 9 March 2014 to identify all published studies that completed a re-analysis of individual patient data from previously published RCTs addressing the same hypothesis as the original RCT. Four data extractors independently screened papers and extracted data. We assessed for changes in direction and magnitude of treatment effect, statistical significance, and interpretation about who should be treated.
Results: We identified 37 eligible reanalyses in 36 published papers, five of which were performed by entirely independent authors, (two based on publicly available data, two on data that were provided on request, the availability of data was not clear for one). Reanalyses differed most commonly in statistical or analytical approaches (n = 18) and in definitions or measurements of the outcome of interest (n = 12). Four reanalyses changed the direction and two changed the magnitude of treatment effect, while four reanalyses led to changes in statistical significance of findings. Thirteen reanalyses (35%) led to interpretations that differed from those of the original paper: three (8%) could be interpreted as showing that different patients should be treated, one (3%) that fewer patients should be treated, and nine (24%) that more patients should be treated.
Conclusions: A small number of reanalyses of RCTs have been published to-date and original study conclusions were altered in 35%.