SAMURAI—a new statistical program for sensitivity analyses using unpublished but registered studies

Article type
Authors
Gartlehner G1, Kim NY2, Bangdiwala SI2, Thaler K1
1Danube University, Austria
2University of North Carolina - Chapel Hill, USA
Abstract
Background/Objectives: The non-release of clinical trial results potentially contributes to publication bias of meta-analyses. Our goals were to use information found in clinical trial registries on the number of and the enrollment of unpublished studies to gauge the sensitivity of a meta-analytic summary to the non-publication of studies and to develop a user-friendly open-source software to facilitate such sensitivity analysis.

Methods/Results: Our software, the R package SAMURAI, can handle meta-analytic data sets of clinical trials with two independent treatment arms. The outcome of interest can be binary or continuous. For each unpublished study, the data set only requires the sample sizes of each treatment arm and the user predicted ‘outlook’ for the studies. Outlooks are chosen by the user from among pre-defined outlooks that vary from those strongly favoring the intervention treatment to those strongly favoring the control treatment. SAMURAI assumes that control arms of unpublished studies have effects similar to the effect across control arms of published studies. For each intervention arm of an unpublished study, utilizing the user-provided outlook, SAMURAI randomly generates an effect estimate using a probability distribution, which may be based on a summary effect across published trials. SAMURAI then calculates the estimated summary intervention effect using a random effects model using the DerSimonian-Laird method, and outputs the results as forest plots.

Conclusion: By utilizing information about sample sizes of treatment groups in registered but unpublished clinical trials, SAMURAI has an advantage over other assessments of publication bias, such as the trim and fillmethod, which come withmore stringent assumptions about the number and enrollment of unpublished studies. The forest plot provides the end-user an easy way to see how the inclusion of unpublished studies could change the meta-analytic summary intervention effect.