Do reviewers and editors detect inadequate descriptions of treatments?

Article type
Authors
Schroter S, Glasziou P, Heneghan C
Abstract
Background: To be useable in clinical practice, the treatments studied in trials and reviews must provide sufficient information to allow others to replicate the treatment (Item 4 of the CONSORT statement). Objectives: To assess (i) the quality of descriptions of treatments in randomised controlled trials published in the BMJ in 2006 and (ii) to determine the extent to which peer reviewers and editors comment on the quality of reporting of treatments during the review process. Methods: We devised a checklist of key details of a treatment sufficient for clinicians and researchers to replicate the treatment. Fifty-one consecutive RCT papers published in the BMJ were independently evaluated by two raters. Disagreements were resolved through consensus discussion. Reviewers’ comments and editors’ notes for papers were obtained and assessed for statements on the clarity of the reporting of the treatment. Results: For 57% of the papers, the treatment description in the published paper was not sufficient to allow replication. The most poorly described aspects of treatments were the actual procedures involved including the sequencing of the technique (what happened and when) and the physical or informational materials used (e.g. training materials): 53% and 43% not clear, respectively. Aspects that were better described included a description of where the treatment was delivered (6% no clear) and descriptions of what the control group received during the study (18% no clear). For a third of the treatments, the dose/duration of individual sessions were not clear and for a quarter the schedule (interval, frequency, duration, or timing) was not clear. Although the majority of the problems were not picked up by peer reviewers and editors, when they were detected about two thirds were fixed before publication. Conclusions: Journals that want to publish high quality research of use to practicing healthcare professionals need to pay more attention to adequate descriptions of treatments. The simple checklist we have developed, or modifications of it, may be useful for authors, peer reviewers, and editors and could help ensure that important details of treatments are provided before the paper is published and in the public domain.