Article type
Abstract
Background: Cochrane review authors are often unable to obtain all the necessary trial information from available reports, especially when reports are conference abstracts. The Cochrane Handbook recommends review authors contact trial investigators to obtain this information. It is not known whether this correspondence is helpful to the review process, and how it impacts on trial information captured by the review, such as risk of bias.
Objectives: To determine how risk-of-bias assessments change after contact with trial investigators.
Methods: This was a substudy undertaken during creation of the review 'Endometrial scratching for pregnancy following sexual intercourse or intrauterine insemination (IUI)' which included 10 RCTs. Attempts were made to contact trial authors by email to clarify methods relating to risk-of-bias assessments. Investigators were emailed with open-ended questions such as 'Please describe in detail the process of allocating participants to the trial arms'.
Results: An initial response was received from 8/10 trial teams. A total of 15 changes to risk-of-bias assessments were made (15/70, 21%). The majority of the changes were from unclear to low risk (10/15). The domain of 'allocation concealment' had the most changes (5). In a number of instances the information provided by the trial team conflicted with information in the published papers, or online trial registrations. For example, differences in the total number of women randomised, or whether participant blinding occurred. This correspondence also enabled discovery of additional and unanticipated risks of bias, such as the undisclosed inclusion of non-randomised participants in the trial denominators. It remains unclear whether trust is best placed in information provided in the published (often peer-reviewed) reports, or in subsequent author correspondence.
Conclusions: Correspondence with trial authors resulted in a large number of changes to risk-of-bias assessments. Review authors must exercise their judgement when amending risk-of-bias assessments based on this correspondence, and clearly report the sources of information in the ‘support for judgement’.
Objectives: To determine how risk-of-bias assessments change after contact with trial investigators.
Methods: This was a substudy undertaken during creation of the review 'Endometrial scratching for pregnancy following sexual intercourse or intrauterine insemination (IUI)' which included 10 RCTs. Attempts were made to contact trial authors by email to clarify methods relating to risk-of-bias assessments. Investigators were emailed with open-ended questions such as 'Please describe in detail the process of allocating participants to the trial arms'.
Results: An initial response was received from 8/10 trial teams. A total of 15 changes to risk-of-bias assessments were made (15/70, 21%). The majority of the changes were from unclear to low risk (10/15). The domain of 'allocation concealment' had the most changes (5). In a number of instances the information provided by the trial team conflicted with information in the published papers, or online trial registrations. For example, differences in the total number of women randomised, or whether participant blinding occurred. This correspondence also enabled discovery of additional and unanticipated risks of bias, such as the undisclosed inclusion of non-randomised participants in the trial denominators. It remains unclear whether trust is best placed in information provided in the published (often peer-reviewed) reports, or in subsequent author correspondence.
Conclusions: Correspondence with trial authors resulted in a large number of changes to risk-of-bias assessments. Review authors must exercise their judgement when amending risk-of-bias assessments based on this correspondence, and clearly report the sources of information in the ‘support for judgement’.