Article type
Year
Abstract
Introduction: Most reports of randomized controlled trials contain inadequate methodological descriptions.
Objective: To explore the association between inadequate methodological descriptions and estimates of treatment effects.
Methods: We analyzed 250 trials from 33 meta-analyses encompassing 62,091 participants. We assessed the methodological quality of the trials on major dimensions, and then analyzed the associations between those assessments and estimated treatment effects, using multiple logistic regression models.
Results: Compared with trials in which authors reported adequately concealed treatment allocation, trials in which authors reported inadequately or unclearly concealed allocation yielded larger estimates of treatment effects (p<0.00l). On average, odds ratios were inflated by 41% or 33%, respectively. Trials in which participants were excluded after randomization were not, however, associated with larger treatment effects. Lack of double-blinding in trials was also associated with larger treatment effects (p=0.01) with odds ratios being inflated by 17%, on average.
Discussion: Inadequate methodological approaches in controlled trials, particularly those reflecting inadequate or unclear allocation concealment, were associated with reported larger treatment effects. Those associations likely represent bias. Analysts conducting systematic reviews need to account for allocation concealment and double-binding in assessing quality and in developing exclusion criteria.
Objective: To explore the association between inadequate methodological descriptions and estimates of treatment effects.
Methods: We analyzed 250 trials from 33 meta-analyses encompassing 62,091 participants. We assessed the methodological quality of the trials on major dimensions, and then analyzed the associations between those assessments and estimated treatment effects, using multiple logistic regression models.
Results: Compared with trials in which authors reported adequately concealed treatment allocation, trials in which authors reported inadequately or unclearly concealed allocation yielded larger estimates of treatment effects (p<0.00l). On average, odds ratios were inflated by 41% or 33%, respectively. Trials in which participants were excluded after randomization were not, however, associated with larger treatment effects. Lack of double-blinding in trials was also associated with larger treatment effects (p=0.01) with odds ratios being inflated by 17%, on average.
Discussion: Inadequate methodological approaches in controlled trials, particularly those reflecting inadequate or unclear allocation concealment, were associated with reported larger treatment effects. Those associations likely represent bias. Analysts conducting systematic reviews need to account for allocation concealment and double-binding in assessing quality and in developing exclusion criteria.