Background: Randomised trials use the play of chance to assign participants to comparison groups. The unpredictability of the process, if not subverted, should prevent systematic differences between comparison groups (selection bias). Differences due to chance will still occur and these are minimised by randomising a sufficiently large number of people. Objectives: To assess the effects of randomisation and concealment of allocation on the results of healthcare trials. Methods: We performed a comprehensive systematic search (September 2009). Eligible study designs were cohorts of trials, systematic reviews or meta-analyses of healthcare interventions designed to compare random allocation versus non-random allocation or adequate versus inadequate/unclear concealment of allocation in randomised trials. We assessed the risk of bias. Outcomes of interest were the magnitude and direction of estimates of effect and imbalances in prognostic factors. Results: A total of 16 studies met our inclusion criteria. All studies were at high risk of bias. When studies compared the impact of random and non-random allocation, we found larger estimates of effect in non-randomised trials in studies of the same intervention and same condition (1 study, 31 trials), we found conflicting evidence across different interventions for the same condition (2 studies, 200 trials), and in studies across different interventions and different conditions (8 studies, 486 trials + 74 meta-analyses). When studies compared adequate and inadequate/unclear concealment of allocation across different interventions and conditions, 3 studies (621 trials) found larger estimates of effect in trials with inadequate concealment of allocation, and 2 studies (376 trials) did not find statistically significant differences. Conclusions: It is generally not possible to predict the magnitude, or even the direction, of possible selection biases and consequent distortions of treatment effects from trials with non-random allocation or randomised trials with inadequate or unclear allocation concealment.