Article type
Year
Abstract
Background: A unique collaboration has enabled us to observe that published trial reports give little data to allow reviewers to make adequate assessments of trial quality. A systematic review on erythropoietin in cancer patients was produced by the Cochrane Haematological Malignancies Group (Cologne, Germany), published in the Cochrane Library 2004. This review was updated in collaboration with The West Midlands Health Technology Assessment Collaboration (WMHTAC) (Birmingham, England), who were required to produce a systematic review as part of a Health Technology Assessment for the National Institute for Clinical Excellence. Due to time constraints the update, did not contact authors for extra data including data that is needed to make quality assessments.
Objectives: To describe the differences in quality assessment when quality is judged solely from published sources in comparison to quality assessment judgements made after contact with authors.
Methods: A systematic review on the effectiveness of erythropoietin in cancer patients was undertaken searching the literature between 2001 and 2004 and was spliced onto a systematic review that used literature from 1985 and 2001. For consistency, quality assessment criteria were identical between the two stages of the review and JB undertook quality assessment for both reviews. Two reviewers independently assessed quality on the following criteria: randomisation, concealment of allocation, baseline similarities, blinding, losses to follow up and intention to treat analysis. The criteria were assessed as being adequate, inadequate or unclear. The only difference between the original review and the update was a process step in that the update did not contact authors for additional information or clarification.
Results: The analysis is still in progress, but it is initially quite striking how many of the trials within the update had many quality criteria elements assessed as unclear, particularly in comparison to the older trials in the original review. We intend to investigate whether contacting authors has led to any bias within the quality assessment and whether quality parameters have any impact on effectiveness assessments.
Conclusion: The difference in the process of judging trial quality, i.e. from published sources or published sources plus author contact, lead to striking differences in quality assessment.
Objectives: To describe the differences in quality assessment when quality is judged solely from published sources in comparison to quality assessment judgements made after contact with authors.
Methods: A systematic review on the effectiveness of erythropoietin in cancer patients was undertaken searching the literature between 2001 and 2004 and was spliced onto a systematic review that used literature from 1985 and 2001. For consistency, quality assessment criteria were identical between the two stages of the review and JB undertook quality assessment for both reviews. Two reviewers independently assessed quality on the following criteria: randomisation, concealment of allocation, baseline similarities, blinding, losses to follow up and intention to treat analysis. The criteria were assessed as being adequate, inadequate or unclear. The only difference between the original review and the update was a process step in that the update did not contact authors for additional information or clarification.
Results: The analysis is still in progress, but it is initially quite striking how many of the trials within the update had many quality criteria elements assessed as unclear, particularly in comparison to the older trials in the original review. We intend to investigate whether contacting authors has led to any bias within the quality assessment and whether quality parameters have any impact on effectiveness assessments.
Conclusion: The difference in the process of judging trial quality, i.e. from published sources or published sources plus author contact, lead to striking differences in quality assessment.