Article type
Year
Abstract
Background: Several empirical studies suggest that the quality of reporting of clinical trials can affect the magnitude of the treatment effect estimate, with poor quality reporting associated with larger biases of treatment effectiveness. It is reasonable to expect a similar effect for systematic reviews (SRs). Previous methodological research has produced conflicting results regarding the association between the validity of SRs and statistical results, however previous studies did not quantify the magnitude of the effect of quality on the final outcome. We plan to determine the relationship between the quality of reporting of SRs and the magnitude and precision of their point estimates.
Objectives: 1) To describe SRs on the same clinical question with respect to concordance/discordance 2) To explore the relationship between quality of SR reporting and magnitude, direction, and precision of point estimates 3) To identify which quality components are most important in influencing the magnitude of the point estimate
Methods: A set of published SRs was identified where at least two of them examined the same clinical question. SRs were included if they were reviews of randomized controlled trials, were published in English, and involved a meta-analysis. Quality was assessed using the Oxman and Guyatt s validated Overview Quality Assessment Questionnaire . In addition, we will extract information on other components that are not addressed in the Oxman and Guyatt Index (e.g., funding source, assessment of publication bias, inclusion of grey literature, etc.), as well as effect estimates and their precision. Differences in treatment effect estimates between SRs in the same content area with varying quality will be examined. A meta-meta-analytic approach will be used to assess the effect of overall quality and various quality components on the treatment effect estimate.
Results: We have identified 121 SRs in 35 clinical topic areas. The reviews have been quality assessed using the Oxman and Guyatt Index and data extraction is in progress. The final results will be available at the Cochrane Colloquium in Ottawa in October 2004.
Conclusions: This study will provide an important first step towards understanding the association between quality of reporting and estimates of effectiveness (and precision) at the SR level. Such knowledge can decrease bias during the performance of SRs, improve their reporting, and ultimately increase their usefulness to clinical medicine.
Objectives: 1) To describe SRs on the same clinical question with respect to concordance/discordance 2) To explore the relationship between quality of SR reporting and magnitude, direction, and precision of point estimates 3) To identify which quality components are most important in influencing the magnitude of the point estimate
Methods: A set of published SRs was identified where at least two of them examined the same clinical question. SRs were included if they were reviews of randomized controlled trials, were published in English, and involved a meta-analysis. Quality was assessed using the Oxman and Guyatt s validated Overview Quality Assessment Questionnaire . In addition, we will extract information on other components that are not addressed in the Oxman and Guyatt Index (e.g., funding source, assessment of publication bias, inclusion of grey literature, etc.), as well as effect estimates and their precision. Differences in treatment effect estimates between SRs in the same content area with varying quality will be examined. A meta-meta-analytic approach will be used to assess the effect of overall quality and various quality components on the treatment effect estimate.
Results: We have identified 121 SRs in 35 clinical topic areas. The reviews have been quality assessed using the Oxman and Guyatt Index and data extraction is in progress. The final results will be available at the Cochrane Colloquium in Ottawa in October 2004.
Conclusions: This study will provide an important first step towards understanding the association between quality of reporting and estimates of effectiveness (and precision) at the SR level. Such knowledge can decrease bias during the performance of SRs, improve their reporting, and ultimately increase their usefulness to clinical medicine.