Article type
Year
Abstract
Background: The results of Cochrane systematic reviews are being used increasingly, both in clinical practice and by deciders. Many users of these reviews will interpret results expressed as odds ratios as being relative risks. This was recently observed in a project (involving health policy deciders) in France, where the results expressed as an odds ratio, when interpreted as a relative risk suggested a 40% reduction in mortality, and when expressed as a relative risk, the reduction was only 20%. Hence it was decided to investigate which summary statistics are presented in Cochrane systematic reviews.
Methods: The summary statistic used to expressed results for binary outcomes in completed systematic reviews in the Cochane Database of Systematic Reviews (Cochrane Library 2000, issue 2) was recorded. This information was obtained from the abstract when possible, or from the results or methods sections.
Results: Among the 79 new systematic reviews published in this issue of the Cochrane Library, in about one-third only continuous data were analysed, or no quantitative analyses were reported. In some of these reviews the results were only presented in words, although data were available in MetaView, thus, the reader was given no indication as to which summary statistic should be used. About one-quarter of the reviews expressed results for binary outcomes as odds ratios, and about one-third expressed them as relative risks. The remains reviews expressed results as rate differences or hazard ratios. Discussion: Even amongst the recently completed reviews the use of the odds ratio summary statistic is high, although the difference with the relative risk has not yet been investigated. Many authors of reviews gave potential users of the reviews no indication which summary statistic should be used. In the Cochrane Handbook the following advice is given: 'For dichotomous (or binary) data there are pros and cons for each of the summary statistics available: odds ratio, relative risk (or relative effect), and risk. Although the odds ratio has been commonly used in meta-analysis, there are concerns about the potential for it to be interpreted incorrectly. The odds ratio has statistical advantages relating to its sampling distribution and its suitability for modelling, but these advantages are not always important in meta-analyses. Moreover, there is no solid basis for assuming that the odds ratio is more constant across studies than other summary statistics. The odds ratio and the relative risk are similar if the outcome is relatively rare.' In addition, the default method in Review Manager is Peto odds ratio.
Conclusions: A clearer policy is needed to help authors of Cochrane systematic reviews to select the appropriate summary statistic. In addition, a standard clear explanation of how to interpret odds ratios could be developed and authors of reviews using odds ratios could be encouraged to include this in their review.
Methods: The summary statistic used to expressed results for binary outcomes in completed systematic reviews in the Cochane Database of Systematic Reviews (Cochrane Library 2000, issue 2) was recorded. This information was obtained from the abstract when possible, or from the results or methods sections.
Results: Among the 79 new systematic reviews published in this issue of the Cochrane Library, in about one-third only continuous data were analysed, or no quantitative analyses were reported. In some of these reviews the results were only presented in words, although data were available in MetaView, thus, the reader was given no indication as to which summary statistic should be used. About one-quarter of the reviews expressed results for binary outcomes as odds ratios, and about one-third expressed them as relative risks. The remains reviews expressed results as rate differences or hazard ratios. Discussion: Even amongst the recently completed reviews the use of the odds ratio summary statistic is high, although the difference with the relative risk has not yet been investigated. Many authors of reviews gave potential users of the reviews no indication which summary statistic should be used. In the Cochrane Handbook the following advice is given: 'For dichotomous (or binary) data there are pros and cons for each of the summary statistics available: odds ratio, relative risk (or relative effect), and risk. Although the odds ratio has been commonly used in meta-analysis, there are concerns about the potential for it to be interpreted incorrectly. The odds ratio has statistical advantages relating to its sampling distribution and its suitability for modelling, but these advantages are not always important in meta-analyses. Moreover, there is no solid basis for assuming that the odds ratio is more constant across studies than other summary statistics. The odds ratio and the relative risk are similar if the outcome is relatively rare.' In addition, the default method in Review Manager is Peto odds ratio.
Conclusions: A clearer policy is needed to help authors of Cochrane systematic reviews to select the appropriate summary statistic. In addition, a standard clear explanation of how to interpret odds ratios could be developed and authors of reviews using odds ratios could be encouraged to include this in their review.