Background:

Relative Risk (RR) is a measure of association used for binary outcomes. It is preferred by clinicians because of its easy interpretability therefore the Drug and Alcohol Group suggests the reviewers to adopt it. However, the results from meta-analyses (MA) using RR can vary if the event chosen is switched.

Objectives:

This study assesses how often the results of MAs using RR vary if the event or the reverse event is considered as outcome.

Methods:

All binary data MAs using RR performed by the CDAG published in the Cochrane Library 1.2005 were considered. This group has decided to use RR as summary statistic for their SRs. MAs were included in the study whether: a) binary data were used;
a) estimate of combined RRs; b) its p-values; c) test of heterogeneity; d) p-value of the test of heterogeneity and e) I2 value.
The events were switched and the results combined using the criteria stated by the authors of the MAs in the methods of review section and the same information stated in points a-e was calculated. The agreement between the results obtained considering the event and the switched event was estimated by using the k-statistic measure.

Results:

15 out of 23 SRs met the inclusion criteria involving 95 comparisons. Table1 below shows the agreement between the results obtained. The values of the k-statistic measures were 0.59 for the test of treatment effect, 0.57 for the test of heterogeneity and 0.39 for the I2.

Conclusions:

The decision about which summary statistic to consider in MAs is often very controversial. RR is easy to interpret and it is preferred by clinicians. Our study has shown that the conclusions from MAs using RR can vary depending on the outcome adopted by the authors of MAs. More methodological research is needed to a priori identify which summary statistics to use. These criteria should depend not only on ease of interpretation but also on: a) the mathematical properties of summary statistics; b) the understanding of the reasons for variation in control group event rates.