Introduction: Effect size obtained from a meta-analysis has been used to classify the level of evidence available for the use of a health care intervention. However, a given effect size obtained from a meta-analysis of only small randomized trials would have lower weight of evidence and hence be less reliable than that obtained from a meta-analysis of large (with small) randomised trials. There is a need to develop methods to assess the reliability of an effect size from a meta-analysis according to the weight of evidence.

Methods: We used log-likelihood ratio as measure of `weight of evidence' expressed in support units (su). Log-likelihood ratio of 'one' was taken as one support unit. Support units for point-estimates were calculated against the null-hypothesis for 69 meta-analyses of randomised trials of antiplatelet therapy in different class of patients for different end-points published by Anti-platelet Trialists' Collaboration. The meta-analyses were classified into three groups: highly reliable (more than 10 su), reliable (five to 10 su) and unreliable (less than five su).

Results: Support unit scores for the meta-analyses ranged from 0.01 to 92.29 (mean 15.45). 30 meta-analyses scored more than 10 support units, seven scored between 5 and 10 and 32 scored less than five support units. On comparison with the collaborators' reliance on the evidence, these categories had good correspondence.

Discussion: We have developed a method to assess the reliability of a meta-analysis based on the weight of evidence. The method will allow us to determine the robustness of conclusions from a systematic review. More experience with the application of the method will help define its utility.