Possible bias presenting the results of systematic reviews: An example and a suggested solution.

Article type
Year
Authors
Hyde C
Abstract
Introduction: The forest plot or "blobbogram" is an extremely powerful tool for presenting the results of the included studies in a systematic review, and is rightly a key component of Cochrane Reviews. However, its ability to convey large amounts of information succinctly, in a way which allows many readers to rapidly draw their own conclusions, also carries dangers. When mistakes or inaccuracies are introduced they are quickly transmitted to readers, who may not have the time or skills to spot these themselves. The author illustrates one situation in which summary odds ratio diagrams can seriously misrepresent the results of included studies in a systematic review.

Objectives:

Methods: Case study of a systematic review performed by the author in association with others (I Robert & A Sinclair) on trials of the effects of supporting discharge from hospital to home in older persons. The nature of the problem arose as part of a detailed reconsideration of the review in response to anonymous referees' comments following submission to a major journal.

Results: One of the key outcomes in the review was the effect on "institutionalisation" i.e. whether a trial participant was admitted to long stay care. Seven of the nine included studies in the review provided information on this outcome. Three studies presented results as point event rates, three as period event rates and four as mean duration of presence in long-stay care. Although sought, individual patient data was not available to allow results to be expressed in a common format. Thus relying on a forest plot to convey the results of included studies in the Cochrane format would have meant that only approximately half the available information about me effect of this intervention on institutionalisation could be conveyed on one diagram, with no means to indicate on this diagram that further valid information on me outcome was available.

Discussion: When viewing a summary odds ratio diagram it is easy to assume that any included studies not represented do not furnish information on the outcome in question. In the case of our review this was not the case and we suspect that this is not the only systematic review where the forest plot has misrepresented the results of all the included studies. The author suggests that each summary odds ratio diagram should have a foot-note indicating the reasons why any included studies do not appear on the figure.