When methodologies conflict: developing tools to compare and select appropriate methodologies for conducting, and appraising evidence from systematic reviews

Tags: Poster
Kuusi T1, Knittle K1
1University of Helsinki


Multiple guidelines, recommendations, and meta-scientific evidence apply at various phases of systematic reviews and meta-analyses: from planning and registering, to analysing and reporting data, and even to appraising evidence. These frameworks are not unanimous in their recommendations or approaches, resulting in a confusing situation for researchers and practitioners who rely on systematic reviews as a gold standard of evidence. One topical example of this is the discussion around the effectiveness of antidepressant drugs for acute depression. Formal tools are therefore needed to help interested parties contrast and select the ideal methodological approaches to conducting and interpreting systematic reviews.


The first aim of this study is to develop a graphical guide for systematic review authors to choose which guidelines and approaches to use from a plethora of (sometimes disagreeing) options. The second aim is to develop a tool for contrasting methodological arguments behind disagreeing interpretations of the quality of evidence in systematic reviews and meta-analyses, in order to inform conclusions about effectiveness and subsequent decisions about implementation.


A literature search identified guidelines, critical appraisal checklists, bias correcting methods, and other scientific literature related to methodology in systematic reviews and meta-analyses. We develop methods to display this information logically to potential end users, including review authors, students, guideline makers and a wider audience.


We present a graphical guide of available review methodologies and guidelines, and a critical appraisal contrasting tool, which consists of the best arguments and counterarguments arising from each approach to evidence appraisal.


The tools in this study can help those conducting systematic reviews to select the appropriate methodological guidelines, and those interpreting the results of systematic reviews to understand better whether the available evidence supports implementation of an intervention, or not.

Patient or healthcare consumer involvement:

We asked for feedback from members of the general audience to maximize understandability.