Article type
Year
Abstract
Background: The number of systematic reviews (SRs) and health technology assessments based on a systematic assessment of primary research literature is constantly growing. About four SRs on medical topics meeting minimum standards were published per day in 2003. Therefore, against a background of restricted resources, it seems questionable to conduct new SRs if summarizing results from existing systematic secondary literature (SSL) could provide a similar level of certainty.
Objectives: On the basis of existing literature on the methodology and quality of SRs, we suggest a typology of five quantifiable dimensions to provide a reasonable basis to decide whether a new SR would be likely to add substantial information to the existing body of evidence.
Methods: The suggested decision matrix is constructed from the following items and combined with a search update to identify the most recent literature: (1) the number of existing SSL on a specific topic (including the number of primary studies considered and patients enrolled); (2) the quality of existing SSL (including the disclosure of conflicts of interests of authors); (3) the comprehensiveness of search strategies; (4) the unexplained heterogeneity between results of existing SSL; (5) the consistency of results of SSL with results of primary studies identified by the search update.
Results: The decision matrix provides 17 possible constellations: In three of these constellations, the evidence from existing SSL is reliable enough to form a basis for decision-making: (1) There is sufficient and high-quality SSL with comprehensive search strategies and consistent results, which are confirmed by a search update. (2) There is sufficient and high-quality SSL; even though search strategies are not comprehensive, results are consistent and confirmed by a search update. (3) There is sufficient and high-quality SSL with comprehensive search strategies; however, all SSL concludes that there is not enough evidence from primary studies to answer the research question posed, and the search update does not identify relevant new studies. In all other constellations, a new SR is likely to add relevant information to the existing body of evidence.
Conclusions: As the number of high-quality SSL is rapidly growing, it will be challenging for HTA agencies to resort to this type of evidence. We specified three conclusive constellations in which the current best evidence may be synthesized from existing SSL instead of preparing resource-intensive new SRs. The feasibility of this methodological approach will be tested in a series of planned SRs on the use of positron emission tomography in various clinical settings.
Objectives: On the basis of existing literature on the methodology and quality of SRs, we suggest a typology of five quantifiable dimensions to provide a reasonable basis to decide whether a new SR would be likely to add substantial information to the existing body of evidence.
Methods: The suggested decision matrix is constructed from the following items and combined with a search update to identify the most recent literature: (1) the number of existing SSL on a specific topic (including the number of primary studies considered and patients enrolled); (2) the quality of existing SSL (including the disclosure of conflicts of interests of authors); (3) the comprehensiveness of search strategies; (4) the unexplained heterogeneity between results of existing SSL; (5) the consistency of results of SSL with results of primary studies identified by the search update.
Results: The decision matrix provides 17 possible constellations: In three of these constellations, the evidence from existing SSL is reliable enough to form a basis for decision-making: (1) There is sufficient and high-quality SSL with comprehensive search strategies and consistent results, which are confirmed by a search update. (2) There is sufficient and high-quality SSL; even though search strategies are not comprehensive, results are consistent and confirmed by a search update. (3) There is sufficient and high-quality SSL with comprehensive search strategies; however, all SSL concludes that there is not enough evidence from primary studies to answer the research question posed, and the search update does not identify relevant new studies. In all other constellations, a new SR is likely to add relevant information to the existing body of evidence.
Conclusions: As the number of high-quality SSL is rapidly growing, it will be challenging for HTA agencies to resort to this type of evidence. We specified three conclusive constellations in which the current best evidence may be synthesized from existing SSL instead of preparing resource-intensive new SRs. The feasibility of this methodological approach will be tested in a series of planned SRs on the use of positron emission tomography in various clinical settings.