Article type
Abstract
Systematic review (SR) methods are often used as part of health technology assessments (HTA) to summarise clinical evidence. EUnetHTA provides guidance documents for SR authors carrying out HTAs. In our practical HTA work, we regularly encounter vagueness in those guidance documents; recommendations that are incomplete, unclear, or leave room for alternative methodological choices lead to substantial discussion and uncertainty. We therefore carried out a study to systematically identify needs for concretisation in current EUnetHTA guidance documents and potential solutions.
Methods:Twenty-two EUnetHTA guidance documents were screened to identify recommendations and text-passages in need for concretisation using transparent criteria. A structured group discussion with eleven experienced HTA researchers from the Austrian HTA institute was conducted to identify additional needs for concretisation. Participants ranked the importance of the needs to their HTA practice on a numerical rating scale. To identify methodological options for the practical challenges, we mapped topics with needs for concretisation in EUnetHTA guidance against core SR guidance including the Cochrane Handbook and the handbooks of IQWIG and NICE.
Results:The group identified 30 topics with needs for concretisation. Most of the topics related to evidence synthesis methods (8 topics), eligibility criteria (8 topics), risk of bias assessment (3 topics) and certainty assessment (3 topics). Other topics related to information sources, search strategy, data collection process, data items, effect measures, and reporting bias. For every identified topic, one or more methodological approaches and recommendations could be identified from the included methodological handbooks. We also identified potential gaps in the individual handbooks. In the Cochrane Handbook, we identified potential gaps related to subgroup analysis, integration of unpublished data, indirect treatment comparisons, and the role of Bayesian statistics.
Conclusions:Our analysis highlights that, from the perspective of HTA practitioners, official HTA-guidance requires further concretisation regarding certain SR elements. For many elements, harmonisation with other SR handbooks would be possible. For some elements, however, the guidance offered in the SR handbooks was rather vague, indicating that further research and collaborative guidance generation might be warranted. In the presentation, we will focus the discussion on topics in the Cochrane Handbook that may benefit from more explicit guidance.
Methods:Twenty-two EUnetHTA guidance documents were screened to identify recommendations and text-passages in need for concretisation using transparent criteria. A structured group discussion with eleven experienced HTA researchers from the Austrian HTA institute was conducted to identify additional needs for concretisation. Participants ranked the importance of the needs to their HTA practice on a numerical rating scale. To identify methodological options for the practical challenges, we mapped topics with needs for concretisation in EUnetHTA guidance against core SR guidance including the Cochrane Handbook and the handbooks of IQWIG and NICE.
Results:The group identified 30 topics with needs for concretisation. Most of the topics related to evidence synthesis methods (8 topics), eligibility criteria (8 topics), risk of bias assessment (3 topics) and certainty assessment (3 topics). Other topics related to information sources, search strategy, data collection process, data items, effect measures, and reporting bias. For every identified topic, one or more methodological approaches and recommendations could be identified from the included methodological handbooks. We also identified potential gaps in the individual handbooks. In the Cochrane Handbook, we identified potential gaps related to subgroup analysis, integration of unpublished data, indirect treatment comparisons, and the role of Bayesian statistics.
Conclusions:Our analysis highlights that, from the perspective of HTA practitioners, official HTA-guidance requires further concretisation regarding certain SR elements. For many elements, harmonisation with other SR handbooks would be possible. For some elements, however, the guidance offered in the SR handbooks was rather vague, indicating that further research and collaborative guidance generation might be warranted. In the presentation, we will focus the discussion on topics in the Cochrane Handbook that may benefit from more explicit guidance.