Article type
Year
Abstract
Background: From July 2011 the Institute for Quality and Efficiency in Health Care (IQWiG) will assess the additional benefit of newly-approved drugs in Germany on the basis of dossiers provided by manufacturers. This early benefit assessment includes an evaluation of the completeness of the evidence base provided.
Objectives: To assess whether the ’related articles' function and a simple structured Boolean search (SSBS) in PubMed, alone or in combination, are efficient search tools to assess the completeness of an evidence base.
Methods: Eligible citations were those included as relevant in 6 randomly-selected systematic reviews (SRs) on drug interventions published by IQWiG. Firstly, these citations were searched for in PubMed and the corresponding ’related articles' function was applied; 3 sets of articles (the first 20, 50, and all related articles) were exported. In a second step, an SSBS was performed for each SR and the articles retrieved were exported. In a third step, the number of citations screened and number of relevant citations identified were determined for each search tool. In addition, sensitivity and precision were calculated to assess efficiency.
Results: A total of 121 relevant citations were identified (4 to 55 per SR). The best sensitivity was achieved by the ’all related articles' function, but precision was very poor (100% 0.44%). The combination of the ’first 20 related articles' function and an SSBS achieved the best optimization between sensitivity and precision (97% vs. 5%). The highest precision was achieved by the ’first 20 related articles' function (13%); however, this was accompanied by a decrease in sensitivity (80%).
Conclusions: Our findings indicate that the combination of the 'first 20 related articles' function and an SSBS in PubMed is an efficient method to assess the completeness of an evidence base. Our approach will be applied and evaluated using the dossiers submitted.
Objectives: To assess whether the ’related articles' function and a simple structured Boolean search (SSBS) in PubMed, alone or in combination, are efficient search tools to assess the completeness of an evidence base.
Methods: Eligible citations were those included as relevant in 6 randomly-selected systematic reviews (SRs) on drug interventions published by IQWiG. Firstly, these citations were searched for in PubMed and the corresponding ’related articles' function was applied; 3 sets of articles (the first 20, 50, and all related articles) were exported. In a second step, an SSBS was performed for each SR and the articles retrieved were exported. In a third step, the number of citations screened and number of relevant citations identified were determined for each search tool. In addition, sensitivity and precision were calculated to assess efficiency.
Results: A total of 121 relevant citations were identified (4 to 55 per SR). The best sensitivity was achieved by the ’all related articles' function, but precision was very poor (100% 0.44%). The combination of the ’first 20 related articles' function and an SSBS achieved the best optimization between sensitivity and precision (97% vs. 5%). The highest precision was achieved by the ’first 20 related articles' function (13%); however, this was accompanied by a decrease in sensitivity (80%).
Conclusions: Our findings indicate that the combination of the 'first 20 related articles' function and an SSBS in PubMed is an efficient method to assess the completeness of an evidence base. Our approach will be applied and evaluated using the dossiers submitted.