Article type
Year
Abstract
Introduction/Objective: To develop a usable quantitative assessment tool which simplifies the analysis of economic studies, such as those estimating the cost effectiveness of prevention and cost of illness associated with sexually transmitted diseases (STDs).
Methods: A quantitative instrument was developed to evaluate economic studies of STDs using criteria similar to those outlined in the British Medical Journal (BMJ) (313:275-283, 1996). The BMJ's evaluation approach uses checklists that are qualitative tools. The new instrument made it possible to evaluate the strength or weakness of individual components on a numerical scale. The tool was pilot tested by reviewers representing multiple disciplines (medical epidemiology, economics, health services research) at the CDC's Division of STD Prevention.
Results: Various configurations of the evaluation instrument were developed. Those that based a numerical score on the presence or absence of contributing characteristics were in many cases inadequate or required several readings of the study to complete. Numerous criteria had to be evaluated, some of which were specific to economic papers and others which were common to all research. While there was a consensus that not all components should receive equal weight in determining a paper's score, there was less agreement on the relative weights each component should contribute to the overall score. Conversely, instruments containing a limited number of characteristics that reviewers used to subjectively evaluate papers were easier to complete but revealed a wider variance of scores.
Discussion: A quantitative ranking of each component of an economic evaluation in theory can enhance the objectivity of the review, but can be unwieldy and difficult to use. A qualitative checklist is easier to complete, but can lead to reviewer-dependent conclusions and can make it difficult to adequately evaluate subtle differences in quality.
Methods: A quantitative instrument was developed to evaluate economic studies of STDs using criteria similar to those outlined in the British Medical Journal (BMJ) (313:275-283, 1996). The BMJ's evaluation approach uses checklists that are qualitative tools. The new instrument made it possible to evaluate the strength or weakness of individual components on a numerical scale. The tool was pilot tested by reviewers representing multiple disciplines (medical epidemiology, economics, health services research) at the CDC's Division of STD Prevention.
Results: Various configurations of the evaluation instrument were developed. Those that based a numerical score on the presence or absence of contributing characteristics were in many cases inadequate or required several readings of the study to complete. Numerous criteria had to be evaluated, some of which were specific to economic papers and others which were common to all research. While there was a consensus that not all components should receive equal weight in determining a paper's score, there was less agreement on the relative weights each component should contribute to the overall score. Conversely, instruments containing a limited number of characteristics that reviewers used to subjectively evaluate papers were easier to complete but revealed a wider variance of scores.
Discussion: A quantitative ranking of each component of an economic evaluation in theory can enhance the objectivity of the review, but can be unwieldy and difficult to use. A qualitative checklist is easier to complete, but can lead to reviewer-dependent conclusions and can make it difficult to adequately evaluate subtle differences in quality.