Article type
Year
Abstract
Background: Very few validated quality appraisal tools exit for case series studies (CSs).
Objectives: To evaluate the construct validity of a 20-criterion checklist that has been developed using a modified Delphi approach.
Methods: Two health technology assessment (HTA) researchers randomly selected 105 CSs of various topics from a broad literature search. Six HTA researchers from Canada, Australia, and Spain used the checklist to assess these studies; each researcher assessed 35 studies, and two researchers in pair assessed the same seven studies. An experienced biostatistician conducted a factor analysis to examine the factor structure and to inform potential refinements of the item pool of the checklist.
Results: Preliminary results of the factor analysis revealed a trend of a separation of 20 items into two components: (1) Ten items on the presence of the traditional features of the execution of a statistical hypothesis-testing paradigm; (2) Seven items on the descriptions of the subjects’ characteristics that might feature in the experimental design, particularly in judgments about the likelihood of confounding. The other 3 items (i.e. multi-center study, consecutive recruitment, and reporting of competing interest and sources of support) do not correlate very highly with either of the two components. The analysis provided no basis for any cutoff scores by which study quality might be distinguished.
Conclusions: The checklist should be tailored to meet the need of different projects, taking into account the relevant importance of hypothesis-testing versus description of subject/intervention characteristics. The set of items for hypothesis testing may be more critical for some conditions (e.g., type 1 diabetes) where causal relationship between the intervention (e.g., islet transplantation) and efficacy/effectiveness outcomes (e.g., insulin independence) can be established from a before-after case series study. The set of items describing study/intervention characteristics may be more important when assessing clinical outcomes such as long-term adverse effects.
Objectives: To evaluate the construct validity of a 20-criterion checklist that has been developed using a modified Delphi approach.
Methods: Two health technology assessment (HTA) researchers randomly selected 105 CSs of various topics from a broad literature search. Six HTA researchers from Canada, Australia, and Spain used the checklist to assess these studies; each researcher assessed 35 studies, and two researchers in pair assessed the same seven studies. An experienced biostatistician conducted a factor analysis to examine the factor structure and to inform potential refinements of the item pool of the checklist.
Results: Preliminary results of the factor analysis revealed a trend of a separation of 20 items into two components: (1) Ten items on the presence of the traditional features of the execution of a statistical hypothesis-testing paradigm; (2) Seven items on the descriptions of the subjects’ characteristics that might feature in the experimental design, particularly in judgments about the likelihood of confounding. The other 3 items (i.e. multi-center study, consecutive recruitment, and reporting of competing interest and sources of support) do not correlate very highly with either of the two components. The analysis provided no basis for any cutoff scores by which study quality might be distinguished.
Conclusions: The checklist should be tailored to meet the need of different projects, taking into account the relevant importance of hypothesis-testing versus description of subject/intervention characteristics. The set of items for hypothesis testing may be more critical for some conditions (e.g., type 1 diabetes) where causal relationship between the intervention (e.g., islet transplantation) and efficacy/effectiveness outcomes (e.g., insulin independence) can be established from a before-after case series study. The set of items describing study/intervention characteristics may be more important when assessing clinical outcomes such as long-term adverse effects.