Article type
Year
Abstract
Background: Although the assessment of harms is almost always included as an outcome in intervention and medical test studies, the manner of capturing, and reporting harms is significantly different than the outcomes of benefit. There is a need to consider criteria for separate assessment of validity specific to harms.
Objectives: To develop criteria for assessing internal validity of collecting and reporting harms and assess the reliability and validity of the items.
Methods: A systematic review of the literature to identify biases and items within existing quality assessment tools was undertaken; criteria related to validity were extracted. A Delphi Consensus exercise with 6 experts was undertaken to reduce the items. Following consensus, the criteria within the tool were pilot testing for phrasing and a help file was developed. Test-retest reliability was undertaken. Construct, and discriminant validity testing was undertaken.
Results: From 5549 citations, 101 full text articles were screened and from this 70 studies had at least one item. A total of 415 items were generated from the review; after elimination of redundant items, 126 criteria were included in the Delphi consensus. Following two rounds of consensus and piloting, 15 criteria were included in the McHarm. Reliability testing showed acceptable levels (internal consistency all studies = 0.82; 95% CI 0.79 to 0.85, and intra-class correlation = 0.95, 95% CI 0.91 to 0.98). Construct validity with the Chou and Helfand scale was 0.66; discriminant validity with the Jadad scale was 0.22. Reliability and validity varied between surgical and pharmacological studies suggesting a potential influence by drug type.
Conclusions: The McHarm contains 15 items related to assessing the internal validity of capturing and reporting harms in intervention studies and was shown to be reliable and valid.
Objectives: To develop criteria for assessing internal validity of collecting and reporting harms and assess the reliability and validity of the items.
Methods: A systematic review of the literature to identify biases and items within existing quality assessment tools was undertaken; criteria related to validity were extracted. A Delphi Consensus exercise with 6 experts was undertaken to reduce the items. Following consensus, the criteria within the tool were pilot testing for phrasing and a help file was developed. Test-retest reliability was undertaken. Construct, and discriminant validity testing was undertaken.
Results: From 5549 citations, 101 full text articles were screened and from this 70 studies had at least one item. A total of 415 items were generated from the review; after elimination of redundant items, 126 criteria were included in the Delphi consensus. Following two rounds of consensus and piloting, 15 criteria were included in the McHarm. Reliability testing showed acceptable levels (internal consistency all studies = 0.82; 95% CI 0.79 to 0.85, and intra-class correlation = 0.95, 95% CI 0.91 to 0.98). Construct validity with the Chou and Helfand scale was 0.66; discriminant validity with the Jadad scale was 0.22. Reliability and validity varied between surgical and pharmacological studies suggesting a potential influence by drug type.
Conclusions: The McHarm contains 15 items related to assessing the internal validity of capturing and reporting harms in intervention studies and was shown to be reliable and valid.