Article type
Abstract
BACKGROUND:
A priority exercise by Cochrane First Aid revealed that the effectiveness of first aid training for laypeople is a high-priority uncertainty among their stakeholders. While conducting a systematic review (SR) addressing this educational question, we were confronted with unanticipated outcome multiplicity for which no solutions were specified at the protocol stage.
OBJECTIVES:
To describe our approach to dealing with outcome multiplicity to enable synthesis using meta-analysis in our SR.
METHODS:
The 36 included (cluster-)randomized controlled trials reported on more than 190 outcome measures across 4 educational outcome domains, that is, first aid–related knowledge, skills, self-efficacy, and willingness to help. When examining the available data for synthesis, multiplicity arose in 2 ways: (1) multiple outcome measures were used for a single outcome domain within a single study and (2) single outcomes were measured at multiple time points after the intervention. In search of a practical solution for this issue of multiplicity, we reviewed Cochrane guidance; consulted the Cochrane Methods Support Unit, first aid experts, and the literature; and had interactive discussions among the core review team.
RESULTS:
We grouped individual outcome measures within the 4 educational outcome domains and categorized them as measured in the short (<1 month), intermediate (1-3 months), or long term (>3 months) after the intervention. In case multiple outcome measures within a study and outcome domain were available for inclusion in a synthesis, we applied the "median effect estimate" approach, ranking the effect estimates of the outcomes within the outcome domain and selecting the outcome with the median value as a representative effect to include in the meta-analysis. In case an outcome was measured multiple times within the same time frame (eg, short-term), the earliest time point was chosen. Applying this solution allowed us to select a single outcome measure per study and outcome domain to include in our meta-analyses.
CONCLUSIONS:
Outcome multiplicity is a common challenge in SRs with broad outcome domains. Our SR provides a workable approach that enables meaningful data synthesis to inform stakeholders. Although anticipating outcome multiplicity may be challenging, review teams may benefit from mapping outcomes through a scoping exercise and prespecifying solutions at the protocol stage.
A priority exercise by Cochrane First Aid revealed that the effectiveness of first aid training for laypeople is a high-priority uncertainty among their stakeholders. While conducting a systematic review (SR) addressing this educational question, we were confronted with unanticipated outcome multiplicity for which no solutions were specified at the protocol stage.
OBJECTIVES:
To describe our approach to dealing with outcome multiplicity to enable synthesis using meta-analysis in our SR.
METHODS:
The 36 included (cluster-)randomized controlled trials reported on more than 190 outcome measures across 4 educational outcome domains, that is, first aid–related knowledge, skills, self-efficacy, and willingness to help. When examining the available data for synthesis, multiplicity arose in 2 ways: (1) multiple outcome measures were used for a single outcome domain within a single study and (2) single outcomes were measured at multiple time points after the intervention. In search of a practical solution for this issue of multiplicity, we reviewed Cochrane guidance; consulted the Cochrane Methods Support Unit, first aid experts, and the literature; and had interactive discussions among the core review team.
RESULTS:
We grouped individual outcome measures within the 4 educational outcome domains and categorized them as measured in the short (<1 month), intermediate (1-3 months), or long term (>3 months) after the intervention. In case multiple outcome measures within a study and outcome domain were available for inclusion in a synthesis, we applied the "median effect estimate" approach, ranking the effect estimates of the outcomes within the outcome domain and selecting the outcome with the median value as a representative effect to include in the meta-analysis. In case an outcome was measured multiple times within the same time frame (eg, short-term), the earliest time point was chosen. Applying this solution allowed us to select a single outcome measure per study and outcome domain to include in our meta-analyses.
CONCLUSIONS:
Outcome multiplicity is a common challenge in SRs with broad outcome domains. Our SR provides a workable approach that enables meaningful data synthesis to inform stakeholders. Although anticipating outcome multiplicity may be challenging, review teams may benefit from mapping outcomes through a scoping exercise and prespecifying solutions at the protocol stage.