Article type
Year
Abstract
Background:
Analyses of effect modification in randomized trials and meta-analyses are important to inform personalized medicine, but can result in spurious inferences. For example, debates regarding the credibility of subgroup analyses and meta-regression are often contentious. A formal, consensus-based, and user-tested instrument to assess the credibility of effect modification in randomized trials and meta-analyses remains unavailable. We are currently developing such an instrument.
Methods:
We are following a rigorous instrument development process. Our systematic survey of the methodological literature on the credibility of effect modification analyses identified 36 candidate items for the new instrument (Table 1). In collaboration with 10 experts (all authors of at least two methodological publications addressing effect modification), we are currently conducting a consensus study in which the team provides feedback on sequential versions of a new instrument. In addition, we will involve potential users of the new instrument including trialists, systematic reviewers, guideline developers, and journals editors. They will apply the draft instrument to a sample of subgroup analyses using formal user testing methods. Once the instrument is finalized, we will perform a sensibility testing with a second group of experts, and a reliability study with a second group of users.
Results:
Having presented the concept and candidate items at the last Cochrane Colloquium, we will now present the instrument that emerges from the consensus study and the user testing. We will demonstrate the use of the credibility instrument in a number of examples and discuss its compatibility with Cochrane Reviews and the GRADE approach.
Conclusion:
We expect that the new instrument will improve the analysis, interpretation, and reporting of effect modification in individual trials and meta-analyses.
Patient or healthcare consumer involvement: Not applicable
Analyses of effect modification in randomized trials and meta-analyses are important to inform personalized medicine, but can result in spurious inferences. For example, debates regarding the credibility of subgroup analyses and meta-regression are often contentious. A formal, consensus-based, and user-tested instrument to assess the credibility of effect modification in randomized trials and meta-analyses remains unavailable. We are currently developing such an instrument.
Methods:
We are following a rigorous instrument development process. Our systematic survey of the methodological literature on the credibility of effect modification analyses identified 36 candidate items for the new instrument (Table 1). In collaboration with 10 experts (all authors of at least two methodological publications addressing effect modification), we are currently conducting a consensus study in which the team provides feedback on sequential versions of a new instrument. In addition, we will involve potential users of the new instrument including trialists, systematic reviewers, guideline developers, and journals editors. They will apply the draft instrument to a sample of subgroup analyses using formal user testing methods. Once the instrument is finalized, we will perform a sensibility testing with a second group of experts, and a reliability study with a second group of users.
Results:
Having presented the concept and candidate items at the last Cochrane Colloquium, we will now present the instrument that emerges from the consensus study and the user testing. We will demonstrate the use of the credibility instrument in a number of examples and discuss its compatibility with Cochrane Reviews and the GRADE approach.
Conclusion:
We expect that the new instrument will improve the analysis, interpretation, and reporting of effect modification in individual trials and meta-analyses.
Patient or healthcare consumer involvement: Not applicable