Article type
Year
Abstract
Background:
Individual participant data (IPD) meta-analyses (MAs) allow more comprehensive analyses (e.g., individual-level subgroup analyses) and enable in-depth risk of bias and trustworthiness checks. Yet often not all eligible studies have IPD available, leading to concerns about data availability bias. Methods have been proposed to combine IPD with aggregate data (AD) from publications, albeit some have raised concerns about lower quality of AD. Limited guidance exists on when to combine IPD with AD.
Objectives:
To develop a checklist on whether to combine IPD and AD and apply this to a large IPD-MA in neonatology.
Methods:
We conducted a literature review and consulted methods experts to collate proposed guidance. Checks were reviewed and consolidated by an advisory group of content and methods experts and documented in a pre-specified statistical analysis plan. The checks were applied to the large IPD-MA of 104 eligible studies, in which 61 provided IPD and 43 only AD.
Results:
The developed checklist encourages researchers to use a range of prespecified criteria for comparing IPD and AD studies, including checking differences in key baseline and study characteristics, effect sizes, risk of bias, integrity concerns, and analysis methods (Table). In the large IPD-MA, we found major differences between IPD and AD in baseline characteristics, risk of bias, and integrity assessments (Table). We also found much larger effect sizes for some outcomes in the AD studies, indicating potential systematic differences between IPD and AD trials. Thus, the advisory group decided to only include IPD studies for the primary analysis, but present combined sensitivity analyses.
Conclusion:
In our case study, AD studies appeared to have higher risk of bias, be less trustworthy, and have much larger effect sizes. This highlights the importance of carefully assessing differences between IPD and AD, to not dilute the high-quality data derived in an IPD-MA with potentially lower quality AD. The developed checklist can be easily adjusted to inform future IPD-MAs.
Relevance to patients: IPD-MAs frequently inform how patients are treated. For instance, our case study will form the evidence base for upcoming international guidelines. A consumer representative was part of our advisory group and authorship team.
Individual participant data (IPD) meta-analyses (MAs) allow more comprehensive analyses (e.g., individual-level subgroup analyses) and enable in-depth risk of bias and trustworthiness checks. Yet often not all eligible studies have IPD available, leading to concerns about data availability bias. Methods have been proposed to combine IPD with aggregate data (AD) from publications, albeit some have raised concerns about lower quality of AD. Limited guidance exists on when to combine IPD with AD.
Objectives:
To develop a checklist on whether to combine IPD and AD and apply this to a large IPD-MA in neonatology.
Methods:
We conducted a literature review and consulted methods experts to collate proposed guidance. Checks were reviewed and consolidated by an advisory group of content and methods experts and documented in a pre-specified statistical analysis plan. The checks were applied to the large IPD-MA of 104 eligible studies, in which 61 provided IPD and 43 only AD.
Results:
The developed checklist encourages researchers to use a range of prespecified criteria for comparing IPD and AD studies, including checking differences in key baseline and study characteristics, effect sizes, risk of bias, integrity concerns, and analysis methods (Table). In the large IPD-MA, we found major differences between IPD and AD in baseline characteristics, risk of bias, and integrity assessments (Table). We also found much larger effect sizes for some outcomes in the AD studies, indicating potential systematic differences between IPD and AD trials. Thus, the advisory group decided to only include IPD studies for the primary analysis, but present combined sensitivity analyses.
Conclusion:
In our case study, AD studies appeared to have higher risk of bias, be less trustworthy, and have much larger effect sizes. This highlights the importance of carefully assessing differences between IPD and AD, to not dilute the high-quality data derived in an IPD-MA with potentially lower quality AD. The developed checklist can be easily adjusted to inform future IPD-MAs.
Relevance to patients: IPD-MAs frequently inform how patients are treated. For instance, our case study will form the evidence base for upcoming international guidelines. A consumer representative was part of our advisory group and authorship team.