Individual participant data meta-analyses compared with meta-analyses based on aggregate data

Article type
Authors
Tudur Smith C1, Marcucci M2, Rovers M3, Riley R4, Oyee J1, Iorio A5, Williamson P1, Clarke M6
1University of Liverpool, UK
2University of Perugia, Italy
3Radboud University Nijmegen, Netherlands
4Birmingham University, UK
5McMaster University, Canada
6Queen’s University Belfast, UK
Abstract
Background: Meta-analysis based on individual participant data (IPD) is widely accepted as the most reliable approach and has been described as the 'gold standard’ for systematic reviews. An IPD approach often allows more powerful, consistent and thorough analyses but does require additional resources compared with meta-analyses based on aggregate data (AD). Several empirical comparisons of IPD with AD meta-analyses have been published, some of which show that IPD meta-analyses can differ in important ways from meta-analyses based on AD. For example, the importance of including as much follow-up as possible on all randomised participants and data from all relevant trials was shown in separate empirical comparisons (Stewart 1993, Clarke 1997, Williamson 2000), whilst Duchateau (2000) found substantial differences between IPD and literature-based AD meta-analyses, mainly due to different approaches to analysis. An unpublished review (Mukhtar et al 2008) summarised results from across 25 studies, showing that for two-thirds of the comparisons AD estimated effect sizes with less precision and tended to overestimate the IPD effect but differences were small in most comparisons.

Objectives: We have undertaken a Cochrane systematic review of empirical studies that compared IPD and AD meta-analysis to explore key reasons for the differences.

Methods: The Cochrane Methodology Register, CENTRAL, MEDLINE and EMBASE were searched using a predefined set of search terms. Studies that report an empirical comparison of IPD meta-analysis against AD meta-analysis of randomised trials were assessed for inclusion, by two reviewers independently. Data were extracted by two reviewers independently and stored in a central database.

Results: Over forty empirical studies have satisfied the inclusion criteria for this review. Estimates of effect size and precision obtained from IPD and AD will be compared and differences will be discussed.

Conclusions: Results will help inform the ongoing debate about whether, and when IPD may be most valuable.