Background: Meta-analyses combine results of specific outcomes from individual primary studies, in which trial authors occasionally publish results with figures or graphs without the exact numerical data needed for further analysis. Although the most appropriate way to deal with missing numbers is to contact authors, sometimes this is not feasible. One alternative is to calculate manually by extracting numbers using measurements in figures as a proxy.
Objectives: To analyse the reliability/accuracy of data extraction using manual measurements from figures published in individual studies.
Methods: Four figures illustrating a clinical outcome without an exact numerical value embedded in them were selected. The corresponding published study did contain an exact value that we used as gold standard. We asked participants to extract from the figures the value of the clinical endpoint of interest using ruler measurements within a PowerPoint slide as a proxy. Raters were blinded to the gold standard. We measured inter-rater reliability of participants using Type A intra-class correlation (ICC) coefficients with absolute agreement definition; we also compared the mean from the raters for each figure to their correspondent gold standard.
Results: Twenty-nine raters agreed to participate. In the four figures, less than 1% of the measurement corresponded to random variation (ICC coefficient 0.99, 95% CI 0.96 to 1.0). Only in one figure there was a statistically significant difference between the participant-estimated mean and the gold standard.
Conclusions: Measurement of an estimate of interest that is not displayed on figures from individual studies could be extracted from them by hand using any presentation program that a methodologist is familiar with. As with any extraction process in a review, extraction by duplicate is warranted to ensure exactitude of the final estimate.