Article type
Year
Abstract
Background:
Data abstraction is at the core of a systematic review (SR). Earlier studies have examined error rates and level of experience, yet data on factors associated with the quality of data abstraction using modern data systems such as the Systematic Review Data Repository (SRDR) are limited.
Objectives:
Estimate the time taken for data abstraction using SRDR and examine factors associated with errors in data abstraction.
Methods:
We developed and pilot-tested a data abstraction form in SRDR to capture the characteristics of study design (68 items), participants (12 items), and results (15-25 items) of a trial report. Eight data abstractors (DA), 4 experienced and 4 less experienced, performed data abstraction for 4 trial reports each. DAs recorded time taken for completing each section of the form. We compared each datum entry against the reference value and calculated the proportion (%) of errors. Linear regression was used to examine the association between time and % errors.
Results:
Experienced DAs had an average (avg) of 3.3 years (range 2-4), and less experienced had an avg of 9 months (range 0-12) working in SRs and participating in an avg of 4.3 (range 3-8) and 0.75 (range 0-1) reviews, respectively. The overall mean time for data abstraction was 47.8 minutes (range 38.0-58.1 min) and the overall error rate was 17.0%. Less experienced DAs took less time to complete data abstraction and made 14.2% more errors across all types of questions (95% CI: 7.2 to 21.2%) (Table). Highest errors were observed in questions collecting numerical results (27.2%), followed by questions about participant characteristics (10.3%), and design (6.2%). For every 5-minute increase in time, 2.4% fewer errors occurred on avg (95% CI: 1.3%-3.6%, P value < 0.001)(Figure), regardless of experience level.
Conclusions:
These are preliminary data, but we found evidence that the faster that data are abstracted, the greater the proportion of errors, for both experienced and less experienced DAs. More research is needed to determine whether quality assurance and rigorous training prior to data abstraction may reduce risk of errors, while maximizing efficiency.
Data abstraction is at the core of a systematic review (SR). Earlier studies have examined error rates and level of experience, yet data on factors associated with the quality of data abstraction using modern data systems such as the Systematic Review Data Repository (SRDR) are limited.
Objectives:
Estimate the time taken for data abstraction using SRDR and examine factors associated with errors in data abstraction.
Methods:
We developed and pilot-tested a data abstraction form in SRDR to capture the characteristics of study design (68 items), participants (12 items), and results (15-25 items) of a trial report. Eight data abstractors (DA), 4 experienced and 4 less experienced, performed data abstraction for 4 trial reports each. DAs recorded time taken for completing each section of the form. We compared each datum entry against the reference value and calculated the proportion (%) of errors. Linear regression was used to examine the association between time and % errors.
Results:
Experienced DAs had an average (avg) of 3.3 years (range 2-4), and less experienced had an avg of 9 months (range 0-12) working in SRs and participating in an avg of 4.3 (range 3-8) and 0.75 (range 0-1) reviews, respectively. The overall mean time for data abstraction was 47.8 minutes (range 38.0-58.1 min) and the overall error rate was 17.0%. Less experienced DAs took less time to complete data abstraction and made 14.2% more errors across all types of questions (95% CI: 7.2 to 21.2%) (Table). Highest errors were observed in questions collecting numerical results (27.2%), followed by questions about participant characteristics (10.3%), and design (6.2%). For every 5-minute increase in time, 2.4% fewer errors occurred on avg (95% CI: 1.3%-3.6%, P value < 0.001)(Figure), regardless of experience level.
Conclusions:
These are preliminary data, but we found evidence that the faster that data are abstracted, the greater the proportion of errors, for both experienced and less experienced DAs. More research is needed to determine whether quality assurance and rigorous training prior to data abstraction may reduce risk of errors, while maximizing efficiency.