Article type
Year
Abstract
Background: Currently, the recommended method of data extraction in systematic reviews is for two reviewers to independently extract data from relevant studies and resolve discrepancies through discussion or in consultation with a third person (double data extraction). This process can be labour-intensive, time-consuming, and costly. There is little evidence comparing double data extraction to alternate approaches to data extraction, such as extraction by one reviewer with verification by a second reviewer (single data extraction). Specifically, it remains unclear whether the types and magnitude of errors vary for different methods of data extraction and the relative impact of alternate methods of data extraction on the results of meta-analysis.
Objectives: 1) To describe and compare the types and frequency of errors that accompany single versus double data extraction 2) To compare the estimate of treatment effect derived from single versus double data extraction 3) To compare the time required for single versus double data extraction
Methods: A comprehensive search was conducted for reports of randomized controlled trials related to the use of melatonin for the treatment of sleep disorders. Two pairs of reviewers will extract data from each study included in the review using a standardized Data Extraction Form. One pair of reviewers will extract data by the double data extraction method, while the other pair will extract data by the single data extraction method. The four individuals involved in data extraction will be systematically allocated to the role of data extractor or verifier for each study included in the review. The reviewers will be blind to the study hypothesis. Data extracted will include details of study design and inclusion/exclusion criteria; details of the population and intervention; and results obtained for various outcomes related to the sleep/ wake cycle. Each pair of reviewers will resolve discrepancies in data extraction by consensus or consultation with a third party. The consensus data for each pair of reviewers will be compared and the rate of agreement calculated. The consensus data for each pair of reviewers will also be compared to a reference data set and the types and rate of error for each method of data extraction will be determined. A log will be kept of the time spent by each reviewer completing data extraction and/or verification and the time required to complete data extraction by each method calculated. Consensus data generated by each method of data extraction will be used to conduct a meta-analysis for the two primary outcomes of the review and results will be compared.
Results: Thirty studies met eligibility criteria and were included in the review. The standardized Data Extraction Form has been devised and data extraction is underway. The results of this study will be available and presented at the Cochrane Colloquium, 2004 in Ottawa.
Conclusions: This study will describe and compare the accuracy and efficiency associated with two methods of data extraction, which may clarify the optimal approach to data extraction in systematic reviews.
Objectives: 1) To describe and compare the types and frequency of errors that accompany single versus double data extraction 2) To compare the estimate of treatment effect derived from single versus double data extraction 3) To compare the time required for single versus double data extraction
Methods: A comprehensive search was conducted for reports of randomized controlled trials related to the use of melatonin for the treatment of sleep disorders. Two pairs of reviewers will extract data from each study included in the review using a standardized Data Extraction Form. One pair of reviewers will extract data by the double data extraction method, while the other pair will extract data by the single data extraction method. The four individuals involved in data extraction will be systematically allocated to the role of data extractor or verifier for each study included in the review. The reviewers will be blind to the study hypothesis. Data extracted will include details of study design and inclusion/exclusion criteria; details of the population and intervention; and results obtained for various outcomes related to the sleep/ wake cycle. Each pair of reviewers will resolve discrepancies in data extraction by consensus or consultation with a third party. The consensus data for each pair of reviewers will be compared and the rate of agreement calculated. The consensus data for each pair of reviewers will also be compared to a reference data set and the types and rate of error for each method of data extraction will be determined. A log will be kept of the time spent by each reviewer completing data extraction and/or verification and the time required to complete data extraction by each method calculated. Consensus data generated by each method of data extraction will be used to conduct a meta-analysis for the two primary outcomes of the review and results will be compared.
Results: Thirty studies met eligibility criteria and were included in the review. The standardized Data Extraction Form has been devised and data extraction is underway. The results of this study will be available and presented at the Cochrane Colloquium, 2004 in Ottawa.
Conclusions: This study will describe and compare the accuracy and efficiency associated with two methods of data extraction, which may clarify the optimal approach to data extraction in systematic reviews.