Article type
Year
Abstract
Background: Randomized controlled trials (RCTs) and controlled clinical trials (CCTs) are often not available to address specific questions in comparative effectiveness reviews (CERs). Therefore, observational studies are frequently included in CERs, particularly when head-to-head trial data are lacking. There is, however, no standardized approach to establishing inclusion criteria for observational studies. The inclusion of these studies may involve significant resources use and it is often unclear early in the review what additional information observational studies might add to the review's conclusions.
Objective: To review criteria for inclusion, quality assessment, approaches to synthesis, and the contribution to conclusions of observational studies in CERs published by U.S. Evidence-based Practice Centers (EPCs) under the Effective Healthcare Program (http://effectivehealthcare.ahrq.gov/).
Methods: We reviewed the use of observational studies in all CERs of drugs, procedures, or medical devices published to date under the Effective Healthcare program. These reviews were performed in duplicate using a standardized abstraction form. A qualitative synthesis was performed.
Results: Twelve CERs have been completed to date: drugs 8; devices: 2; procedures: 2. Four drug reviews were restricted to RCTs for questions of benefits; of these, 2 were also restricted to RCTs for the assessment of harms. Six reviews used specific search terms for observational studies. Of the 10 reports that reported formal quality assessment of observational studies, 4 used tools developed in-house. Data from observational studies were generally synthesized in a qualitative fashion; the text, however, was not always clear on whether trials or observational studies were being discussed. There was no pooling of data from observational studies. In 2 reports that included observational studies, data from these studies were not mentioned in the discussion. Data from observational studies were used in the conclusions on benefits in 2 reviews and in the conclusions on adverse events in 6 reviews. No review explicitly examined observational studies for information on applicability or adherence.
Conclusions: Approaches to the identification, inclusion, synthesis, and the integration of observational studies data into conclusions vary widely among CERs produced by EPCs. Most CERs include observational studies in some part of their reports, but frequently there is an incomplete description of the rationale for their inclusion or exclusion, quality assessment is variably performed, and data from observational studies are often not incorporated into the discussion or conclusions. Observational studies are used more commonly for adverse events than for questions of benefit. Reasons for this variation are unclear, but may include: unstated concerns regarding potential biases; resource limitations; experience and expertise with observational studies; and different considerations for questions of benefits versus harms. Although our early experiences identified examples where data from observational studies were helpful, more work is needed to standardize approaches and to determine early in the review process where such studies might provide useful information.
Objective: To review criteria for inclusion, quality assessment, approaches to synthesis, and the contribution to conclusions of observational studies in CERs published by U.S. Evidence-based Practice Centers (EPCs) under the Effective Healthcare Program (http://effectivehealthcare.ahrq.gov/).
Methods: We reviewed the use of observational studies in all CERs of drugs, procedures, or medical devices published to date under the Effective Healthcare program. These reviews were performed in duplicate using a standardized abstraction form. A qualitative synthesis was performed.
Results: Twelve CERs have been completed to date: drugs 8; devices: 2; procedures: 2. Four drug reviews were restricted to RCTs for questions of benefits; of these, 2 were also restricted to RCTs for the assessment of harms. Six reviews used specific search terms for observational studies. Of the 10 reports that reported formal quality assessment of observational studies, 4 used tools developed in-house. Data from observational studies were generally synthesized in a qualitative fashion; the text, however, was not always clear on whether trials or observational studies were being discussed. There was no pooling of data from observational studies. In 2 reports that included observational studies, data from these studies were not mentioned in the discussion. Data from observational studies were used in the conclusions on benefits in 2 reviews and in the conclusions on adverse events in 6 reviews. No review explicitly examined observational studies for information on applicability or adherence.
Conclusions: Approaches to the identification, inclusion, synthesis, and the integration of observational studies data into conclusions vary widely among CERs produced by EPCs. Most CERs include observational studies in some part of their reports, but frequently there is an incomplete description of the rationale for their inclusion or exclusion, quality assessment is variably performed, and data from observational studies are often not incorporated into the discussion or conclusions. Observational studies are used more commonly for adverse events than for questions of benefit. Reasons for this variation are unclear, but may include: unstated concerns regarding potential biases; resource limitations; experience and expertise with observational studies; and different considerations for questions of benefits versus harms. Although our early experiences identified examples where data from observational studies were helpful, more work is needed to standardize approaches and to determine early in the review process where such studies might provide useful information.