Article type
Year
Abstract
Background:
Randomised controlled trials (RCTs) are the most powerful design type to minimise bias and confounding and produce valid and reliable results. However, in certain clinical contexts RCTs are impractical, unethical or even impossible to implement. In these areas the only sources of evidence to inform important questions for systematic reviews are studies with a non-randomised design. A number of tools have been developed for assessing quality in non-randomised studies, but generally such tools are most appropriate for studies of a prospective controlled or cohort design and less appropriate for other non-randomised designs.
Methods:
In 2011, the Cochrane Epilepsy Group initiated a protocol for a review of 'Surgery for epilepsy'. A wide non-randomised evidence base exists in this clinical area. The Quality Assessment Tool for Quantitative Studies (from The Effective Public Health Practice Project (EPHPP)) was chosen as the most appropriate quality assessment tool available at the time of this review.
Results:
One-hundred and eighty-three (183) studies, mostly of a retrospective case series design, were included in the review. Quality assessment was carried out by four reviewers; a clinical specialist and a methodological specialist independently assessed each study. Agreement between reviewers was variable due to subjective interpretation of quality domains. Differences in methodological quality of the included studies were not adequately captured by the tool due to the influence of specific quality domains on the overall quality judgement.
Conclusions:
Within certain clinical contexts, RCTs are unfeasible. Currently, studies of surgical treatment for epilepsy are of poor quality, mainly of a retrospective case series design. Until higher quality evidence from better designed trials is available, a quality assessment tool is needed which is applicable to studies of such design with transparent and specific criteria to allow use by both clinical and methodological reviewers.
Randomised controlled trials (RCTs) are the most powerful design type to minimise bias and confounding and produce valid and reliable results. However, in certain clinical contexts RCTs are impractical, unethical or even impossible to implement. In these areas the only sources of evidence to inform important questions for systematic reviews are studies with a non-randomised design. A number of tools have been developed for assessing quality in non-randomised studies, but generally such tools are most appropriate for studies of a prospective controlled or cohort design and less appropriate for other non-randomised designs.
Methods:
In 2011, the Cochrane Epilepsy Group initiated a protocol for a review of 'Surgery for epilepsy'. A wide non-randomised evidence base exists in this clinical area. The Quality Assessment Tool for Quantitative Studies (from The Effective Public Health Practice Project (EPHPP)) was chosen as the most appropriate quality assessment tool available at the time of this review.
Results:
One-hundred and eighty-three (183) studies, mostly of a retrospective case series design, were included in the review. Quality assessment was carried out by four reviewers; a clinical specialist and a methodological specialist independently assessed each study. Agreement between reviewers was variable due to subjective interpretation of quality domains. Differences in methodological quality of the included studies were not adequately captured by the tool due to the influence of specific quality domains on the overall quality judgement.
Conclusions:
Within certain clinical contexts, RCTs are unfeasible. Currently, studies of surgical treatment for epilepsy are of poor quality, mainly of a retrospective case series design. Until higher quality evidence from better designed trials is available, a quality assessment tool is needed which is applicable to studies of such design with transparent and specific criteria to allow use by both clinical and methodological reviewers.