Article type
Year
Abstract
Background: Similar to primary studies, the assessment of quality or reporting of systematic reviews (SRs) is of importance. A means of reviewing the way in which quality or reporting of SRs is assessed is to conduct an overview of reports evaluating the quality of SRs.
Objectives: The objective of this study is to retrieve reports assessing the quality or the reporting of SRs to evaluate their number, the specified characteristics of SRs, reported methodology over time and in particular to consider how these reviews assessed quality of SRs.
Methods: The Cochrane Library, PubMed, Embase were searched from 1990 to 2012 for such studies considering the quality or reporting of SRs. Independent, duplicate title and abstract screening, and full text screening were conducted. Several outcomes were extracted from the identified studies including: journal of publication, year of publication, affiliation with epidemiological centers or experts, aims of the study, search strategy, selection of systematic reviews, reviewers, assessment of quality and statistical analyses. All primary results are reported descriptively, however, where appropriate adherence to specified quality or reporting items were reported across SRs.
Results: Of 11 606 independent reports retrieved from electronic searching, 99 methodological overviews were eligible for inclusion. Eleven scales or checklists were used to assess methodological or reporting quality in 69/99 reports, the remaining 30/99 developed their own criteria to assess SR quality. Only 11/99 reports assessed quality in Cochrane Reviews specifically. Almost one third (32/99) of reports did not define SRs included. Heterogeneity was considered as assessment criteria in 55/99 reports. Across reports, 259/496 (52%) SRs reported source of funding and 240/316 SRs reported review limitations.
Conclusions: Literature exists evaluating the quality and, or reporting of SRs across a number of fields, how quality is assessed varies and raises questions about how SRs should be critically appraised.
Objectives: The objective of this study is to retrieve reports assessing the quality or the reporting of SRs to evaluate their number, the specified characteristics of SRs, reported methodology over time and in particular to consider how these reviews assessed quality of SRs.
Methods: The Cochrane Library, PubMed, Embase were searched from 1990 to 2012 for such studies considering the quality or reporting of SRs. Independent, duplicate title and abstract screening, and full text screening were conducted. Several outcomes were extracted from the identified studies including: journal of publication, year of publication, affiliation with epidemiological centers or experts, aims of the study, search strategy, selection of systematic reviews, reviewers, assessment of quality and statistical analyses. All primary results are reported descriptively, however, where appropriate adherence to specified quality or reporting items were reported across SRs.
Results: Of 11 606 independent reports retrieved from electronic searching, 99 methodological overviews were eligible for inclusion. Eleven scales or checklists were used to assess methodological or reporting quality in 69/99 reports, the remaining 30/99 developed their own criteria to assess SR quality. Only 11/99 reports assessed quality in Cochrane Reviews specifically. Almost one third (32/99) of reports did not define SRs included. Heterogeneity was considered as assessment criteria in 55/99 reports. Across reports, 259/496 (52%) SRs reported source of funding and 240/316 SRs reported review limitations.
Conclusions: Literature exists evaluating the quality and, or reporting of SRs across a number of fields, how quality is assessed varies and raises questions about how SRs should be critically appraised.