Article type
Abstract
Background
The Cochrane policy for managing potentially problematic studies defines a problematic study as one “where there are serious questions about the trustworthiness of the data or findings.” These studies contain false data, and some are entirely fabricated. In systematic reviews, we identify all relevant randomized controlled trials (RCTs), subject them to critical appraisal, and synthesize their results. Critical appraisal usually considers methodological quality of the included studies but does not evaluate their authenticity. Consequently, problematic studies are not identified, and contribute to systematic review conclusions, posing a risk to patient care. The INSPECT-SR (INveStigating ProblEmatic Clinical Trials in Systematic Reviews) project has developed a tool to identify problematic studies in health systematic reviews.
Methods
INSPECT-SR was developed in 5 stages: (1) a survey of experts to assemble an extensive list of checks for assessing trustworthiness, (2) the list was applied to 50 Cochrane Reviews to evaluate feasibility and impact, (3) a Delphi survey to establish which checks were supported by expert consensus, (4) consensus meetings to determine the content and form of the tool, and (5) prospective testing in the production of new systematic reviews.
Results
The INSPECT-SR tool has been developed with input from more than 150 experts and potential users of the tool from around the globe. The development process will be described, and the resulting tool will be presented. INSPECT-SR guides the user through a series of trustworthiness checks in 4 domains: inspecting text and publication details; inspecting the results in the paper; inspecting conduct, governance, and transparency; and inspecting the research team and their work. The tool should be used while undertaking a systematic review to identify problematic RCTs.
Conclusion
INSPECT-SR has been developed using empirical evidence, theoretical considerations, expert consensus, and user testing. It is a feasible, open-access tool that can be used by systematic reviews to identify problematic studies.
The Cochrane policy for managing potentially problematic studies defines a problematic study as one “where there are serious questions about the trustworthiness of the data or findings.” These studies contain false data, and some are entirely fabricated. In systematic reviews, we identify all relevant randomized controlled trials (RCTs), subject them to critical appraisal, and synthesize their results. Critical appraisal usually considers methodological quality of the included studies but does not evaluate their authenticity. Consequently, problematic studies are not identified, and contribute to systematic review conclusions, posing a risk to patient care. The INSPECT-SR (INveStigating ProblEmatic Clinical Trials in Systematic Reviews) project has developed a tool to identify problematic studies in health systematic reviews.
Methods
INSPECT-SR was developed in 5 stages: (1) a survey of experts to assemble an extensive list of checks for assessing trustworthiness, (2) the list was applied to 50 Cochrane Reviews to evaluate feasibility and impact, (3) a Delphi survey to establish which checks were supported by expert consensus, (4) consensus meetings to determine the content and form of the tool, and (5) prospective testing in the production of new systematic reviews.
Results
The INSPECT-SR tool has been developed with input from more than 150 experts and potential users of the tool from around the globe. The development process will be described, and the resulting tool will be presented. INSPECT-SR guides the user through a series of trustworthiness checks in 4 domains: inspecting text and publication details; inspecting the results in the paper; inspecting conduct, governance, and transparency; and inspecting the research team and their work. The tool should be used while undertaking a systematic review to identify problematic RCTs.
Conclusion
INSPECT-SR has been developed using empirical evidence, theoretical considerations, expert consensus, and user testing. It is a feasible, open-access tool that can be used by systematic reviews to identify problematic studies.