Article type
Year
Abstract
Background: when communicating the findings of a systematic review authors sometimes use phrases which may be confusing to readers, such as “possibly large but non-significant results”. In addition, authors may communicate the results without providing information about how certain they are about those results.
Objectives: to develop a list of statements that can be used by authors to write the results and conclusions of a systematic review; and provide guidance in the Cochrane Handbook and other resources.
Methods: we revised a set of statements first developed in 2010. The original list was based on user-testing, interviews, and a randomized controlled trial primarily in consumers. This list included few options for communicating results and was therefore expanded by a small working group to include more options. We then conducted multiple workshops for feedback and an electronic survey of producers and users of systematic reviews. During the workshops and surveys, we provided multiple examples that included an analysis for an outcome from a systematic review, the certainty of evidence (using the GRADE approach), and the importance of the effect to stakeholders. We calculated the acceptability of the statements, incorporated the general feedback, and iteratively revised the wording of those statements unacceptable to ≥ 40% of participants.
Results: eighty participants at the workshops provided feedback, and 110 respondents to the survey. Participants agreed that conclusions should take into account the effect estimate and the certainty of evidence. From the survey, statements for low, moderate and high certainty evidence were acceptable to > 60% of participants. For example, statements such as “x results in a large reduction…”; “x likely results in a large reduction…”; “x may result in a large reduction…” for high, moderate and low certainty evidence (respectively) for a large effect were acceptable. In the workshops and surveys, participants expressed concern about describing statistically non-significant results as ‘no effect’, in particular when confidence intervals are wide. Participants also were concerned about providing estimates of effect when the evidence was of very low certainty. These comments were incorporated.
Conclusions: producers and users of systematic reviews found statements to communicate findings that combine size and certainty of an effect acceptable. These statements can be used in many sections of a review: the Abstract, Plain language summary, Discussion and Conclusions, and 'Summary of findings' tables. Guidance and a template to formulate statements was developed and is provided in the update of the Cochrane Handbook for Systematic Reviews of Interventions.
Patient or healthcare consumer involvement: this template and guidance to communicate the results of systematic reviews was originally based on research with and for patients and consumers; and further developed to reach other stakeholders.
Objectives: to develop a list of statements that can be used by authors to write the results and conclusions of a systematic review; and provide guidance in the Cochrane Handbook and other resources.
Methods: we revised a set of statements first developed in 2010. The original list was based on user-testing, interviews, and a randomized controlled trial primarily in consumers. This list included few options for communicating results and was therefore expanded by a small working group to include more options. We then conducted multiple workshops for feedback and an electronic survey of producers and users of systematic reviews. During the workshops and surveys, we provided multiple examples that included an analysis for an outcome from a systematic review, the certainty of evidence (using the GRADE approach), and the importance of the effect to stakeholders. We calculated the acceptability of the statements, incorporated the general feedback, and iteratively revised the wording of those statements unacceptable to ≥ 40% of participants.
Results: eighty participants at the workshops provided feedback, and 110 respondents to the survey. Participants agreed that conclusions should take into account the effect estimate and the certainty of evidence. From the survey, statements for low, moderate and high certainty evidence were acceptable to > 60% of participants. For example, statements such as “x results in a large reduction…”; “x likely results in a large reduction…”; “x may result in a large reduction…” for high, moderate and low certainty evidence (respectively) for a large effect were acceptable. In the workshops and surveys, participants expressed concern about describing statistically non-significant results as ‘no effect’, in particular when confidence intervals are wide. Participants also were concerned about providing estimates of effect when the evidence was of very low certainty. These comments were incorporated.
Conclusions: producers and users of systematic reviews found statements to communicate findings that combine size and certainty of an effect acceptable. These statements can be used in many sections of a review: the Abstract, Plain language summary, Discussion and Conclusions, and 'Summary of findings' tables. Guidance and a template to formulate statements was developed and is provided in the update of the Cochrane Handbook for Systematic Reviews of Interventions.
Patient or healthcare consumer involvement: this template and guidance to communicate the results of systematic reviews was originally based on research with and for patients and consumers; and further developed to reach other stakeholders.