Article type
Year
Abstract
Background:
Cochrane has adopted the principles of the GRADE approach for evaluating the quality of evidence for outcomes reported in systematic reviews. Our aim was to determine the effect of alternative formats and columns of summary of findings (SoF) tables on stakeholder’s understanding and acceptability of the presentation of review results of continuous and dichotomous outcomes.
Methods:
We presented alternative and current SoF table formats and conducted direct comparative tests. Two researchers conducted semi-structured interviews with end users (clinicians, guideline developers, researchers and policy makers) using standardized scripts. The iterative process included three main rounds: (1) GRADE working group meetings (2 sessions; 10 to 20 participants each); (2) one-on-one, 30-minute interviews (10 participants); (3) international conferences and meetings (10 participants). We asked 7-point Likert-type and open-ended questions. The information collected was tabulated and coded focusing mainly on limitations and barriers against understanding, usability and acceptability of the formats.
Results:
All participants agreed that SoF tables are helpful to present systematic review results. There was preference for simplification of tables (e.g. reducing the number of columns, minimising comments and reducing text). For dichotomous outcomes, participants considered that the number needed to treat for benefit (NNTB) and harm (NNTH) and the risk difference expressed as percentage (RD%) are useful to display review results (Likert 1-7=5.3; SD 1.7). For continuous outcomes, the minimal important difference units (MIDU) of improvement and the proportion of participants showing an improvement (PPSI) showed to be easier to understand (Likert 1-7= 6.1; SD 2.1). The 'what happens' column was perceived as very valuable by most participants.
Conclusions:
Participants prefer simple and less crowded tables. NNTs and RD%, and MIDU and PPSI seem to facilitate understanding of dichotomous and continuous outcomes results, respectively.
Cochrane has adopted the principles of the GRADE approach for evaluating the quality of evidence for outcomes reported in systematic reviews. Our aim was to determine the effect of alternative formats and columns of summary of findings (SoF) tables on stakeholder’s understanding and acceptability of the presentation of review results of continuous and dichotomous outcomes.
Methods:
We presented alternative and current SoF table formats and conducted direct comparative tests. Two researchers conducted semi-structured interviews with end users (clinicians, guideline developers, researchers and policy makers) using standardized scripts. The iterative process included three main rounds: (1) GRADE working group meetings (2 sessions; 10 to 20 participants each); (2) one-on-one, 30-minute interviews (10 participants); (3) international conferences and meetings (10 participants). We asked 7-point Likert-type and open-ended questions. The information collected was tabulated and coded focusing mainly on limitations and barriers against understanding, usability and acceptability of the formats.
Results:
All participants agreed that SoF tables are helpful to present systematic review results. There was preference for simplification of tables (e.g. reducing the number of columns, minimising comments and reducing text). For dichotomous outcomes, participants considered that the number needed to treat for benefit (NNTB) and harm (NNTH) and the risk difference expressed as percentage (RD%) are useful to display review results (Likert 1-7=5.3; SD 1.7). For continuous outcomes, the minimal important difference units (MIDU) of improvement and the proportion of participants showing an improvement (PPSI) showed to be easier to understand (Likert 1-7= 6.1; SD 2.1). The 'what happens' column was perceived as very valuable by most participants.
Conclusions:
Participants prefer simple and less crowded tables. NNTs and RD%, and MIDU and PPSI seem to facilitate understanding of dichotomous and continuous outcomes results, respectively.