Article type
Year
Abstract
ackground: the minimal important difference (MID) is the smallest change in a treatment outcome that is considered important for patients but also for the health system. As Cochrane requires GRADE to classify the quality of the evidence, MID is needed to define the imprecision and the magnitude of the effect. Review authors should use their judgment in deciding what constitutes appreciable benefit and harm and provide a rationale for their choice. If review authors fail to find a compelling rationale for a threshold, the GRADE guideline suggests a default threshold (25 ± 5% in relative terms) for appreciable benefit and harm to rate imprecision.
Objectives: to analyze how Cochrane Systematic Reviews (SRs) and Cochrane protocols compare to non-Cochrane SRs regarding the reporting and interpretation of results, if appropriate, according to MID.
Methods: we analyzed all Cochrane SRs and protocols published in November to December 2018. We excluded updates. We also analyzed the top five Impact Factors Journal List (1 New England Journal of Medicine, 2 Lancet, 3 BMJ, 4 JAMA, 5 Annals of Internal Medicine), based on 2017 Thomson Reuters Journal Citation in medicine general and internal category, where Cochrane ranked 13th. The PubMed search strategy to identify non-Cochrane SRs was: "The New England journal of medicine"[Jour] OR "Lancet (London, England)"[Jour] OR "British medical journal"[Jour] OR "JAMA"[Jour] OR "Annals of internal medicine"[Jour]. Filters: Meta-Analysis; Systematic Reviews; Publication date from 2018/01/01 to 2018/12/31.
We explored how Cochrane SRs and protocols, and non-Cochrane SRs dealt with the MID concept as defined above, in every relevant review/protocol section.
Results: we have analzed eight Cochrane SRs, eight Cochrane protocols and nine non-Cochrane SRs so far. We will give the rest of the references at the Colloquium. No Cochrane SRs, 38% of Cochrane protocols and 56% of non-Cochrane SRs reported the MID. All the analyzed protocols were from the same author, who assessed a dichotomous outcome for different drugs in the same condition. Sixty-six percent of the non-Cochrane SRs evaluated continuous primary outcomes and 33% the risk difference, and in all cases the statements were supported by references (see the statements used in Box 1). Only the two SRs that evaluated continuous outcomes formally incorporated patients’ point of view. Only three of 14 (21%) of the SRs not reporting a clear MID statement mentioned some effect size considerations (see Box 2) but it was not possible to assess reporting consistency in these cases.
Conclusions: when MID was considered in the methods section, the reporting of the results was consistent with the definition used. In most of the cases, MID was not reported and it was not stated how authors defined it to analyze imprecision and magnitude of the effect.
Patient or healthcare consumer involvement: none, since it was a methodological study about published systematic reviews.
Objectives: to analyze how Cochrane Systematic Reviews (SRs) and Cochrane protocols compare to non-Cochrane SRs regarding the reporting and interpretation of results, if appropriate, according to MID.
Methods: we analyzed all Cochrane SRs and protocols published in November to December 2018. We excluded updates. We also analyzed the top five Impact Factors Journal List (1 New England Journal of Medicine, 2 Lancet, 3 BMJ, 4 JAMA, 5 Annals of Internal Medicine), based on 2017 Thomson Reuters Journal Citation in medicine general and internal category, where Cochrane ranked 13th. The PubMed search strategy to identify non-Cochrane SRs was: "The New England journal of medicine"[Jour] OR "Lancet (London, England)"[Jour] OR "British medical journal"[Jour] OR "JAMA"[Jour] OR "Annals of internal medicine"[Jour]. Filters: Meta-Analysis; Systematic Reviews; Publication date from 2018/01/01 to 2018/12/31.
We explored how Cochrane SRs and protocols, and non-Cochrane SRs dealt with the MID concept as defined above, in every relevant review/protocol section.
Results: we have analzed eight Cochrane SRs, eight Cochrane protocols and nine non-Cochrane SRs so far. We will give the rest of the references at the Colloquium. No Cochrane SRs, 38% of Cochrane protocols and 56% of non-Cochrane SRs reported the MID. All the analyzed protocols were from the same author, who assessed a dichotomous outcome for different drugs in the same condition. Sixty-six percent of the non-Cochrane SRs evaluated continuous primary outcomes and 33% the risk difference, and in all cases the statements were supported by references (see the statements used in Box 1). Only the two SRs that evaluated continuous outcomes formally incorporated patients’ point of view. Only three of 14 (21%) of the SRs not reporting a clear MID statement mentioned some effect size considerations (see Box 2) but it was not possible to assess reporting consistency in these cases.
Conclusions: when MID was considered in the methods section, the reporting of the results was consistent with the definition used. In most of the cases, MID was not reported and it was not stated how authors defined it to analyze imprecision and magnitude of the effect.
Patient or healthcare consumer involvement: none, since it was a methodological study about published systematic reviews.