Article type
Year
Abstract
Background: despite the increasing use and popularity of systematic reviews (SRs), it is crucial to assess the certainty or quality of evidence (QoE) that they provide, since the design itself does not guarantee high QoE. The GRADE approach is the soundest system for rating the quality of a body of evidence in SRs and other evidence syntheses.
Objectives: to evaluate if authors of published SRs reported the level of QoE in infertility journals and to analyze if they used an appropriate wording to describe that QoE.
Methods: we selected the five journals focused on human reproduction with the highest impact factors, according to the 2017 impact factor. Firstly, we performed a search in September 2018 on PubMed, identifying the potential SRs with meta-analysis (limits, type of article: meta-analysis) published in 2017. Secondly, we screened the studies by title and abstract to include only those that were SRs of interventions, and where the main subject of study was infertility. We analyzed if they classified the published evidence in the full text and also if they did it in the abstract. When study authors did not evaluate the QoE, we used GRADE to analyze it. We described how often the authors used a tool for QoE and the level of the QoE published in the selected SRs. We also analyzed if using a tool for QoE was associated with the P value or not, and if the P value was associated with the level of the evidence or not. Finally, we analyzed if the study authors made any effort to adapt the wording used in the abstract to the QoE and the magnitude of the described estimated effect.
Results: study authors reported QoE in only 21.4% of the included SRs and in less than 10% of the abstracts. Although we did not find important differences in the reports of QoE of those that showed statistically significant differences and those that did not, P value was associated with the wording chosen by study authors. In general, magnitude of the effect was not expressed with consistent wording in 54.8% (23/42) of the SRs, while level of QoE was not expressed with consistent wording in 92.9% (39/42) of them. Whereas magnitude of the effect was more consistently expressed in studies with statistically significant findings, QoE was better expressed in those cases in which the P value was over 0.05.
Conclusions: we found that in 2017, in major infertility journals, less than 25% of study authors reported the overall QoE when publishing a systematic review. Study authors still focus more on the discussion if the found difference was by chance or not, and less on limitations in the study design, imprecision, indirectness, inconsistency among the primary studies and publication bias. Study authors should make efforts to interpret results in the context of those evaluations.
Patient or healthcare consumer involvement: one of the authors is patient.
Objectives: to evaluate if authors of published SRs reported the level of QoE in infertility journals and to analyze if they used an appropriate wording to describe that QoE.
Methods: we selected the five journals focused on human reproduction with the highest impact factors, according to the 2017 impact factor. Firstly, we performed a search in September 2018 on PubMed, identifying the potential SRs with meta-analysis (limits, type of article: meta-analysis) published in 2017. Secondly, we screened the studies by title and abstract to include only those that were SRs of interventions, and where the main subject of study was infertility. We analyzed if they classified the published evidence in the full text and also if they did it in the abstract. When study authors did not evaluate the QoE, we used GRADE to analyze it. We described how often the authors used a tool for QoE and the level of the QoE published in the selected SRs. We also analyzed if using a tool for QoE was associated with the P value or not, and if the P value was associated with the level of the evidence or not. Finally, we analyzed if the study authors made any effort to adapt the wording used in the abstract to the QoE and the magnitude of the described estimated effect.
Results: study authors reported QoE in only 21.4% of the included SRs and in less than 10% of the abstracts. Although we did not find important differences in the reports of QoE of those that showed statistically significant differences and those that did not, P value was associated with the wording chosen by study authors. In general, magnitude of the effect was not expressed with consistent wording in 54.8% (23/42) of the SRs, while level of QoE was not expressed with consistent wording in 92.9% (39/42) of them. Whereas magnitude of the effect was more consistently expressed in studies with statistically significant findings, QoE was better expressed in those cases in which the P value was over 0.05.
Conclusions: we found that in 2017, in major infertility journals, less than 25% of study authors reported the overall QoE when publishing a systematic review. Study authors still focus more on the discussion if the found difference was by chance or not, and less on limitations in the study design, imprecision, indirectness, inconsistency among the primary studies and publication bias. Study authors should make efforts to interpret results in the context of those evaluations.
Patient or healthcare consumer involvement: one of the authors is patient.