Article type
Abstract
"Background: Non-randomized studies can generate valuable evidence for decision-making on pharmacologic treatments in the real-world setting. However, shortcomings in their conduct, analysis and reporting can diminish their utility, leading to increased research waste.
Objective: To examine non-randomized studies that assess the effectiveness and/or safety of pharmacologic treatment(s).
Methods: This was a cross-sectional study of a representative sample. We searched MEDLINE (OVID) for reports published in June-August 2022. We included reports of comparative non-randomized studies that assess the effectiveness and/or safety of pharmacologic treatment(s). We screened the titles and abstracts and the full-texts of a randomly ordered sample, until obtaining 200 eligible reports. We extracted data and assessed the risk of bias by design using a piloted form inspired from reporting guidelines and the target trial emulation framework. We considered six key reporting elements (eligibility criteria, description of treatment, treatment deviations, causal contrast, primary outcome(s) and confounding factors). Data extraction was done in duplicate (20% independently and 80% as quality control).
Results: Of 454 reports of non-randomized studies identified, 56% were not eligible (32% had no comparator and 24% did not account for confounding factors). Out of 200 included reports, 71% were at risk of bias by design: 27% due to inclusion of prevalent users, 31% due to post-treatment eligibility criteria, 43% due to immortal time periods and 23% due to classification of treatment. Furthermore, the reporting was incomplete; only 2% completely reported all key elements: eligibility criteria (87%), description of treatment (44%), treatment deviation (25%), causal contrast (10%), confounding factors (87%) and primary outcomes (77%). Most studies used routinely collected data (61%), however, only 7% reported using validation studies of codes/algorithms applied to select the population. Only 12% reported that participants did not have any contraindications to the treatment arms. Only 7% mentioned registration on a registry and 1% had an available protocol.
Conclusions: While access to real-world evidence is valuable for advancing the SDG agenda, it is necessary to strengthen the robustness and transparency of non-randomized studies, in order to enhance their value and minimize research waste.
Importance/Relevance to patients: The findings can contribute to more robust evidence production."
Objective: To examine non-randomized studies that assess the effectiveness and/or safety of pharmacologic treatment(s).
Methods: This was a cross-sectional study of a representative sample. We searched MEDLINE (OVID) for reports published in June-August 2022. We included reports of comparative non-randomized studies that assess the effectiveness and/or safety of pharmacologic treatment(s). We screened the titles and abstracts and the full-texts of a randomly ordered sample, until obtaining 200 eligible reports. We extracted data and assessed the risk of bias by design using a piloted form inspired from reporting guidelines and the target trial emulation framework. We considered six key reporting elements (eligibility criteria, description of treatment, treatment deviations, causal contrast, primary outcome(s) and confounding factors). Data extraction was done in duplicate (20% independently and 80% as quality control).
Results: Of 454 reports of non-randomized studies identified, 56% were not eligible (32% had no comparator and 24% did not account for confounding factors). Out of 200 included reports, 71% were at risk of bias by design: 27% due to inclusion of prevalent users, 31% due to post-treatment eligibility criteria, 43% due to immortal time periods and 23% due to classification of treatment. Furthermore, the reporting was incomplete; only 2% completely reported all key elements: eligibility criteria (87%), description of treatment (44%), treatment deviation (25%), causal contrast (10%), confounding factors (87%) and primary outcomes (77%). Most studies used routinely collected data (61%), however, only 7% reported using validation studies of codes/algorithms applied to select the population. Only 12% reported that participants did not have any contraindications to the treatment arms. Only 7% mentioned registration on a registry and 1% had an available protocol.
Conclusions: While access to real-world evidence is valuable for advancing the SDG agenda, it is necessary to strengthen the robustness and transparency of non-randomized studies, in order to enhance their value and minimize research waste.
Importance/Relevance to patients: The findings can contribute to more robust evidence production."