Article type
Abstract
Background: A MeaSurement Tool to Assess systematic Reviews (AMSTAR 2) and the Risk of Bias in Systematic Reviews (ROBIS) are currently the most commonly used tool for systematic reviews of healthcare interventions. A new tool to assess the risk of bias in network meta-analysis (RoB NMA) is under development: This pilot tool is recommended to be used in conjunction with ROBIS, although network meta-analysis (NMA) authors might use it also with AMSTAR 2.
Objective: To assess methodological quality and risk of bias by using AMSTAR-2, ROBIS, and the pilot ROB-NMA tool
Methods: A cross-sectional study. We searched for all NMAs indexed in PubMed in January 2023. We rated NMAs' methodological quality and risk of bias using AMSTAR 2 and ROBIS. Selection, data extraction, and quality assessments were performed independently by 2 researchers. Data were analyzed descriptively. The ROB NMA tool was available for the piloting phase in April 2023; thus, piloting of the RoB NMA tool is ongoing. Stratified analysis and correlations will be conducted to explore factors that might affect the quality of conduct. Study protocol is stored at https://osf.io/pa6dz/.
Results: 139 NMAs with a median of 19 studies (IQR 12 to 40.5) and 3415 (IQR 1472 to 9384) participants were included. Overall, 83.45% (n = 116) and 12.23% (n = 17) of NMAs were judged by AMSTAR 2 as having “critically low” and “low” quality. Table 1 reported AMSTAR 2 assessments. Preliminary results on 10 NMA for ROBIS showed that ROBIS judged 83.3% (n = 10) of NMAs as having a high risk of bias, 1 as unclear risk, and 1 as low risk. Domain 2 was at highest risk in 50% of NMAs while domain 3 was judged as low risk of bias in 75%, resulting from better-conducted NMAs. Table 2 showed ROBIS preliminary assessments. The ROB NMA tool’s assessment is ongoing.
Conclusion: Preliminary results showed that the majority of recently published NMAs had an overall low methodological quality.
Patient involvement: Poor control of potential sources of bias in NMAs might affect results’ trustworthiness and translation into clinical practice that could be beneficial for patients.
Objective: To assess methodological quality and risk of bias by using AMSTAR-2, ROBIS, and the pilot ROB-NMA tool
Methods: A cross-sectional study. We searched for all NMAs indexed in PubMed in January 2023. We rated NMAs' methodological quality and risk of bias using AMSTAR 2 and ROBIS. Selection, data extraction, and quality assessments were performed independently by 2 researchers. Data were analyzed descriptively. The ROB NMA tool was available for the piloting phase in April 2023; thus, piloting of the RoB NMA tool is ongoing. Stratified analysis and correlations will be conducted to explore factors that might affect the quality of conduct. Study protocol is stored at https://osf.io/pa6dz/.
Results: 139 NMAs with a median of 19 studies (IQR 12 to 40.5) and 3415 (IQR 1472 to 9384) participants were included. Overall, 83.45% (n = 116) and 12.23% (n = 17) of NMAs were judged by AMSTAR 2 as having “critically low” and “low” quality. Table 1 reported AMSTAR 2 assessments. Preliminary results on 10 NMA for ROBIS showed that ROBIS judged 83.3% (n = 10) of NMAs as having a high risk of bias, 1 as unclear risk, and 1 as low risk. Domain 2 was at highest risk in 50% of NMAs while domain 3 was judged as low risk of bias in 75%, resulting from better-conducted NMAs. Table 2 showed ROBIS preliminary assessments. The ROB NMA tool’s assessment is ongoing.
Conclusion: Preliminary results showed that the majority of recently published NMAs had an overall low methodological quality.
Patient involvement: Poor control of potential sources of bias in NMAs might affect results’ trustworthiness and translation into clinical practice that could be beneficial for patients.