Article type
Year
Abstract
Background: A variety of tools have been developed to evaluate the quality of randomized controlled trials (RCTs) in different health areas. Seven quality tools have been frequently used in physical therapy (PT), however, it is not known how these tools compare with the Cochrane 'Risk of bias' (RoB) domain approach when deciding trials of adequate quality to be included in meta-analyses.
Objectives: To determine the level of agreement between Delphi List, PEDro, Maastricht, Maastricht- Amsterdam Lists, Bizzini, van Tulder and Jadad tools and the Cochrane RoB domain approach.
Methods: RCTs in PT were identified by searching for meta-analysis of PT interventions in the Cochrane Database of Systematic Reviews. Final scores for PT tools and Cochrane assessments were performed by our team. Trials of adequate quality were defined as having adequate sequence generation, concealment of allocation, and blinding of outcome assessors according to the Cochrane RoB tool. For each quality tool, cut-offs reported in the literature were used to define trials of adequate quality. We used kappa to calculate the agreement between each tool and the Cochrane approach for trial quality classification.
Results: A total of 393 trials (44,622 patients) included in 43 meta-analyses contributed to this study. Agreement between the tools used in PT and the Cochrane domain approach was slight for Delphi (k = 0.23, 95% CI 0.16 to 0.3), Maastricht list (k = 0.30, 95% CI 0.22 to 0.39), Van Tulder scale (k = 0.31, 95% CI 0.21 to 0.40), and Jadad scale (k = 0.23, 95% CI 0.14 to 0.31) and poor for PeDro (k = 0.12, 95% CI 0.07 to 0.16), Maastricht- Amsterdam List (k = 0.076, 95% CI 0.05 to 0.11), and Bizzini scale (k = 0.021, 95% CI 0.010 to 0.032).
Conclusion: There are important inconsistencies in determining trial quality between the Cochrane domain approach and several quality assessment tools. These disagreements have important implications for decision making.
Objectives: To determine the level of agreement between Delphi List, PEDro, Maastricht, Maastricht- Amsterdam Lists, Bizzini, van Tulder and Jadad tools and the Cochrane RoB domain approach.
Methods: RCTs in PT were identified by searching for meta-analysis of PT interventions in the Cochrane Database of Systematic Reviews. Final scores for PT tools and Cochrane assessments were performed by our team. Trials of adequate quality were defined as having adequate sequence generation, concealment of allocation, and blinding of outcome assessors according to the Cochrane RoB tool. For each quality tool, cut-offs reported in the literature were used to define trials of adequate quality. We used kappa to calculate the agreement between each tool and the Cochrane approach for trial quality classification.
Results: A total of 393 trials (44,622 patients) included in 43 meta-analyses contributed to this study. Agreement between the tools used in PT and the Cochrane domain approach was slight for Delphi (k = 0.23, 95% CI 0.16 to 0.3), Maastricht list (k = 0.30, 95% CI 0.22 to 0.39), Van Tulder scale (k = 0.31, 95% CI 0.21 to 0.40), and Jadad scale (k = 0.23, 95% CI 0.14 to 0.31) and poor for PeDro (k = 0.12, 95% CI 0.07 to 0.16), Maastricht- Amsterdam List (k = 0.076, 95% CI 0.05 to 0.11), and Bizzini scale (k = 0.021, 95% CI 0.010 to 0.032).
Conclusion: There are important inconsistencies in determining trial quality between the Cochrane domain approach and several quality assessment tools. These disagreements have important implications for decision making.