Article type
Year
Abstract
Background: there are concerns about the quality and credibility of randomised clinical trials (RCTs) from various regions in the world. Current tools and procedures to prevent and detect scientific misconduct in clinical medicine are limited.
Objectives: to develop a checklist with tools to check the scientific integrity of RCTs.
Methods: we developed a checklist with toolbox to assess the integrity of published RCTs. Our checklist contained five domains:
1) presence of research ethics, documentation of placebo or other study medication, adequacy of trial registration;
2) feasibility of recruitment, feasibility of clinical data (do the data make sense?);
3) quality of statistical analysis (description of statistical methods, reproducibility of univariable statistical analysis);
4) possible inadequate data handling as indicated by the violation of Benford-Newcomb’s law for the distribution of the first leading digits;
5) Monte Carlo simulations and Kolmogorov-smirnov test to evaluate the probability of randomization.
For individual participant data, we propose four domains:
1) presence of original data such as date of birth or birthweight;
2) presence of normal distributions;
3) order of randomization;
4) excessive arithmetic progression and repeating numbers in continuous variables.
We systematically reviewed RCTs from one North-African country that were published in leading journals of Obstetrics and Gynaecology between 2013 and 2018, and applied the part of the checklist that assessed published papers.
Results: we identified 52 RCTs. The proportion of studies with proper registration was 35% (18/52). We could not reproduce sample size calculation in 44.4% (14/36) of studies. Regarding statistical analysis, 23.1% (12/52) of studies had critical concerns. The proportions of studies with not repeatable tests for continuous and categorical baseline variables were 68.8% (22/32) and 38.7% (12/31), respectively. The statistical result of the primary outcome was not repeatable in 29.4% (10/34) of studies. The distribution of first leading digits was found to violate the Benford-Newcomb’s law in 32.6% (15/46) of studies. With Monte Carlo simulations and Kolmogorov-smirnov test, the overall probability that studies were properly randomized as assessed from the baseline data across RCTs was 0.0009216.
With respect to individual participant data, we obtained data from two studies. The individual participant data indicated a high suspicion of fabricated data, with excessive arithmetic progression in both studies.
Conclusions: we report serious concerns about the overall quality of published RCTs in the field of women’s health from a North-African country. Our checklist to assess integrity can help to assess papers with fabricated data.
Objectives: to develop a checklist with tools to check the scientific integrity of RCTs.
Methods: we developed a checklist with toolbox to assess the integrity of published RCTs. Our checklist contained five domains:
1) presence of research ethics, documentation of placebo or other study medication, adequacy of trial registration;
2) feasibility of recruitment, feasibility of clinical data (do the data make sense?);
3) quality of statistical analysis (description of statistical methods, reproducibility of univariable statistical analysis);
4) possible inadequate data handling as indicated by the violation of Benford-Newcomb’s law for the distribution of the first leading digits;
5) Monte Carlo simulations and Kolmogorov-smirnov test to evaluate the probability of randomization.
For individual participant data, we propose four domains:
1) presence of original data such as date of birth or birthweight;
2) presence of normal distributions;
3) order of randomization;
4) excessive arithmetic progression and repeating numbers in continuous variables.
We systematically reviewed RCTs from one North-African country that were published in leading journals of Obstetrics and Gynaecology between 2013 and 2018, and applied the part of the checklist that assessed published papers.
Results: we identified 52 RCTs. The proportion of studies with proper registration was 35% (18/52). We could not reproduce sample size calculation in 44.4% (14/36) of studies. Regarding statistical analysis, 23.1% (12/52) of studies had critical concerns. The proportions of studies with not repeatable tests for continuous and categorical baseline variables were 68.8% (22/32) and 38.7% (12/31), respectively. The statistical result of the primary outcome was not repeatable in 29.4% (10/34) of studies. The distribution of first leading digits was found to violate the Benford-Newcomb’s law in 32.6% (15/46) of studies. With Monte Carlo simulations and Kolmogorov-smirnov test, the overall probability that studies were properly randomized as assessed from the baseline data across RCTs was 0.0009216.
With respect to individual participant data, we obtained data from two studies. The individual participant data indicated a high suspicion of fabricated data, with excessive arithmetic progression in both studies.
Conclusions: we report serious concerns about the overall quality of published RCTs in the field of women’s health from a North-African country. Our checklist to assess integrity can help to assess papers with fabricated data.