Article type
Year
Abstract
Background: A variety of factors, including methodological quality, study topic, novelty, and readership interest contribute to the acceptance of manuscripts for publication in biomedical journals. One of the functions of the editorial process is to ensure publication of high methodological quality original research. Yet, there is limited empirical data supporting this assumption.
Objective: To compare the methodological quality of accepted versus rejected original research articles from a sample of articles initially submitted for publication at three major biomedical journals.
Methods: We obtained a sample of 1,043 original research articles submitted for publication between January 2003 and April 2003 at three biomedical journals. Original research articles included studies of experimental and observational design and systematic reviews or meta-analyses. We excluded case-reports and qualitative, ethnographic, or non-human studies. Accepted and rejected articles were identified by tracking the original submission throughout each journal's editorial process. Each accepted article was matched by study design to a rejected article. Two reviewers independently assessed the methodological quality of the initial article submission using a previously validated instrument that can be applied to experimental and observational study designs. A separate, validated instrument was used for systematic reviews/meta-analyses. We assessed individual components of methodological quality and calculated a score ranging from 0 (lowest quality) to 2 (highest quality) for each component. Mean quality scores between accepted and rejected articles were compared using t-tests. Scores for the individual components of methodological quality were dichotomized (adequate = 2, inadequate < 2) and then odds ratios were calculated using logistic regression.
Results: A total of 50 articles were accepted at the three journals (acceptance rates of original research articles at the journals ranged from approximately 4 to 6%). These original research articles consisted of 10 randomized controlled trials [RCT], 33 observational studies, 6 systematic reviews/meta-analyses, and 1 non-randomized, experimental study. The mean methodological quality score of the accepted articles (1.26, 95% confidence interval [CI], 1.17-1.35) was significantly higher than that of matched rejected articles (1.02, 95% CI, 0.93-1.11) (P<0.001). In analyses by study design, accepted articles continued to score higher than rejected papers for RCTs (n=10; P=0.03), observational studies (n=33; P=0.03), and systematic reviews/meta-analyses (n=6; P=0.08). Accepted articles were also of higher methodological quality in analyses by journal (Journal 1; n=23; P<0.01, Journal 2; n=20; P=0.02, Journal 3; n=7; P=0.12). In further analysis of RCTs (n=10), accepted articles were more likely to adequately conceal random allocation of subjects (OR, 16.00; 95% CI 1.32-194.62; P=0.03), but did not significantly differ in double blinding when compared to rejected articles (OR, 5.00; 95% CI 0.58-42.80; P=0.14).
Conclusions: Accepted original research articles are of higher methodological quality at submission than rejected articles, suggesting that the editorial process contributes to the publication of high quality research.
Acknowledgements: The authors would like to thank the journal editors for participating in this project. This study was funded by the National Institutes of Health.
Objective: To compare the methodological quality of accepted versus rejected original research articles from a sample of articles initially submitted for publication at three major biomedical journals.
Methods: We obtained a sample of 1,043 original research articles submitted for publication between January 2003 and April 2003 at three biomedical journals. Original research articles included studies of experimental and observational design and systematic reviews or meta-analyses. We excluded case-reports and qualitative, ethnographic, or non-human studies. Accepted and rejected articles were identified by tracking the original submission throughout each journal's editorial process. Each accepted article was matched by study design to a rejected article. Two reviewers independently assessed the methodological quality of the initial article submission using a previously validated instrument that can be applied to experimental and observational study designs. A separate, validated instrument was used for systematic reviews/meta-analyses. We assessed individual components of methodological quality and calculated a score ranging from 0 (lowest quality) to 2 (highest quality) for each component. Mean quality scores between accepted and rejected articles were compared using t-tests. Scores for the individual components of methodological quality were dichotomized (adequate = 2, inadequate < 2) and then odds ratios were calculated using logistic regression.
Results: A total of 50 articles were accepted at the three journals (acceptance rates of original research articles at the journals ranged from approximately 4 to 6%). These original research articles consisted of 10 randomized controlled trials [RCT], 33 observational studies, 6 systematic reviews/meta-analyses, and 1 non-randomized, experimental study. The mean methodological quality score of the accepted articles (1.26, 95% confidence interval [CI], 1.17-1.35) was significantly higher than that of matched rejected articles (1.02, 95% CI, 0.93-1.11) (P<0.001). In analyses by study design, accepted articles continued to score higher than rejected papers for RCTs (n=10; P=0.03), observational studies (n=33; P=0.03), and systematic reviews/meta-analyses (n=6; P=0.08). Accepted articles were also of higher methodological quality in analyses by journal (Journal 1; n=23; P<0.01, Journal 2; n=20; P=0.02, Journal 3; n=7; P=0.12). In further analysis of RCTs (n=10), accepted articles were more likely to adequately conceal random allocation of subjects (OR, 16.00; 95% CI 1.32-194.62; P=0.03), but did not significantly differ in double blinding when compared to rejected articles (OR, 5.00; 95% CI 0.58-42.80; P=0.14).
Conclusions: Accepted original research articles are of higher methodological quality at submission than rejected articles, suggesting that the editorial process contributes to the publication of high quality research.
Acknowledgements: The authors would like to thank the journal editors for participating in this project. This study was funded by the National Institutes of Health.