Article type
Abstract
Background:
Preprints are scientific articles that have not undergone the peer-review process. They allow the latest evidence to be rapidly shared and encourage open peer review; however, it is unclear whether the results can be confidently used for decision-making during a public health emergency.
Objectives:
This study compared the data and quality of preprints released in the first 4 months of the 2022 mpox outbreak with their published counterparts and evaluated the quality between unpublished and published preprints.
Methods:
Preprints (n = 76) posted between May and August 2022 from an established mpox literature database were followed to June 2023 for changes in publication status. Quality assessment was conducted where a validated tool was available for the study design (n = 33) by 2 independent reviewers. Changes in the quality and data for preprint-published pairs (n = 53) were assessed. The overall quality between unpublished and published preprints was also compared. Data examined included general reporting characteristics, abstracts, methods, outcomes, and results.
Results:
Most included studies, regardless of whether they were preprints or published, were considered low quality, and minimal to no change in quality from preprints to published versions was identified. When examining preprint-published pairs, outcomes were often added in the published version (53%) and less commonly removed (17%). Numerical results changed from preprint to published in 51% of studies; however, most of these changes (21/27) were minor and did not impact the main conclusions of the study. Five studies had a change in the magnitude of the outcome, and 1 study had a reversal of the main conclusion along with other concerning changes in the methods.
Conclusions:
This study suggests the minimal changes in quality, results, and main conclusions from preprint to published versions supports the use of preprints, and the use of the same critical evaluation tools on preprints as applied to published studies, in decision-making during a public health emergency.
Preprints are scientific articles that have not undergone the peer-review process. They allow the latest evidence to be rapidly shared and encourage open peer review; however, it is unclear whether the results can be confidently used for decision-making during a public health emergency.
Objectives:
This study compared the data and quality of preprints released in the first 4 months of the 2022 mpox outbreak with their published counterparts and evaluated the quality between unpublished and published preprints.
Methods:
Preprints (n = 76) posted between May and August 2022 from an established mpox literature database were followed to June 2023 for changes in publication status. Quality assessment was conducted where a validated tool was available for the study design (n = 33) by 2 independent reviewers. Changes in the quality and data for preprint-published pairs (n = 53) were assessed. The overall quality between unpublished and published preprints was also compared. Data examined included general reporting characteristics, abstracts, methods, outcomes, and results.
Results:
Most included studies, regardless of whether they were preprints or published, were considered low quality, and minimal to no change in quality from preprints to published versions was identified. When examining preprint-published pairs, outcomes were often added in the published version (53%) and less commonly removed (17%). Numerical results changed from preprint to published in 51% of studies; however, most of these changes (21/27) were minor and did not impact the main conclusions of the study. Five studies had a change in the magnitude of the outcome, and 1 study had a reversal of the main conclusion along with other concerning changes in the methods.
Conclusions:
This study suggests the minimal changes in quality, results, and main conclusions from preprint to published versions supports the use of preprints, and the use of the same critical evaluation tools on preprints as applied to published studies, in decision-making during a public health emergency.