Article type
Year
Abstract
Background:
Studies with statistically non-significant results are known as negative studies. Published negative studies should ideally have the same rigor of reported methodological quality as studies with positive findings. However, methodological quality of negative versus positive studies is not known.
Objective:
To assess the reported methodological quality of positive versus negative studies.
Methods:
We conducted a systematic review (SR) of all comparative studies published between 2011-2013 in Indian medical journals with a clinical science focus and an impact factor > 1. Methodological quality of randomized controlled trials (RCTs) was assessed using the Cochrane risk of bias assessment tool, and the Newcastle Ottawa scale for observational studies. A study was considered positive if the primary outcome was statistically significant and negative otherwise. When primary outcome was not reported, we used data on the first outcome reported in background followed by results section. Differences in various methodological quality domains between positive versus negative studies were assessed by Fisher’s exact test.
Results:
The SR included six journals with 260 comparative studies. 25% (65/260) were RCTs, 25% (64/260) cohort studies, 50% (130/260) case control studies, and one cross-sectional study. 53% (137/260) of studies explicitly reported the primary outcome. Statistical significance was determined by P value in 78% (203/260), confidence intervals (CI) in 3% (7/260), both P value and CI in 12% (31/260) and descriptive method in 6% (16/260) of studies. 24% (62/260) studies had negative findings and 72% (188/260) studies had positive findings. 4% (10/260) of studies had unclear findings. No statistically significant differences between reporting of methodological quality were detected between studies with positive versus negative findings (see table).
Conclusion: The difference in reported methodological quality of positive versus negative studies was not statistically significant. However, the results are compromised by uneven reporting of positive versus negative studies (72% versus 28%) indicating a tremendous publication bias.
Studies with statistically non-significant results are known as negative studies. Published negative studies should ideally have the same rigor of reported methodological quality as studies with positive findings. However, methodological quality of negative versus positive studies is not known.
Objective:
To assess the reported methodological quality of positive versus negative studies.
Methods:
We conducted a systematic review (SR) of all comparative studies published between 2011-2013 in Indian medical journals with a clinical science focus and an impact factor > 1. Methodological quality of randomized controlled trials (RCTs) was assessed using the Cochrane risk of bias assessment tool, and the Newcastle Ottawa scale for observational studies. A study was considered positive if the primary outcome was statistically significant and negative otherwise. When primary outcome was not reported, we used data on the first outcome reported in background followed by results section. Differences in various methodological quality domains between positive versus negative studies were assessed by Fisher’s exact test.
Results:
The SR included six journals with 260 comparative studies. 25% (65/260) were RCTs, 25% (64/260) cohort studies, 50% (130/260) case control studies, and one cross-sectional study. 53% (137/260) of studies explicitly reported the primary outcome. Statistical significance was determined by P value in 78% (203/260), confidence intervals (CI) in 3% (7/260), both P value and CI in 12% (31/260) and descriptive method in 6% (16/260) of studies. 24% (62/260) studies had negative findings and 72% (188/260) studies had positive findings. 4% (10/260) of studies had unclear findings. No statistically significant differences between reporting of methodological quality were detected between studies with positive versus negative findings (see table).
Conclusion: The difference in reported methodological quality of positive versus negative studies was not statistically significant. However, the results are compromised by uneven reporting of positive versus negative studies (72% versus 28%) indicating a tremendous publication bias.