Article type
Year
Abstract
Background:
Risk of bias tools are used to evaluate bias in most randomized controlled trials (RCTs). Having the ability to assess risk of bias automatically would streamline the evaluation process. However, at the present time no study has identified the relationship between linguistic characteristics of articles and risk of bias.
Objectives:
To investigate the lexical and syntactic characteristics of RCTs in terms of risk of bias tools and provide the best approach to detect high-risk-of-bias RCTs.
Methods:
We searched five databases (one Japanese and four international databases: the Japan Medical Abstract Society Database, MEDLINE, EMBASE, CINAHL and PsycINFO) for all RCTs with an English abstract published in Japan in 2010. Sixty per cent of the included RCTs were randomly sampled and risk of bias was assessed using a risk-of-bias tool. We classified risk-of-bias articles as follows: low risk-of-bias group (G1) (more than 6/12 domains were scored as 'low risk of bias'), and high risk-of-bias group (G2) (less than 6/12 domains). After omitting unnecessary items or words such as punctuation and stop-words, and stemming to retrieve their radicals, baseline characteristics and lexical choices such as vocabulary were compared between high and low risk of bias.
Results:
Of 2957 studies in 2010, 514 articles were identified as RCTs with an English abstract and 302 were evaluated by risk of bias (high: 166; low: 136). Although the mean length of one word was not different (G1 = 5.93 and G2 = 5.88 (P value 0.55)), mean words per a sentence were significantly different between G1 = 15.0 and G2 = 12.7 (P value < 0.001). The top five frequently used words in both groups were 'group', 'patients', 'significantly', 'study' and 'treatment'. Other frequently used words were 'placebo' and 'randomized' in G1, and 'control' and 'level' in G2.
Conclusions:
Although two groups used words of a similar length and vocabulary, high-risk-of-bias RCTs had a tendency to use longer sentences using specific words. Further research is needed to develop a risk-of-bias tool that allows automatic assessment.
Risk of bias tools are used to evaluate bias in most randomized controlled trials (RCTs). Having the ability to assess risk of bias automatically would streamline the evaluation process. However, at the present time no study has identified the relationship between linguistic characteristics of articles and risk of bias.
Objectives:
To investigate the lexical and syntactic characteristics of RCTs in terms of risk of bias tools and provide the best approach to detect high-risk-of-bias RCTs.
Methods:
We searched five databases (one Japanese and four international databases: the Japan Medical Abstract Society Database, MEDLINE, EMBASE, CINAHL and PsycINFO) for all RCTs with an English abstract published in Japan in 2010. Sixty per cent of the included RCTs were randomly sampled and risk of bias was assessed using a risk-of-bias tool. We classified risk-of-bias articles as follows: low risk-of-bias group (G1) (more than 6/12 domains were scored as 'low risk of bias'), and high risk-of-bias group (G2) (less than 6/12 domains). After omitting unnecessary items or words such as punctuation and stop-words, and stemming to retrieve their radicals, baseline characteristics and lexical choices such as vocabulary were compared between high and low risk of bias.
Results:
Of 2957 studies in 2010, 514 articles were identified as RCTs with an English abstract and 302 were evaluated by risk of bias (high: 166; low: 136). Although the mean length of one word was not different (G1 = 5.93 and G2 = 5.88 (P value 0.55)), mean words per a sentence were significantly different between G1 = 15.0 and G2 = 12.7 (P value < 0.001). The top five frequently used words in both groups were 'group', 'patients', 'significantly', 'study' and 'treatment'. Other frequently used words were 'placebo' and 'randomized' in G1, and 'control' and 'level' in G2.
Conclusions:
Although two groups used words of a similar length and vocabulary, high-risk-of-bias RCTs had a tendency to use longer sentences using specific words. Further research is needed to develop a risk-of-bias tool that allows automatic assessment.