Article type
Year
Abstract
Introduction: Searches of electronic databases can miss some true. RCTs that are identified by hand Searching journals. However, all citations retrieved from an optimally sensitive search are not neccesarily true reports of RCTS. One practical question is the degree to which individial citations may accurately be classified as true RCTs (on the basis of the database's abstract alone.
Objective: To ascertain the degree to which
1) individual MEDLINE (Index Medicus) and EMBASE, (Excerpta Medica) citations May accurately be, classified as true RCTs on the basis of the abstract and
2) a detailed reading of full text articles, particularly methods sections, is neccssary.
Methods: A total of 108 citations were retrieved in electronic searches of MEDLINE and EMBASE databases from 1980- 1993 based on the standard Cochrane Collaboration search strategy for RCTs in combination with "and asthma or wheez*" and "and education". Each citation was classified on the basis of the abstract alone by two independent raters as 1) a true RCT or 2) not a true RCT or too ambiguous to classify without additional information (Kappa=0.83). Citations were then classified on life basis of the full text article, with disagreements resolved by consensus, i.e., so that there was 100% agreement.
Results: The sensitivity (accuracy) of abstract classification of RCTs was 84%, the specificity (precision) 100%, the positive predictive value 100%, and the negative predictive values 85%. Classifying abstracts in a manner designed to be maximally specific and predictive of actual RCTs in the literature, as expected, results in some RCTs being missed that could only be identified on the basis of reading the full text of the article. Approximately 16% of true RCTs would be missed by abstract classification alone.
Objective: To ascertain the degree to which
1) individual MEDLINE (Index Medicus) and EMBASE, (Excerpta Medica) citations May accurately be, classified as true RCTs on the basis of the abstract and
2) a detailed reading of full text articles, particularly methods sections, is neccssary.
Methods: A total of 108 citations were retrieved in electronic searches of MEDLINE and EMBASE databases from 1980- 1993 based on the standard Cochrane Collaboration search strategy for RCTs in combination with "and asthma or wheez*" and "and education". Each citation was classified on the basis of the abstract alone by two independent raters as 1) a true RCT or 2) not a true RCT or too ambiguous to classify without additional information (Kappa=0.83). Citations were then classified on life basis of the full text article, with disagreements resolved by consensus, i.e., so that there was 100% agreement.
Results: The sensitivity (accuracy) of abstract classification of RCTs was 84%, the specificity (precision) 100%, the positive predictive value 100%, and the negative predictive values 85%. Classifying abstracts in a manner designed to be maximally specific and predictive of actual RCTs in the literature, as expected, results in some RCTs being missed that could only be identified on the basis of reading the full text of the article. Approximately 16% of true RCTs would be missed by abstract classification alone.