Article type
Year
Abstract
Introduction:
Objectives: To determine the influence on article selection for relevance, of deletion of identifying features of journal and authors, and of results, discussion and conclusion sections, in a systematic overview of treatment effectiveness.
Methods: This study was done in the course of the systematic overview: Conservative treatment of mechanical neck disorders, by the Cervical Overview Working Group. In a factorial design, for assessment of relevance, 293 articles initially identified from sequenced computerized searches and personal files were each randomly allocated to paired independent assessments by either both reviewers "blind", both "open", or one "blind" and one "open". In blind evaluation, the following were deleted from articles: 1) title and abstract; 2) authors' names and affiliations; 3) journal names; 4) sources of financial support; 5) acknowledgments; 6) results, discussion and conclusion; and 7) particular identifying features of journal. In open evaluation, the full text was intact. The 6 reviewers' disciplines were physiotherapy, chiropractic and physical medicine. Selection criteria were: randomized/controlled trials; neck disorder +- radicular signs/symptoms or with headache; drug, physiotherapy, manual therapy or educational intervention; pain/clinically relevant measure. Disagreement was resolved by consensus. Reviewers, both blind and open, rated on a 7 point scale, their familiarity with the author and journal, and the degree to which this influenced their selection decision.
Results: 53 articles were selected as relevant and 223 not relevant; 17 could not be resolved through consensus and the authors are being contacted for information. Open vs blind reviewing status was not associated with reviewers' familiarity with author or journal or with their rating of the influence of this familiarity. In logistic regression, neither open vs blind reviewing status by reviewer 1 or reviewer 2, nor their interaction was related to selection for relevance. However, journal familiarity increased the probability of selection (OR = 3.48, p=0.03).
Discussion: This study focusing on the effect of blinding on article selection is consistent with others in indicating its limited value in systematic overviewing.
Objectives: To determine the influence on article selection for relevance, of deletion of identifying features of journal and authors, and of results, discussion and conclusion sections, in a systematic overview of treatment effectiveness.
Methods: This study was done in the course of the systematic overview: Conservative treatment of mechanical neck disorders, by the Cervical Overview Working Group. In a factorial design, for assessment of relevance, 293 articles initially identified from sequenced computerized searches and personal files were each randomly allocated to paired independent assessments by either both reviewers "blind", both "open", or one "blind" and one "open". In blind evaluation, the following were deleted from articles: 1) title and abstract; 2) authors' names and affiliations; 3) journal names; 4) sources of financial support; 5) acknowledgments; 6) results, discussion and conclusion; and 7) particular identifying features of journal. In open evaluation, the full text was intact. The 6 reviewers' disciplines were physiotherapy, chiropractic and physical medicine. Selection criteria were: randomized/controlled trials; neck disorder +- radicular signs/symptoms or with headache; drug, physiotherapy, manual therapy or educational intervention; pain/clinically relevant measure. Disagreement was resolved by consensus. Reviewers, both blind and open, rated on a 7 point scale, their familiarity with the author and journal, and the degree to which this influenced their selection decision.
Results: 53 articles were selected as relevant and 223 not relevant; 17 could not be resolved through consensus and the authors are being contacted for information. Open vs blind reviewing status was not associated with reviewers' familiarity with author or journal or with their rating of the influence of this familiarity. In logistic regression, neither open vs blind reviewing status by reviewer 1 or reviewer 2, nor their interaction was related to selection for relevance. However, journal familiarity increased the probability of selection (OR = 3.48, p=0.03).
Discussion: This study focusing on the effect of blinding on article selection is consistent with others in indicating its limited value in systematic overviewing.