Introduction: At the 6th Annual Cochrane Colloquium in Baltimore a Cochrane Methods Working Group on Non-randomized Studies (NRS) was proposed. As part of this initiative a group was formed to examine the validity and utility of existing scales and checklists for assessing the methodological quality of non-randomized studies of health interventions. The first task the group undertook was to produce an annotated bibliography of such scales and checklists.

Objectives: It is expected that the resulting annotated bibliography of scales and checklists assessing the methodological quality of NRS will contribute towards the development of a standard scale or checklist for use within the Cochrane Collaboration.

Methods: Existing scales and checklists for the methodological quality of NRS are being identified systematically through a variety of methods including on-line searches of MEDLINE, EMBASE and Web of Science to identify published articles describing scales or checklists and systematic reviews using scales and checklists; through reviews of references ; and through personal contacts with individuals and journal editors for unpublished scales and checklists. Abstracts of potentially eligible articles are being reviewed independently by two reviewers with disagreements being resolved by consensus. The full-text of those articles selected for further review are being assessed for inclusion by two independent reviewers using explicit criteria. Information is being extracted form these articles using a standard form by two independent reviewers. Information is being collated regarding: 1) The source of the scale or checklist 2) Intended study design e.g. case-control 3) Intended content area e.g. ophthalmology 4) Criteria included in checklist-categorized hierarchically under different domains 5) Reliability and validity testing of criteria 6) Description of scales developed from criteria 7) Reliability and validity testing of scales.