Introduction: Many methodological criteria have been established to maximize validity of controlled trials and systematic reviews. Although these criteria are justified on theoretical considerations, many have not been examined empirically, and in a systematic manner, to evaluate whether they really have an effect on the validity of such studies.

Objective: Development of a database of studies, which empirically examine methodological issues pertaining to controlled trials and systematic reviews.

Methods: Preliminary inclusion criteria were established dividing studies into categories according to quality criteria. Searches were done by each member of the study group using resources such as online computer databases, handsearches, personal contacts with methodologists, and personal files. A pilot study was then conducted on a sample of articles to test agreement in categorization of the studies among group members and to refine the inclusion criteria. A structured abstract format was developed and tested in a second pilot study. All relevant studies will be indexed by key words and author on a central database.

Results: Good agreement was found in the pilot study examining the application of the inclusion criteria. Inclusion criteria and the structured abstract format were further refined to form the method now being used in the development of the database.

Discussion: At a preliminary point of the project a number of high quality studies have been identified which form the beginning of the database. The formation of the database will allow us to identify areas of methodological research requiring further study. We have applied for recognition as a Cochrane methods working group.