Introduction: Assigning labels of study design has many advantages. It helps scientists to plan, analyze and report the results of their studies; it facilitates communication because everyone can refer to the study easily; it helps consumers to find the studies they want in a more efficient way and it generates a hierarchy of evidence because some designs can be more susceptible to bias than others. There is no single taxonomy of study design of study assessing the effects of health care interventions; however, in most cases it is a variation of the traditional taxonomy that classifies studies into the following categories: experimental, observational and descriptive studies. Experimental studies are randomized controlled trials (RCTs) and controlled clinical trials (CCTs). Observational studies are cohort, case-control and cross-sectional studies. Descriptive studies are case series and case reports.

Purpose: The purposes of this study were to assess the inter-rater reliability of this traditional taxonomy to a set of studies assessing the effects of health care interventions in the field of low-back pain and to propose improvements to the taxonomy in terms of defining the categories and operational definitions.

Methods: This study is conducted in three parts: A. One study was presented to 15 clinical epidemiologists to respond: What is the study design? No framework or taxonomy was suggested; B. Thirty randomly selected studies from a database of studies of effects of health care interventions for low-back pain were given to five experienced reviewers for assigning labels of study design according to the traditional taxonomy . The definitions were taken from the Dictionary of Epidemiology (Last, 2001). C. The next step will consist of a meeting between the reviewers (from part B) to discuss causes for disagreement and possible improvements to the taxonomy. Then, part B will be repeated with a different set of studies and different reviewers for inter-rater reliability.
Results: A. The labels of study design given to the study were: Retrospective case-control (n=1), Retrospective study (n=1), Retrospective cohort (n=5), Prospective cohort (n=1), Cohort study (n=2), Case series (n=2), Uncontrolled trial (n=2), Retrospective case-control (n=1). B. The five reviewers agreed on the same label in 10 studies. Four reviewers agreed on 10 studies. Three reviewers agreed on seven studies and the remaining three studies were considered total disagreements. C. Results will be presented at the Colloquium in Ottawa

Conclusions: Many people rely on the labels of study design to include/exclude studies from reviews or to make recommendations about effectiveness of health care interventions, such as in clinical practice guidelines that uses only a hierarchy as level of evidence. This study showed that the assignment of labels is not reliable, even when specific instructions and definitions is provided.

Future development: Our group will continue to study methods to improve the classification system of study design in the field of health care interventions. Collaboration with other entities interested in this topic will be sought.