Systematic review when there is no evidence . . . (2) Developing a methodology for a qualitative analysis of expert reviews on IMEs

Article type
Authors
Kunz R1, Guyatt G2, Busse J3
1University Hospital Basel, Asim, Basel, Switzerland
2CLARITY, Dept. of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, Ontario, Canada
3Research, Institute for Work and Health, Toronto, Canada
Abstract
Background: We have undertaken a systematic review of Independent Medical Evaluations (IMEs); however, 74 of the 88 published studies that we retrieved on this topic are low quality: narrative reviews, editorials, case reports, or letters to the Editor. Objective: To develop a coding system for reviewing narrative reports on IMEs. Methods: In an iterative process on a heterogeneous test set of reviews, 2 reviewers independently identified major themes. We developed coding rules through discussion and disagreement was solved with a third reviewer. After several rounds of coding eligible articles, clusters around themes emerged. We built a coding tree with domains, sub-domains and items. When the tree structure became stable, as evidenced by new articles generating no new codes and disagreement among reviewers became minimal, we applied our coding strategy to all eligible studies. Results: In 7 rounds of code-development we identified 8 domains, 49 sub-domains, and 67 items within sub-domains. Table 1 demonstrates examples of the final the coding tree. Two reviewers applied the coding tree to each eligible study, independently and in duplicate. Following agreement on coding for an article, we stored codes and the associated text in a database and produced a transcript, sorted according to sub-domains and items. Conclusions: Our empirically developed coding system captures and reorganizes all relevant themes on IMEs addressed in narrative reviews, case reports and letter to the Editor. The resulting transcript will form the basis for a comprehensive qualitative analysis1 which we anticipate will be completed for the colloquium. 1See abstracts Kunz, Guyatt, Busse (1)