To produce unbiased reviews of the effects of health care one essential condition is the inclusion of all relevant published and unpublished literature. As insufficient indexing or the nonavailability of articles in electronic databases (e.g. Medline) do not allow complete identification [1], handsearching medical journals for reports of controlled and randomised clinical trials is necessary to minimize the number of trials not taken into account. The handsearching is usually done by persons of grossly varying skills and qualification and therefore needs quality control to avoid the nonidentification of too many relevant trials. This may be achieved by letting handsearchers read a certain amount of articles in overlapping time periods and comparing the results [2]. A second approach is to repeat the handsearching for one or several years completely and checking these results against the original search. This should be done routinely but at present a major drawback is that there seems to be no decision rules for the treatment of potential observed discrepancies. In industrial quality control there is a long history of quality control of manufactured goods. Performance measures of various sampling plans describe the risk of wrongly accepting bad lots or rejecting good lots and allow estimates of the average outgoing quality if a certain plan is applied [3]. OC curves for single and double sampling plans are available to describe the behaviour of these plans as function of the quality of a lot. This paper investigates and discusses how these well established methods may be adapted and used to develop decision rules for the quality control of the handsearching of medical literature. The experiences from a project in which journals of general health care are handsearched systematically are used to set realistic conditions for the principal considerations.

References:

[I] Dickersin, K., Scherer, R., Lefebvre, C. (1994). Identifying relevant studies for systematic reviews. BMJ 309, 1286-91.
[2] The Cochrane Collaboration handbook (1994). Cochrane Collaboration, Oxford.
[3] Wadsworth, H.M., Stephens, K.S., Godfrey, A.B. (1986). Modern methods for quality control and improvement. John Wiley & Sons, New York.