Background: Reviews of adverse effects typically include data sources for which there are no specific and standard critical appraisal instruments. There is also no widely accepted set of criteria yet for assessing the quality of reviews of adverse effects.

Objectives: To develop an understanding of what characterizes a good review of adverse effects, and to contribute to the
development of quality assessment criteria for this type of review.

Methods: A sample of 16 reviews of adverse effects of variable quality were identified during searches conducted for the development of evidence-based consumer health information. This sample was appraised to develop a brief questionnaire. A pool of experts, including the Cochrane Adverse Effects Subgroup were invited to appraise the reviews, using the questionnaire as a guide to eliciting the reasons for their judgments of quality. The responses will be used to guide the development of detailed methodology for assessing the quality of reviews of adverse effects.

Results: Systematic searches for recent reviews of adverse effects in three topic areas for evidence-based consumer health information identified 16 reviews: seven for treatments for the menopause; five for corticosteroids for asthma and/or chronic obstructive pulmonary disease (COPD); and four for isotretinoin for acne. These were appraised to identify themes specific to adverse effects reviews to form the basis of the questionnaire, including duration of follow up and methods for monitoring or eliciting adverse effects. The results of the questionnaire, subsequent discussions and the implications for quality assessment of these reviews will be presented.

Conclusions: Practical and valid quality assessment criteria for reviews of adverse effects would help both the authors and the users of reviews. Recommendations for development of this area will be made.