Introduction: Most systematic reviews of treatment interventions exclude non-randomized trials. Although non-randomized trials cannot provide good evidence for effectiveness the inclusion of such trials in a systematic review might provide useful information on prognosis, especially in case of chronic diseases and if evidence from randomized trials is insufficient or lacking.

Objectives: We aimed to investigate to what extent characteristics, quality and results of different study types on acupuncture treatment for idiopathic headaches differ.

Methods: We tried to identify all studies reporting clinical outcome data for acupuncture treatment for idiopathic headaches. Descriptive characteristics, quality aspects and proportions of patients improving after treatment were extracted. Summary estimates were calculated for different study types and studies meeting predefined quality criteria.

Results: A total of 61 studies was identified: 25 randomized trials, 5 non-randomized controlled trials, 9 uncontrolled prospective trials, 14 case series, and 8 cross-sectional studies. Randomized trials and prospective uncontrolled studies had significantly better quality than the other studies but both categories contained studies of completely insufficient quality. Proportions of patients reporting an improvement after acupuncture were significantly lower in randomized, prospective uncontrolled trials, and cross-sectional studies.

Discussion: In the study set investigated the quality of non-randomized controlled trials and case series was in average so low that it would not seem advisable to include such studies in a systematic review. Some prospective uncontrolled trials provided potentially relevant information on long-term prognosis which could complement data from RCTs.