Background: Blinding is a key element of treatment evaluation in systematic reviews, and is considered more difficult to obtain in trials assessing psychosocial treatment than in those on pharmacotherapy.
Objectives: To evaluate the concordance rate and kappa values for agreement between the right allocation and the guessed allocation by blinded assessors in a randomized controlled trial (RCT) on psychotherapy. Factors associated with concordance between actual assignments and guessed assignments are also explored.
Methods: Data on both actual and guessed assignments by six blind assessors were obtained at four- and eight-week assessments in an RCT assessing the added value of brief behavioral therapy for insomnia over treatment as usual for residual depression and refractory insomnia in 37 adults at three psychiatric outpatient departments. The severity of depression was assessed by blind raters. Neither the patients nor physicians of TAU were blind to allocation. However, all patients were requested not to reveal their allocated treatment to the assessors for the severity of depression. After each assessment, an assessor guessed which group the patient had been assigned to, making it possible to examine whether the blinding was successful. Information about the degree of confidence in and reasons for guessing such was also gathered from the assessors.
Results: The concordance rates and kappa values for agreement between the actual allocation and the allocation guessed by blind assessor at each assessment were 56.7% and 0.15 at four weeks and 70.2% and 0.41 at eight weeks, respectively. This indicated that the blinding of the assessors was satisfactory. With regard to factors associated with the right allocation, only 'intuition' of the assessors was statistically significant (P value = 0.02).
Conclusions: Assessors in a trial on psychotherapy can be successfully blinded, where a sufficient effort is made.

Reference:
Watanabe N et al. Psychotherapy and Psychosomatics. 2013;82(6):401-3.