Combining Qualitative and Quantitative Information in Reviews

Tags: Oral
Braunholtz D, Lilford R, Chard J

Introduction: When making a decision, such as whether to treat a patient with a drug or not, directly relevant quantitative evidence is most useful in evaluating the likely balance of costs (i.e. side effects) and benefits. However, qualitative evidence (in which we include indirectly relevant quantitative evidence) can, and should, also inform and influence decisions, especially when quantitative evidence is scarce. If the function of systematic reviews is to provide decision-makers with a succinct and complete summary of relevant evidence, then qualitative evidence is clearly relevant.

Discussion: If qualitative research is available, the question arises of how to take it into account, together with any quantitative evidence, when making policy decisions. We argue that this should be done explicitly: first the decision-maker should 'convert' the qualitative data into a quantitative 'prior' belief about the true value of the key parameter on which the decision turns. Bayes' law can then be used to combine this prior with quantitative results to produce a 'posterior' quantitative belief about the key parameter - which (along with utilities for the outcomes) is what is required for a decision analysis, or less formal decision making. Making the whole process explicit means each stage can be examined, and criticised or supported. Any non-explicit alternative can only be supported or rejected as a whole, which does not help clarify the arguments. The implications for systematic reviewers include:

* Where quantitative evidence is plentiful, it will usually 'overwhelm' any qualitative evidence, so there is no need to collect the latter.

* Where quantitative evidence is scanty, qualitative evidence should be collated and presented in easily assimilated form.

* Reviewers could present examples of priors based on the qualitative evidence (ranging from very sceptical to very enthusiastic), combining these with the quantitative evidence to produce a range of posteriors. Readers could then decide which of the priors most closely fits their beliefs on reading the qualitative evidence - and hence which posterior is most appropriate for their use in decision making