A qualitative and visual assessment approach of evidence - exemplified by Xiao Chai Hu Tang formula for chronic hepatitis B

Article type
Authors
Kong DZ1, Liang N2, Yang GL3, Zhang Z4, Liu Y4, Liang SB2, Li J4, Liu XH4, Wang Y4, Jakobsen JC5, Gluud C6, Liu JP7
1Liaoning University of Traditional Chinese Medicine and Centre for Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine
2Beijing University of Chinese Medicine
3Liaoning University of Traditional Chinese Medicine
4Affiliated hospital of Liaoning University of Traditional Chinese Medicine
5Department of Cardiology, Holbæk Hospital
6Cochrane Hepato-Biliary Group, Copenhagen Trial Unit, Centre for Clinical Intervention Research
7Centre for Evidence-Based Chinese Medicine
Abstract
Background: evidence should have minimized threats to the internal and external validity in order to guide clinical practice. Both internal and external validity can be influenced by the risk of three dimensional errors (i.e. systematic errors, random errors, and design errors). However, we lack a qualitative and visual assessment approach to evaluate clinical evidence for the three dimensional errors.

Objectives: to provide a qualitative and visual assessment approach for clinical evidence. To assess the clinical evidence on a traditional Chinese medicine (TCM) intervention, Xiao Chai Hu Tang formula (XCHT), for chronic hepatitis B (CHB). To propose recommendations for improving clinical trial design and evidence quality.

Methods: we searched databases and trial registry platforms for published and ongoing studies on XCHT for CHB. We used the Assessment of Multiple Systematic Reviews Tool 2 to assess the quality of systematic reviews and meta-analyses. We constructed a visual evidence matrix for ranking the identified evidence to the three dimensional errors.

Results: two meta-analyses and 52 randomized clinical trials (RCTs) explored the benefits and harms of XCHT for CHB. Through the evidence matrix approach, we assessed one meta-analysis as being of moderate quality and the other of critically low quality due to high risks of systematic errors and design errors, but small risks of random errors. Neither meta-analysis was conducted as a systematic review. All the 52 RCTs showed high risks of systematic errors and design errors, and 28 out of 52 RCTs showed high risks of random errors, substantial risks of random errors, or moderate risks of random errors.

Conclusions: the evidence matrix approach provides an overview of the reliability of evidence and may assist in safe and more correct implementation of interventions in clinical practice. The evidence matrix approach is consistent with commonly used evidence quality assessments methods, and it may work as a good supplement to draw attention to random errors and design errors. The included evidence on XCHT for CHB showed high risks of systematic errors, random errors, and design errors. We proposed relevant recommendations for improving clinical evidence quality: we should strictly design, perform, and report important factors of systematic errors. Trials should have an adequate sample size and diversity of participants to minimize random errors. Trials should clearly define and report the diagnostic criteria, inclusion criteria, and exclusion criteria of participants. There should be more multi-center clinical trials, superiority studies, and pragmatic studies. Clinical research on TCM therapy needs the details of individualization and eligibility criteria to be clearly defined and reported for medical care practitioners.

Patient or healthcare consumer involvement: the retrieved evidence come from clinical studies which enrolled patients or healthcare consumers.