Article type
Year
Abstract
Background: Clinical data and measures that do not come from clinical trials (e.g. electronic health records, social media) bring additional information that may enhance traditional systematic reviews of the evidence. Current methods for critical appraisal cannot be applied to such data. Emerging methods for interpreting evidence, (e.g. crowdsourcing, automatic systematic reviews [1]), may provide opportunities to incorporate non-trial data whilst ensuring integrity and reliability.
Objective: To examine the opportunities and risks of integrating non-trial data into systematic reviews using novel methods and technologies.
Results: Non-trial data offer the following advantages:
- More data and longer follow-up [2]
- Better representation of healthcare services including comorbidities and settings
- Include non-drug therapies
- Data for circumstances where randomized controlled trials are unethical or infeasible
- Broader sets of patient outcomes
- Capture rare events
The following risks also exist:
- Lack of randomisation, control and unmeasured confounders introduce biases
- Self-reported data (e.g. from social media) may have selection bias
- Limited to retrospective analysis and probably unavailable for new therapies
These risks may be mitigated by:
- Transparent, objective and repeatable protocols in automatic systematic reviews [1]
- Crowdsourcing editorial services for robust fact-checking
- Control of bias using multiple datasets in lieu of multiple arms, and statistical methods that ensure significance of results
- Monitor conventional and social media for reports on harms or harmful sentiment (e.g. anti-vaccine) that may affect practice [3]
Conclusion: Non trial sources of clinical data are more prone to bias but provide crucial information unavailable from clinical trials. More research is needed on when and how non-trial data can be used in systematic reviews.
1. Tsafnat et al. The automation of systematic reviews. BMJ 2013;346:f139.
2. Gallego et al. Role of EHR in comparative effectiveness research. JCE 2013;2:529-32.
3. Zhou et al. Using social connection information to improve opinion mining. MedInfo 2015
Objective: To examine the opportunities and risks of integrating non-trial data into systematic reviews using novel methods and technologies.
Results: Non-trial data offer the following advantages:
- More data and longer follow-up [2]
- Better representation of healthcare services including comorbidities and settings
- Include non-drug therapies
- Data for circumstances where randomized controlled trials are unethical or infeasible
- Broader sets of patient outcomes
- Capture rare events
The following risks also exist:
- Lack of randomisation, control and unmeasured confounders introduce biases
- Self-reported data (e.g. from social media) may have selection bias
- Limited to retrospective analysis and probably unavailable for new therapies
These risks may be mitigated by:
- Transparent, objective and repeatable protocols in automatic systematic reviews [1]
- Crowdsourcing editorial services for robust fact-checking
- Control of bias using multiple datasets in lieu of multiple arms, and statistical methods that ensure significance of results
- Monitor conventional and social media for reports on harms or harmful sentiment (e.g. anti-vaccine) that may affect practice [3]
Conclusion: Non trial sources of clinical data are more prone to bias but provide crucial information unavailable from clinical trials. More research is needed on when and how non-trial data can be used in systematic reviews.
1. Tsafnat et al. The automation of systematic reviews. BMJ 2013;346:f139.
2. Gallego et al. Role of EHR in comparative effectiveness research. JCE 2013;2:529-32.
3. Zhou et al. Using social connection information to improve opinion mining. MedInfo 2015