Article type
Year
Abstract
Background: Electronic resources are increasingly used in the medical workplace, and some forms of knowledge translation technology (KTT) may facilitate education. [1,2] While dissemination of printed materials is unlikely to produce rapid and substantial changes in practice, the impact of computerized KTT is potentially greater by enabling clinicians to pursue more clinical questions than they otherwise would. [3,4] A good example of KTT is InfoRetriever for the handheld (PDA) and desktop computer. [5] InfoRetriever searches seven databases for primary care, one of which is the Cochrane Database of Systematic Reviews (abstracts only).
Objectives: To assess the use and impact of seven databases provided to family medicine residents via InfoRetriever (versions 2003 and 2004) on the PDA.
Methods: We tracked usage of InfoRetriever on the PDA using a built-in function to automatically record information "hits"when residents opened an item of information. With each hit, data on five variables were written to a log file on the PDA: a unique ID number, title of information, type of database, date and time of the hit. To reduce bias arising from human memory, usage data was linked to an impact assessment questionnaire on the PDA. This questionnaire prompted the resident to report the impact of information hits. (Figure 1) Log file data on the use and impact of information hits were analyzed prior to a semi-structured interview of each resident. In this presentation, we descriptively report on the use and impact of information hits by database type (e.g. Cochrane abstracts, guideline summaries, POEMs, clinical rules and calculators), as derived from the impact assessment questionnaire.
Results: 25 residents documented 3,016 information hits between September 26 2003 and March 10 2004. Linked to these hits, we received 1,460 electronic reports of impact, most frequently in the areas of "learning" and "practice improvement". In comparison to the clinical rules database, information hits in the text-only databases (e.g. 5 Minute Clinical Consult, Cochrane, POEMs and guideline summaries) are more frequently associated with learning and recall of knowledge. Information hits in the clinical rules database are more frequently associated with reports of practice improvement in primary care. Residents are more likely to report no impact after hits in the text-only databases compared to the clinical rules database. Frustration is infrequently reported (3.7% of all information hits).
Conclusions: Emerging from this study-in-progress is a promising generic method to measure the impact of information retrieval in medicine.
Acknowledgements: Funded by the Population and Public Health Institute of the Canadian Institutes of Health Research.
References: 1. Criswell DF, Parchman ML. Handheld computer use in U.S. family practice residency programs. J Am Med Inform Assoc. 2002;9(1):80-6. 2. Leung GM, Johnston JM, Tin KYK et al. Randomised controlled trial of clinical decision support tools to improve learning of evidence based medicine in medical students. BMJ. 2003;327:1090. 3. Farmer AP, Legare F, McAuley LM, Thomas R, Harvey EL, McGowan J, Grimshaw JM, Wolf FM.. Printed educational materials: effects on professional practice and health care outcomes (Protocol for a Cochrane Review). In: The Cochrane Library, Issue 2, 2004. Chichester, UK: John Wiley & Sons, Ltd. 4. Hersh WR. Information retrieval: A health and biomedical perspective. New York: Springer; 2003. 5. Ebell MH, Messimer SR, Barry HC. Putting computer-based evidence in the hands of clinicians [letter; comment]. JAMA. 1999;281(13):1171-2.
Objectives: To assess the use and impact of seven databases provided to family medicine residents via InfoRetriever (versions 2003 and 2004) on the PDA.
Methods: We tracked usage of InfoRetriever on the PDA using a built-in function to automatically record information "hits"when residents opened an item of information. With each hit, data on five variables were written to a log file on the PDA: a unique ID number, title of information, type of database, date and time of the hit. To reduce bias arising from human memory, usage data was linked to an impact assessment questionnaire on the PDA. This questionnaire prompted the resident to report the impact of information hits. (Figure 1) Log file data on the use and impact of information hits were analyzed prior to a semi-structured interview of each resident. In this presentation, we descriptively report on the use and impact of information hits by database type (e.g. Cochrane abstracts, guideline summaries, POEMs, clinical rules and calculators), as derived from the impact assessment questionnaire.
Results: 25 residents documented 3,016 information hits between September 26 2003 and March 10 2004. Linked to these hits, we received 1,460 electronic reports of impact, most frequently in the areas of "learning" and "practice improvement". In comparison to the clinical rules database, information hits in the text-only databases (e.g. 5 Minute Clinical Consult, Cochrane, POEMs and guideline summaries) are more frequently associated with learning and recall of knowledge. Information hits in the clinical rules database are more frequently associated with reports of practice improvement in primary care. Residents are more likely to report no impact after hits in the text-only databases compared to the clinical rules database. Frustration is infrequently reported (3.7% of all information hits).
Conclusions: Emerging from this study-in-progress is a promising generic method to measure the impact of information retrieval in medicine.
Acknowledgements: Funded by the Population and Public Health Institute of the Canadian Institutes of Health Research.
References: 1. Criswell DF, Parchman ML. Handheld computer use in U.S. family practice residency programs. J Am Med Inform Assoc. 2002;9(1):80-6. 2. Leung GM, Johnston JM, Tin KYK et al. Randomised controlled trial of clinical decision support tools to improve learning of evidence based medicine in medical students. BMJ. 2003;327:1090. 3. Farmer AP, Legare F, McAuley LM, Thomas R, Harvey EL, McGowan J, Grimshaw JM, Wolf FM.. Printed educational materials: effects on professional practice and health care outcomes (Protocol for a Cochrane Review). In: The Cochrane Library, Issue 2, 2004. Chichester, UK: John Wiley & Sons, Ltd. 4. Hersh WR. Information retrieval: A health and biomedical perspective. New York: Springer; 2003. 5. Ebell MH, Messimer SR, Barry HC. Putting computer-based evidence in the hands of clinicians [letter; comment]. JAMA. 1999;281(13):1171-2.