Article type
Abstract
Background and Objectives: Nearly half of Austrian internet users struggle to determine the objectivity and credibility of health information they find online. The study aimed to develop a checklist to assist laypeople in assessing the trustworthiness of online health information. To ensure the checklist is user-friendly, a diverse group of potential users was involved in the development process.
Methods and Results: We searched for existing checklists, tools and conceptual papers on health information quality assessment and extracted all items/indicators, grouped them into content categories, de-duplicated them and translated them into German. After six health information experts assessed the suitability and applicability of the items in a modified Delphi, the resulting 'short-list' was tested with 19 potential users in cognitive interviews. After revising the checklist and amending the answer scheme, we asked another group of 20 individuals to apply the revised draft checklist to evaluate a sample of 15 online health information webpages. The research team also applied the draft checklist to evaluate 100 health information webpages. Two researchers independently assessed the objective certainty of evidence for each health information.
The final checklist items were selected based on all results, including the qualitative results from laypersons, effect estimates of each item, inter-reliability measures among laypersons and experts, and between laypersons and experts. The final checklist comprises seven items: absence of advertising, balanced presentation of information, limited use of professional jargon, from an independent organisation, citation of sources, mention of scientific validation, and presence of a publication date.
The final checklist omitted certain items that experts considered crucial. For instance, it did not include involving the audience in the preparation of health information, clarifying the target audience for the health information (e.g. people with a specific condition), or indicating whether the text was reviewed by someone with appropriate scientific training.
Conclusions: By involving potential users in the checklist development process and equally considering their results alongside others, the final checklist became more accessible and useful for laypeople than it would have been without their involvement.
Methods and Results: We searched for existing checklists, tools and conceptual papers on health information quality assessment and extracted all items/indicators, grouped them into content categories, de-duplicated them and translated them into German. After six health information experts assessed the suitability and applicability of the items in a modified Delphi, the resulting 'short-list' was tested with 19 potential users in cognitive interviews. After revising the checklist and amending the answer scheme, we asked another group of 20 individuals to apply the revised draft checklist to evaluate a sample of 15 online health information webpages. The research team also applied the draft checklist to evaluate 100 health information webpages. Two researchers independently assessed the objective certainty of evidence for each health information.
The final checklist items were selected based on all results, including the qualitative results from laypersons, effect estimates of each item, inter-reliability measures among laypersons and experts, and between laypersons and experts. The final checklist comprises seven items: absence of advertising, balanced presentation of information, limited use of professional jargon, from an independent organisation, citation of sources, mention of scientific validation, and presence of a publication date.
The final checklist omitted certain items that experts considered crucial. For instance, it did not include involving the audience in the preparation of health information, clarifying the target audience for the health information (e.g. people with a specific condition), or indicating whether the text was reviewed by someone with appropriate scientific training.
Conclusions: By involving potential users in the checklist development process and equally considering their results alongside others, the final checklist became more accessible and useful for laypeople than it would have been without their involvement.