Low quality corneal disease systematic reviews limit the amount of usable evidence for informing practice guidelines

Article type
Authors
Ng SM1, Qureshi R1, Leslie L1, Yim TW1, Liu S1, Li T1
1Department of Ophthalmology, School of Medicine, University of Colorado Anschutz Medical Campus
Abstract
Background: High-quality systematic reviews and meta-analyses (SRMAs) underpin trustworthy clinical practice guidelines. Cochrane Eyes and Vision US Project (CEV@US) maintains a database of SRMAs in eyes and vision to support the development and update of practice guidelines in ophthalmology and optometry in the United States and worldwide. Previous work by CEV@US has shown that a majority of SRMAs in many eye conditions are unreliable.

Objectives: To summarize the reliability of SRMAs in ocular surface diseases published between 2017 and 2022.

Methods: SRMAs of cornea-related interventions (2017-2022) excluding dry eye were searched through the CEV@US database. The reliability of SRMAs was assessed by one reviewer and verified by another based on a set of minimal criteria (Table 1). A single reviewer assessed adherence to SRMA reporting guidelines and registration.

Results: We assessed 221 SRMAs of cornea-related interventions, of which 87 (39%) were judged as reliable. Of the remaining 134 unreliable SRMAs, 27 (20%) failed to describe eligibility criteria, 83 (62%) used only one database or unrefined search strategies, 64 (48%) did not assess risk of bias of included studies, 50 (37%) did not conduct meta-analysis appropriately, and conclusions from 34 (25%) were discordant with their findings. Additionally, 26 (19%) SRMAs had other concerns: predominantly financial conflicts. A total of 96 (43%) of the 221 SRMAs stated that they followed relevant reporting guidelines, but 53/96 (55%) were judged as unreliable. Of 200 non-Cochrane SRMAs, 34 (17%) were prospectively registered in the International Prospective Register of Systematic Reviews (PROSPERO) or International Platform of Registered Systematic Review and Meta-analysis Protocols (INPLASY), whereas all 21 Cochrane reviews were prospectively registered and published (Figure 1).

Conclusions: Nearly 60% of the published SRMAs in ocular surface disease over the past 5 years were unreliable, primarily because they failed to conduct comprehensive literature searches or assess the risk of bias of included studies. The methodology of future SRMAs should be well-planned, preregistered and clearly reported to produce trusted evidence and avoid research waste.

Patient, public and/or healthcare consumer involvement: Patients or healthcare consumers were not directly involved in this study. The findings, however, may indirectly contribute to them to produce more trusted evidence in healthcare.