Article type
Year
Abstract
Background: overlap of primary studies (PS) is a challenge and a source of bias when conducting an overview. It refers to the inclusion of the same PS in two or more included systematic reviews (SR). Assessment and consideration of overlap leads to a better interpretation of findings across SR. The degree of overlap for an overview is usually assessed as the corrected covered area (CCA) of a customised matrix of evidence (Figure 1). Usually CCA refers only to the overall overlap for the overview, and it does not discriminate which pairs of reviews present more overlap. Such a feature would be desirable to refine the approach of SRs in overviews.
Objective: to provide an overlap assessment tool that allows visual representation of the degree of overlap of PS between each pair of SR in an overview, based on the CCA formula.
Methods: we developed a matrix of evidence using Excel, with each included SR in the columns and all PS in the rows. We calculated the CCA for the matrix, and also for each pair of SR. To do this, we applied the CCA formula to a matrix that considered two columns. The spreadsheet repeated this calculus automatically for each possible pair of SR. Then, we presented the results in a visual representation in the form of a triangle, showing the CCA result for a customised matrix for each pair of SR. We used traffic light colours to highlight pairs of SR with more overlap percentages. We are currently using this method in four overviews, for which we have obtained the overall CCA and the specific CCA for each pair of SR. To avoid misinterpretation of findings, we decided that if any pair of SR had more than 25% overlap between each other, we would only analyse data from the SR with better methodological quality. Also, we planned to integrate these results in our discussion section.
Results: we present the results of the type 2 diabetes mellitus overview. This overview included 53 SR comprising 1031 unique PS (Figure 2). Overall CCA was 0.55% (slight overlap). The graphical output showed three pairs of SR with CCA above 25%, five pairs between 15% to 25%, seven pairs between 10% to 15%, and 40 between 5% to 10%. Figure 3 presents part of this graphical output of the CCA assessment for each pair of SR. With these results, three SR will not be considered for the analysis in our overview, and we will carefully discuss the results obtained by other SRs showing high overlap.
Conclusions: graphical representation of overlap using CCA formula for each pair of included SR is an easy and pragmatic method to detect pairs of SRs with high overlap, even in scenarios with low overall degree of overlap. This approach could help authors of overviews to make better decisions about how to deal with overlap at the moment of defining inclusion criteria, data extraction, analysis or discussing the results. We will soon publish the Excel file used.
Patient or healthcare consumer involvement: no patients were involved in this research.
Objective: to provide an overlap assessment tool that allows visual representation of the degree of overlap of PS between each pair of SR in an overview, based on the CCA formula.
Methods: we developed a matrix of evidence using Excel, with each included SR in the columns and all PS in the rows. We calculated the CCA for the matrix, and also for each pair of SR. To do this, we applied the CCA formula to a matrix that considered two columns. The spreadsheet repeated this calculus automatically for each possible pair of SR. Then, we presented the results in a visual representation in the form of a triangle, showing the CCA result for a customised matrix for each pair of SR. We used traffic light colours to highlight pairs of SR with more overlap percentages. We are currently using this method in four overviews, for which we have obtained the overall CCA and the specific CCA for each pair of SR. To avoid misinterpretation of findings, we decided that if any pair of SR had more than 25% overlap between each other, we would only analyse data from the SR with better methodological quality. Also, we planned to integrate these results in our discussion section.
Results: we present the results of the type 2 diabetes mellitus overview. This overview included 53 SR comprising 1031 unique PS (Figure 2). Overall CCA was 0.55% (slight overlap). The graphical output showed three pairs of SR with CCA above 25%, five pairs between 15% to 25%, seven pairs between 10% to 15%, and 40 between 5% to 10%. Figure 3 presents part of this graphical output of the CCA assessment for each pair of SR. With these results, three SR will not be considered for the analysis in our overview, and we will carefully discuss the results obtained by other SRs showing high overlap.
Conclusions: graphical representation of overlap using CCA formula for each pair of included SR is an easy and pragmatic method to detect pairs of SRs with high overlap, even in scenarios with low overall degree of overlap. This approach could help authors of overviews to make better decisions about how to deal with overlap at the moment of defining inclusion criteria, data extraction, analysis or discussing the results. We will soon publish the Excel file used.
Patient or healthcare consumer involvement: no patients were involved in this research.