Article type
Year
Abstract
Objectives: To identify sources of heterogeneity in systematic reviews of diagnostic studies and to discuss the meaning of heterogeneity, its effect on the estimators of diagnostic validity.
Methods: Exploration of heterogeneity in two recent diagnostic systematic reviews by meta-regression of the diagnostic odds ratio (DOR).
Results: All 11 studies in a systematic review of the straight leg raising test were observational case-series; only two had a bi-directionally blind interpretation of the tests. Verification bias was possible present in all studies, but clear in one study. The log(DOR) was homogeneous after exclusion of this study (DerSimonian&Laird test of heterogeneity, p 0.2). The log(DOR) was associated with exclusion of previous surgery (6 studies, p 0.062). Year of publication was also negatively associated with the log(DOR) (p 0.02). In a systematic review of 48 studies on the use of the dipstick for detection of urinary infection, 44% of the variation of the log(DOR) (unweighted) for nitrates could be explained by the cut-point, type of dipstick, exclusion criteria and executor of the test. Verification bias was present in one study, but the log(DOR) was not different with the other studies. Bi-directionally blind interpretation of the tests was done in 20 studies, without consequences for the log(DOR).
Discussion: There seems to be less variation in the design compared to other domains of research, as the majority of diagnostic studies are observational cohorts. Estimation of validity is mostly limited to sensitivity and specificity. Yet clinical heterogeneity remains important. Statistical heterogeneity of sensitivity and/or specificity can be overcome by using the log diagnostic odds ratio. Sources of heterogeneity in execution of the study and in study population can be explored by meta-regression. Unexplained heterogeneity can be adjusted for by random effect pooling. Weighted random effect pooling by the inverse of the variance gives small studies relatively more weight and these are more prone to publication bias. This is probably a bigger problem in diagnostic research. More data will be presented and related methodological issues will be discussed during the meeting.
Methods: Exploration of heterogeneity in two recent diagnostic systematic reviews by meta-regression of the diagnostic odds ratio (DOR).
Results: All 11 studies in a systematic review of the straight leg raising test were observational case-series; only two had a bi-directionally blind interpretation of the tests. Verification bias was possible present in all studies, but clear in one study. The log(DOR) was homogeneous after exclusion of this study (DerSimonian&Laird test of heterogeneity, p 0.2). The log(DOR) was associated with exclusion of previous surgery (6 studies, p 0.062). Year of publication was also negatively associated with the log(DOR) (p 0.02). In a systematic review of 48 studies on the use of the dipstick for detection of urinary infection, 44% of the variation of the log(DOR) (unweighted) for nitrates could be explained by the cut-point, type of dipstick, exclusion criteria and executor of the test. Verification bias was present in one study, but the log(DOR) was not different with the other studies. Bi-directionally blind interpretation of the tests was done in 20 studies, without consequences for the log(DOR).
Discussion: There seems to be less variation in the design compared to other domains of research, as the majority of diagnostic studies are observational cohorts. Estimation of validity is mostly limited to sensitivity and specificity. Yet clinical heterogeneity remains important. Statistical heterogeneity of sensitivity and/or specificity can be overcome by using the log diagnostic odds ratio. Sources of heterogeneity in execution of the study and in study population can be explored by meta-regression. Unexplained heterogeneity can be adjusted for by random effect pooling. Weighted random effect pooling by the inverse of the variance gives small studies relatively more weight and these are more prone to publication bias. This is probably a bigger problem in diagnostic research. More data will be presented and related methodological issues will be discussed during the meeting.