Challenges in conducing systematic reviews for diagnostic and prognostic testing in genomics and personalized medicine (GPM)

Tags: Oral
Lin J1, Whitlock E2, Webber B2, Lai S2, Holmes R2
1Oregon EPC, Kaiser Center for Health Research, Portland, Oregon, United States, 2Kaiser Center for Health Research, Portland, Oregon, United States

Background: GPM is a rapidly growing field that spans multiple applications, e.g., screening, diagnostic, prognostic, pharmacogenomic. GPM tests, however, generally lack readily-available population-based evidence about clinical utility and net health impacts to support informed decision-making. Objective: To describe the important methodological challenges to reviewers in the field of GPM given the differences in the development of evidence in genomic testing. Methods: We are working with the CDC Office of Public Health Genomics and the Evaluation of Genomic Applications in Practice and Prevention (EGAPP) Working Group to develop systematic review methods in GPM, using a systematic review of epidermal growth factor receptor (EGFR) related tests in guiding monoclonal antibody treatment in colorectal cancer as a case example. Results: Building on the work of the USPSTF and EGAPP, we developed an analytic framework to help reviewers and decision makers conceptualize and operationalize important questions when synthesizing and considering the evidence for a genomic test (see Figure). This framework helps reviewers understand the differences in analytic validity, clinical validity and clinical utility, as established by EGAPP. This framework also includes important questions around harms, systematically addressing both serious adverse effects requiring clinical attention, as well as ethical, legal, and social issues that concern patients. Evidence generation for genetic testing is complicated by a weak regulatory environment, strong proprietary interests, and direct-to-consumer marketing. Therefore, much of the evidence is either unpublished or proprietary. We are piloting horizon scanning activities that will help us develop consistent search strategies for grey literature and determine when to update searches. We are developing minimum standards for identifying proprietary data, most relevant for reviewing analytic validity. Analytic validity will be important to consider, for example, if there is concern about reproducibility of test across platforms or laboratories, or issues around test performance and optimal diagnostic thresholds. Conclusions: Given the methodologic challenges unique to synthesizing the evidence in the clinical benefit from genetic testing, systematic reviewers interested in this growing field need to collaborate with one another to aid the development of the science of reviewing GPM applications. 1. Does the use of genetic prognostic markers (BRAF/PIK3CA/PTEN) in patients with metastatic colorectal cancer (mCRC) for decisions related to anti-EGFR use or dosing decisions reduce morbidity and/or mortality compared to those who are not tested? 2. How are persons with metastatic colorectal cancer (mCRC) who would be eligible for use of genetic testing to determine cancer treatment decisions using anti-EGFR antibodies identified in clinical care? 3.What genetic prognostic markers (BRAF/PIK3CA/PTEN) related to use of anti-EGFR antibodies have been developed in mCRC? What tests (including consideration of manufacturer, mutations/codons, and method or technology) are in use for each marker? In what treatment settings (i.e., first line metastatic treatment, second line metastatic treatment, neoadjuvant treatment, other) are they applied? For each of these genetic tests in each treatment setting, what is their analytic validity (i.e., sensitivity and specificity for detecting the genetic mutations of interest)? 4. For each of these genetic tests in each treatment setting, what is their clinical validity (i.e., how well do they predict non-response to anti-EGFR antibody treatments as measured by disease progression/tumor growth?) Do factors such as race/ethnicity, diet, or other medications, affect these associations? 5. For each of these genetic tests in each treatment settings, what is their clinical utility (i.e., do treatment choices or management decisions related to not using anti-EGFR antibodies based on these test results lead to improvement in all-cause mortality, colorectal cancer-specific mortality, progression-free survival, reduced morbidity (e.g. reducing harms by avoiding exposure to ineffective treatments), and/or quality of life, compared with treatment choices or management decisions made in the absence of this prognostic information)? (a) Are these tests useful in medical, personal, or public health decision making? 6. How strong is the association between tumor response as measured by imaging studies and health outcomes and over what time period has it been established? 7. What are the harms of performing genetic tests (i.e. BRAF/ PIK3CA/PTEN) related to anti-EGFR therapy for patients, including incorrect genotype assignment leading to potential use of ineffective treatments, ELSI risks, and other risks associated with this testing? 8. What are the harms of decisions surrounding the use or non-use of anti-EGFR therapy informed by the results of genetic testing compared to treatment management decisions made without this prognostic information, including delayed treatment or other concerns?