JCO PO Article Insights: Analytical Validation of Tumor-Informed ctDNA Assays for MRD
In this JCO PO Article Insights episode, host Jordan Goldstein summarizes the article, "Generic Protocols for Analytical Validation of Tumor-Informed Circulating Tumor DNA Assays for Molecular Residual Disease: The Blood Profiling Atlas in Cancer's Molecular Residual Disease Analytical Validation Working Group Consensus Recommendation" by Baden et al. TRANSCRIPT Jordan Goldstein: Hello, and welcome to JCO Precision Oncology Article Insights. I'm your host, Jordan Goldstein from Stanford University. Today we're discussing a consensus recommendation published in JCO Precision Oncology titled "Generic Protocols for Analytical Validation of Tumor-Informed Circulating Tumor DNA Assays for Molecular Residual Disease" by lead author Jonathan Baden, senior author Lauren Leiman, and colleagues on behalf of the BLOODPAC Consortium. The liquid biopsy space is one of the most exciting frontiers in oncology right now, with rapid development and many potential uses. However, the field has really lacked a shared framework for how these assays should actually be validated. This paper attempts to solve part of that problem. Before going further, I want to mention that BLOODPAC stands for Blood Profiling Atlas in Cancer Consortium. This was developed in 2016 with the goal of accelerating liquid biopsy development through shared standards. It includes the leading cancer diagnostics companies alongside academics, pharmaceutical companies, not for profits, and regulatory agencies. So, what actually is ctDNA MRD, and why does it matter? ctDNA is short for circulating tumor DNA, which is DNA shed by tumors into the bloodstream and can be detected by genomic profiling of a simple blood draw. Compared to tissue biopsies, it's minimally invasive, easily accessible, and can reflect the genetic diversity of the entire tumor across anatomic sites. This can allow for comprehensive genomic profiling, identifying target mutations, and understanding anatomic heterogeneity prior to treatment. It also allows for repeated sampling during and after treatment to explore evolutionary dynamics and, most promisingly, to detect molecular residual disease or MRD, which is what we focus on in this article. MRD is the presence of tumor-derived DNA in blood following therapy at levels below the threshold of conventional imaging or standard pathologic assessment. Accurate MRD detection can transform therapeutic strategies, enabling more precise risk-adapted approaches. But detecting MRD is not simple. There's often a very small amount of tumor DNA in plasma after treatment, even going below one part per million or 0.0001% of the total circulating DNA, most of which is healthy, normal, cell-free DNA. Detecting a signal that faint, reliably and reproducibly, is quite technically demanding. Tumor-informed ctDNA assays address this by first sequencing the patient's primary tumor to identify somatic variants that are unique to that cancer. A personalized panel is then constructed to track those exact variants in serial blood samples. This allows greater sensitivity. However, the methods and protocols for pre-analytical, analytical, and clinical validation for tumor-informed MRD assays can vary greatly. This presents major challenges for regulatory approval and clinical implementation. With these consensus recommendations in this article, BLOODPAC focuses on developing a standardized framework for the analytical validation of any tumor-informed ctDNA MRD assay. Analytical validation ensures these assays are in fact measuring what they claim to measure with defined performance characteristics. BLOODPAC intentionally set out to keep their protocols as generic as possible with the only requirements being intended uses of the assay for: one, patients with cancer who have undergone curative-intent therapy; and two, for prognosis, treatment efficacy, detection of residual disease or recurrence, or serving as the basis for a novel clinical trial strategy. With the goal of accelerating the clinical development and validation of tumor-informed MRD assays, BLOODPAC worked closely with the FDA throughout this process, holding three separate pre-submission meetings, precisely to ensure that assay developers who follow these protocols are well positioned for regulatory approval. Now, let's delve deeper into this paper and highlight the significant challenges of validating tumor-informed MRD assays. These challenges primarily stem from the low concentration of ctDNA found in the bloodstream. These levels can be further impacted by tumor characteristics such as tumor type, heterogeneity, histology, size, stage, and cell turnover or proliferative rate that can impact ctDNA shedding. One complex problem here is sampling heterogeneity or stochastic variation when looking for a single specific variant. When ctDNA is extremely low, a given variant may be present in one blood draw but may be absent in a replicate taken at the same time point, not because the biology changed, but due to random sampling effects at low levels. Tumor-informed assays handle this by evaluating MRD at the sample level rather than at the variant level. If enough variants from the personalized panel are detected collectively, the sample is called positive, even if no single variant is consistently detected. This is a strength for sensitivity, but it complicates traditional validation designs that assume consistent variant level assessments. Additionally, as we previously discussed, tumor-informed assays use personalized panels of mutations unique to the tumor to their advantage to improve their sensitivity. They filter out normal germline variants and non-tumor-derived somatic variants such as those from clonal hematopoiesis or CHIP. This leads to a smaller but highly specific assay. The smaller panel enables more targeted, deeper sequencing, focused on the most likely tumor-derived variants, and reduces the risk of false positives. However, the personalized nature also makes validation difficult because each patient's panel is different. For this, novel approaches are needed to really validate that key performance measures are acceptable and consistent. A final challenge here is the blood sample volume required for ctDNA detection at low levels. A standard blood draw simply doesn't yield enough ctDNA to support the extensive replication and dilution series that conventional analytical validation requires. To address this, test developers can use contrived samples, synthetic DNA sequences from a known cancer patient spiked at defined allele frequencies into healthy donor plasma. These serve as a surrogate for true clinical material when volumes are constrained. The test developer should then perform a contrived sample functional characterization study to demonstrate to the FDA that the contrived samples actually perform equivalently to real clinical specimens. So now that we've actually covered some of the major challenges here, let's dig into the analytical validation protocols that are recommended for the seven key performance characteristics that are defined in this paper. The first two performance characteristics, limit of blank and limit of detection, focus on establishing the analytical performance of the assay, which is the assay's ability to detect a known signal when present in the sample or vice versa. To be clear, this is distinct from the clinical performance, which is defined by the assay's ability to correctly identify patients who do or do not have residual cancer and ultimately relapse. The first performance metric is limit of blank or LOB. This is the analytical specificity, or the highest signal expected in a sample that does not contain tumor DNA. To establish this, BLOODPAC suggests using a minimum of 60 blank samples from healthy donors, in a minimum of two replicates with two reagent lots and multiple panel designs across a range of DNA inputs. Then, the LOB should be set to zero and 60 blank samples should again be tested to determine the false positive rate on a per-sample basis. Typically, the LOB represents the 95th percentile of signal observed in samples without tumor variants, also known as the background signal. The limit of detection or LOD, on the other hand, is the analytical sensitivity, or the lowest concentration of tumor-derived molecules that can be reliably detected in a sample. To establish the LOD, an appropriate number of low allele fraction contrived positive samples or specimen blend panels should be tested in five different dilution levels with a minimum of 10 replicates per dilution level. Two reagent lots must be used with each testing across these 50 measurements. If developers are to use a probit regression model approach to determine the LOD, they should use at least 100 of these measurements. The LOD is ultimately determined as the tumor concentration corresponding to a 95% hit rate. The next two performance metrics prove the accuracy and precision of the assay. The analytical accuracy is how often the assay correctly identifies positive and negative samples. This should be determined by testing a minimum of 100 specimens, ideally from a clinical trial or procured from clinical care that are known to be negative for ctDNA, as well as 10 to 20 cancer positive samples. From this, the sample level percent positive, percent negative, and overall agreement can be used to determine the accuracy. Precision of the assay is determined in two different ways: repeatability and reproducibility, using true patient samples. Repeatability assesses the intra-assay precision. It is measured by assessing the consistency of the assay under the same operating conditions over a short period. For this, multiple samples are tested in replicates of two, using a single operator and single testing site with a minimum 20-day testing interval for 80 total observations. Reproducibility, on the other hand, measures the inter-assay precision. This evaluates the assay's performance across different variables to ensure stable results. To validate this, positive samples at or near the LOD and at least one negative sample need to be included. Then, a minimum of two operators, three manufacturer reagent lots, and two technical replicates per sample per run must be assessed again spanning at least a 20-day interval. Agreement metrics are calculated based on the assay's binary output, detected or undetected, for both to establish the precision of the assay. The final three performance characteristics focus on demonstrating the durability of the assay under stress. This includes measuring interfering substances, robustness, and the prepared specimen stability. All of these can be validated using pooled patient or contrived samples. Interfering substances are evaluated by spiking known clinical interference into ctDNA positive and negative samples. Robustness, also known as guard banding, intentionally perturbs critical steps in collection or NGS processing to validate operational guardrails. Stability testing evaluates specimen viability using three positive specimens near the LOD and one negative baseline at 3-month intervals up to the desired stability claim. You may have noticed that analytical validation for each of these seven performance characteristics requires multi-variable studies varying tumor fraction, lots, operators, and instruments and would demand massive sample volumes that go far beyond what's clinically realistic. BLOODPAC addresses this by endorsing fractional factorial designs. These reduce the number of replicates per clinical sample by modeling performance characterization of the tumor-informed MRD assay across a spectrum of tumor fractions and inputs. When sample yields are too low for multiple replicates of a sample, using more diverse clinical samples can allow maintenance of degrees of freedom and statistical robustness without exhausting the material. It's worth noting that analytical validation is just one very important piece of the puzzle. Many other aspects can affect ctDNA assay performance that are not addressed in this paper including pre-analytical aspects such as sample collection, genomic profiling, bioinformatics software, and CHIP processing, as well as clinical aspects, which also require adequate testing and validation. Ultimately, these consensus recommendations from BLOODPAC for analytical validation of tumor-informed ctDNA MRD assays are much needed. They should lead to faster clinical validation and regulatory approval, getting these vital, highly sensitive MRD assays into clinical trials and ultimately to patients. For clinicians, it provides a concrete lens for evaluating the performance claims of commercially available ctDNA MRD assays. If you're interested in learning more about the details of these protocols for analytical validation, I highly encourage you to read the full article at JCO Precision Oncology. Thank you for joining me at JCO Precision Oncology Article Insights. Please subscribe and join us next time as we explore groundbreaking research shaping the future of precision oncology. The purpose of this podcast is to educate and to inform. This is not a substitute for professional medical care and is not intended for use in the diagnosis or treatment of individual conditions. Guests on this podcast express their own opinions, experience, and conclusions. Guest statements on the podcast do not express the opinions of ASCO. The mention of any product, service, organization, activity, or therapy should not be construed as an ASCO endorsement.
