Bioinformatics, Sequencing Accuracy, and the Credibility of Clinical Genomics

“The adoption of clinical exome and whole-genome sequencing based on next-generation sequencing technologies has increased rapidly over the last decade; this has been accelerated by increasing coverage of these services by private and public insurers. Examples of use include tumor and germline sequencing in patients with cancer, rapid turn-around sequencing of the genomes of critically ill neonates to diagnose mendelian conditions, and noninvasive prenatal testing for reproductive decision-making. The accuracy of sequencing results is of paramount importance to patients, clinicians, and those paying for testing services; inaccuracy can affect not only the tested individual, but their extended biological family. Understanding what accuracy means in the context of genome sequencing is a challenge.

[..] There are 3 important take-home points from this report. First, performance of any molecular diagnostic laboratory providing genome sequencing depends at least in part on the bioinformatics algorithms used in the sequencing process. The choice of algorithm can be a source of differential reporting of life-altering test results for patients among clinical laboratories running the same sequencing machines provided with the same sample for analysis.

Second, there is work to be done to arrive at a reference standard for clinical variant detection across the genome. Until such a standard is developed, measuring the performance of any given sequencing pipeline remains challenging. The creation of variably reproducible genome sequence data influenced by the methods and expertise of the individual testing laboratory is a limitation to the field. There is an absolute truth in each affected individual’s genome and reputable clinical laboratories should be reliably concordant for pathogenic findings. At a minimum, laboratories should be required to provide some metric of performance interpretable by nongeneticist clinicians of their process in comparison with similar laboratories, for similar indications for testing.

Third, the concept of “improved” variant detection is subjective. In this study the deep learning approach was associated with detection of a higher number of manually validated deleterious variants vs the standard method. However, the spectrum of variants detected by the 2 methods was not wholly overlapping, eg, while the deep learning method found variants that the standard method did not, the converse was also true. If the true pathogenic variant for an affected individual resided in the set of variants detected by the standard method but was not detected by the deep learning approach, the deep learning approach was not an improvement. The authors acknowledged that sequential use of several algorithms would provide the highest sensitivity.

At this time in the US, clinical genome sequencing remains largely unregulated and accuracy is highly dependent on the expertise of individual testing laboratories. To help ensure accuracy, the US Centers for Medicare & Medicaid Services requires, under its national coverage determination for genome sequencing in cancer care, that genome sequencing be performed in laboratories certified under the Clinical Laboratory Improvement Amendments (CLIA). CLIA provides a framework for ensuring basic quality of laboratory services in the US but is not specific to genomic testing, nor does it address clinical validity.

[..] Professional societies such as the American College of Medical Genetics and Genomics, the Association of Molecular Pathologists, and the College of American Pathologists are also actively working to ensure the accuracy of clinical genome sequencing. Published professional guidelines are not proscriptive and leave considerable discretion to clinical laboratories regarding details of how exome or whole-genome testing is conducted. Given that sequencing methods and bioinformatics approaches are rapidly evolving, this seems prudent. However, this same discretion allows for heterogeneity across institutions. As the study by AlDubayan et al highlights, decisions about the software used to analyze genome data can have profound effects on molecular diagnostic findings with potentially life-altering consequences.

The genomics community needs to act as a coherent body to ensure reproducibility of outcomes from clinical genome or exome sequencing, or provide transparent quality metrics for individual clinical laboratories. Issues related to achieving accuracy are not new, are not limited to bioinformatics tools, and will not be surmounted easily. However, until analytic and clinical validity are ensured, conversations about the potential value that genome sequencing brings to clinical situations will be challenging for clinical centers, laboratories that provide sequencing services, and consumers. For the foreseeable future, nongeneticist clinicians should be familiar with the quality of their chosen genome-sequencing laboratory and engage expert advice before changing patient management based on a test result.”

Full editorial, Feero WG. JAMA 2020.11.17