VEEVA APPROVED Run 14 Variant Classification EQA Summary Webinar recording

This webinar provides an in-depth summary and analysis of the outcomes from Run 14 of the BRCA variant classification External Quality Assessment (EQA) scheme, co-hosted by EMQN CIC and GenQA. The primary goal of the EQA is to assess and improve the competency of clinical scientists and molecular pathologists globally in classifying variants within BRCA1, BRCA2, and other Homologous Recombination Repair (HRR) genes, such as ATM. The session, funded by AstraZeneca and MSD, highlights the critical challenges and discrepancies encountered when applying complex, evolving variant classification guidelines, specifically focusing on the differences between ClinGen's Variant Curation Expert Panel (VCEP) guidelines (e.g., ATM VSEP, BRCA VSEP) and the UK-specific Canvig guidelines.

The assessment utilized the Genie platform, GenQA’s genomic online education platform, and involved 359 participants from 58 countries. A key finding was the significant improvement in performance compared to previous runs, with 50% of participants correctly classifying all six variants—the highest percentage of accurate classifications recorded to date. The webinar detailed the classification of six specific variants spanning Class 1 (benign) through Class 5 (pathogenic), covering missense, frameshift, and deletion variants. A major theme was the variability introduced by different guideline combinations. For instance, the majority of participants (40%) used a three-way combination of guidelines (ACMG 2015, UK ACGS 2020, and Canvig UK 2020), demonstrating a lack of universal standardization, which leads to classification discrepancies even when the clinical outcome remains the same (e.g., benign vs. likely benign).

Detailed discussions focused on the nuanced application of specific ACMG/ClinGen criteria, such as PVS1 (very strong evidence for loss-of-function), PM3 (co-occurrence with a pathogenic variant), BS1 (allele frequency in controls), and BP1 (missense variant outside a key functional domain). For example, the analysis of an ATM missense variant (Variant 1) highlighted the ambiguity in applying PM2 (absence in control databases) due to conflicting recommendations between the latest ClinGen VSEP version (which removed specific Nomad version requirements) and the corresponding publication or Canvig guidelines (which recommend using Nomad version 2.1). Furthermore, the classification of a BRCA1 missense variant (Variant 3) showed how the same functional study evidence (Stoita et al. 2018) is weighted differently—strong (BS3) under ClinGen VSEP but only moderate under Canvig UK, resulting in benign versus likely benign classifications, respectively.

The experts also addressed common pitfalls, such as the misuse of criteria (e.g., applying BP1 and BP4 together, which constitutes double-counting evidence) and the incorrect application of criteria based on domain location (e.g., applying PP3 for in silico predictions outside BRCA1 functional domains). The discussion around Variant 5 (a BRCA1 deletion) introduced the use of PS4 (case-control data) under Canvig UK guidelines, leveraging NHS and UK Biobank data, a powerful criterion currently unavailable in the ClinGen VSEP framework. This highlights a crucial difference in how regional groups incorporate local, high-quality population data into their classification schemes. The session concluded by emphasizing that while performance is improving, ongoing education and harmonization efforts are essential, particularly as the community anticipates the release of the new ACMG/AMP guidelines (Version 4).

Key Takeaways: • Improved Classification Accuracy: Run 14 saw a significant improvement, with 50% of participants correctly classifying all six variants, suggesting that increased use of standardized guidelines and training tools is raising the global standard of variant interpretation. • Guideline Discrepancies are Common: Differences exist between the ClinGen VSEP (e.g., ATM, BRCA) and Canvig UK guidelines, particularly in the strength assigned to specific evidence codes (e.g., BS1, BP1, BS3), leading to classifications like benign vs. likely benign for the same variant. • Ambiguity in PM2 Application: For ATM variants, there is current ambiguity regarding which version of the Nomad database should be used to apply PM2 (absent in controls), with ClinGen VSEP recommendations conflicting slightly with their own publications and Canvig guidelines. • Regional Data Advantage (PS4): The Canvig UK guidelines allow for the application of PS4 (case-control data) using UK-specific resources like NHS data and UK Biobank controls, enabling a "very strong" classification strength that is not currently supported by the ClinGen VSEP framework. • Avoid Double-Counting Evidence: Participants frequently misused criteria, such as applying both BP1 (missense outside functional domain) and BP4 (in silico prediction benign), which is explicitly discouraged as it double-counts evidence towards a benign classification. • Domain-Specific Criteria: Criteria like PP3 (in silico prediction) and BP1 are highly domain-specific; for BRCA1, they should only be applied if the variant is located within specific functional domains (Ring, Coiled-Coil, BRCT repeats) or outside them, respectively. • Importance of PM5 Repurposing: For truncating variants in ATM and BRCA, the PM5 code has been repurposed by both VSEP and Canvig to apply at a strong strength, yet nearly half of participants failed to apply this criterion correctly for the BRCA2 frameshift variant (Variant 6). • Functional Study Weighting Varies: The strength assigned to functional studies (BS3/PS3) can differ between guidelines; the same study on a BRCA1 variant was rated as strong evidence for benign classification by ClinGen VSEP but only moderate by Canvig UK. • Impact of Paywalls on Classification: Access to crucial publications hidden behind paywalls remains a challenge, potentially limiting the evidence available to some labs and affecting the application of criteria like PM3 (co-occurrence/phenotype evidence). • Future Guideline Adaptation: The upcoming release of the new ACMG/AMP Version 4 guidelines will necessitate a significant effort by VCEPs to review and adapt gene-specific criteria, creating a temporary lag where labs must continue to rely on the existing gene-specific VCEP criteria.

Tools/Resources Mentioned:

• Genie Platform: GenQA genomic online individual education platform used for delivering the EQA.
• Nomad (v2.1, v4.1): Genomic databases used for assessing allele frequency and applying BS1/PM2 criteria.
• ClinVar: Database for variant classifications.
• BRCA Exchange: Database for BRCA variant information.
• LOVD (Leiden Open Variation Database): Database mentioned for variant data.
• SpliceAI: Tool used for predicting splicing impact.
• Canvar UK: Variant database used by UK scientists, providing access to functional study scoring sheets and case counts from NHS data.
• CardioB Calculator: Used for calculating the maximum tolerated allele count for BS1 application.

Key Concepts:

• External Quality Assessment (EQA): A scheme used to monitor and improve the quality and consistency of laboratory performance, in this case, genomic variant classification.
• Variant Classification Expert Panel (VCEP): ClinGen working groups that develop gene-specific rules and modifications for the ACMG/AMP guidelines (e.g., ATM VSEP, BRCA VSEP).
• PVS1 (Pathogenic Very Strong): Criterion applied to predicted loss-of-function variants (frameshifts, nonsense) that are likely to undergo Nonsense-Mediated Decay (NMD).
• PM3 (Pathogenic Moderate): Criterion applied when a variant is found in trans with a known pathogenic variant for a recessive disorder, or when phenotype evidence is strong.
• BS1 (Benign Strong): Criterion applied when the allele frequency of a variant in control populations (Nomad) is too high to be consistent with the disease prevalence.
• BP1 (Benign Supporting): Criterion applied to missense variants in genes where truncating variants are the primary cause of disease, provided the missense variant is outside a known functional domain.
• PM5 PTC (Protein Truncating Codon): A repurposed criterion used for protein truncating variants (frameshifts, nonsense) in ATM and BRCA to provide additional strong evidence of pathogenicity.