Aug2013 reference material selection and design working group
Download as PPTX, PDF1 like1,054 views
The document summarizes discussions from a working group meeting regarding the selection and design of reference materials for genome sequencing. It was attended by approximately 25 people from federal agencies, clinical labs, and technology companies. Options discussed included plasmids with cancer mutations from NCI, tumor-normal cell line pairs, and cell lines from HorizonDx and Acrometrix that contain engineered variants. The working group aims to define testing protocols to evaluate reference material performance across labs and assess their ability to characterize sequencing platforms and tests.
Aug2013 reference material selection and design working group
- 1. Genome in a Bottle Working Group Reference Material (RM) Selection and Design NIST Workshop August 15, 2013
- 2. Main Meetings – NIST Reference Materials • April 13, 2012 (NIST) – Genome in a Bottle consortium initiation • August 16, 2012 (NIST) – Intended uses of RMs – RM selection strategies • November 7, 2012 (ASHG) – Status updates • December 6, 2012 – Selection of initial RMs • March 21, 2013 (XGEN Congress) • August 15, 2016 (NIST) | 2
- 3. Workgroup Attendees NIST Workshop – Aug 16, 2012 • Approximately 25 attendees – Federal, incl. FDA, CDA, NIST – Lab accreditation – Clinical reference labs – Platform technologies – Reference material / reagent providers – Research | 3
- 4. Reference Materials Needed For • Clinical platform validation – Sequencing System – Bioinformatics/Analysis Pipeline • Clinical test development and validation – Whole genome – Targeted – Germline vs. tumor – Other? • Research – Process development and QC • Product development – Sequencing Systems – Software development | 4
- 5. NY State Guidelines – Oncology NGS Minimum Data Requirement - Validation • Accuracy: Sequence a well-characterized reference sample (e.g. HapMap DNA GM12878) to determine error rate across all amplicons. • Analytical sensitivity: Establish the analytical sensitivity of the assay by interrogating all variants in the 3 amplicons with the consistently poorest coverage, and all variants in 3 amplicons with consistently good coverage. This can initially be established with defined mixtures of cell line DNAs (not plasmids), but needs to be verified with 3-5 patient samples. • Analytical specificity: Establish the analytical specificity of the assay by interrogating all variants in the 3 amplicons with the consistently poorest coverage, and all variants in 3 amplicons with consistently good coverage. This can initially be established with defined mixtures of cell line DNAs (not plasmids), but needs to be verified with 3-5 patient samples. | 5
- 6. Reference Material – Intended Uses • Characterize Platforms & Methods – DNA sequencing – Existing & upcoming NGS technologies – Research applications – Clinical diagnostics applications • Not intended as reference material for – Validation of specific mutations in a panel | 6
- 7. Preferred Reference Standard Criteria Reference Lab Perspective • Commercially available (renewable and constant) • Cost-effective • Similar in performance to anticipated test samples • Contain representative type and number of mutations tested • Ability to adjust mutation frequency to monitor assay sensitivity • Recognized by agencies responsible for recommending standards
- 8. Status Summary – RM Working Group • Defined required RM characteristics • Initial set of RM samples selected – NA12878 • Many existing public and proprietary datasets • Listed in clinical guidelines to establish validation parameters • Consent limitations – Commercial use, re-identification through sequence • Under legal and IRB review by NIST – Three PGP trios • One trio already available at Coriell • Consent without withdrawal option may not meet ethical review standards | 8
- 9. Reference Material Portfolio Synthetic DNA • What types are most useful – Variant types, sequence context, phasing • Assess strengths & weaknesses of platform • Test validation for existing and medically relevant variants • What size • How many synthetic RMs – Platform validation – Test validation • Which existing sources should be considered – Available for research and for profit • Source of commercial development and distribution – Manufactured under quality system for diagnostic applications | 9
- 10. Agenda RM Selection and Design Working Group • Presentations: – NCI Cancer spike-ins – Jason Lih – TGen-Illumina tumor-normal pair study – Stephanie Pond – HorizonDx cell lines – Jonathan Frampton – Acrometrix FFPE controls – Mona Shahbazian • What role should synthetic spike-ins play in GIAB? – Spike-ins could include NCI cancer spike-ins, TGen cancer rearrangement spike-ins, or new synthetic DNA designs • Should GIAB start to develop tumor-normal cell line pairs as Reference Materials • What is the relative importance of having genomic Reference Materials from a variety of ancestry groups – What are the relative priorities of ancestral diversity, trios, and the broad consent of the PGP for commercial redistribution – Trios in PGP are currently all Caucasian except for one Asian (Chinese) trio. If PGP cannot have trios from other ancestry groups, should we use PGP Caucasian trios, PGP individuals without consented parents, or trios from other projects like 1000 Genomes
- 11. Workgroup Attendees NIST Workshop – Aug 15, 2013 • Approximately 25 attendees – Federal, incl. FDA, CDA, NIST – Clinical reference lab – Platform technologies – Reference material / reagent providers – Research | 11
- 12. Summary Jason Lih / NCI • 53 plasmids with engineered cancer mutations – 1kb fragments – Mutation in center – Alien barcode in vicinity for differentiation from human DNA – Sequence is Sanger confirmed • Low level mutations not observed in NGS verification • Quantitation by qPCR – Observed concentrations in dilution series are linear, similar to cell line DNA controls • Spike in controls for reference material or clinical samples • Linearized, no chromatin structure or FFPE exposure • No blocking I.P. for plasmid | 12 Proposal: Interlab comparison of plasmid controls to assess performance and resemblance of FFPE derived DNA Possibility of QC Standard?
- 13. Summary Stephanie Pond / Illumina • Tumor / Normal Cell Line Pair – COLO829B / COLO829 – Pleasance et al, Nature 2010, 463:191-196 – SNVs, InDels, CNVs, SVs (37) • Confirmed by PCR & sequencing – HiSeq 90x WGS at Illumina & TGEN – Complete Genomics WGS – Multiple dilution series analyzed – Consent needs investigation | 13 Monitor: Offers no clear advantage over existing selected normal cell line trios
- 14. Summary Stephanie Pond / Illumina • 9 synthetic fusion constructs – Spiked into COLO829 DNA – One Spike-In not detected by any fusion caller • Confirmed to be present in sequence | 14 Opportunity: Assess and optimize fusion caller software Recommendation: Recommend to make results available in data repository for software optimization Early status for fusion constructs, continue to monitor
- 15. Summary Jonathan Frampton / Horizon Diagnostics • Large set of engineered cancer mutations in 3 cancer cell lines – Proprietary engineered cell lines – Confirmed identities of parental cell lines – Digital PCR confirmed mutational frequencies – FFPE embedded cells / DNA available – Linear serial dilution response to 0.05% – 11 core mutations are multiplexed with frequencies between 1% - 25% • digital PCR confirmed – 19 additional mutations planned for multiplex with similar range of frequencies in mixture | 15 Assess performance in interlab comparison Combine with assessment of synthetic plasmids and Acrometrix controls? Opportunity for independently supplied reference materials
- 16. Summary Kara Norman / AcroMetrix • Engineered cell line controls – 8 different multi mix controls – 12-26 variants per control • 1-7 COSMIC variants / control – Standards produced under cGMP / Quality System – Proprietary cell lines – FFPE embedded cells / DNA available | 16 Assess performance in interlab comparison Combine with assessment of synthetic plasmids and Horizon Dx controls? Opportunity for independently supplied reference materials
- 17. Next Steps • Working group to define testing protocol(s) and assess plasmid controls in interlab study – Combine with assessment of Horizon & Acrometrix cell line controls? • Confirm that performance is similar to clinical FFPE samples | 17