SPADA at AOAC Annual Meeting 2023

AOAC INTERNATIONAL STAKEHOLDER PROGRAM ON AGENT DETECTION ASSAYS Meeting of the Working Groups

AOAC Annual Meeting New Orleans, Louisiana Saturday, August 26, 2023 1 :00PM – 3 : 00 PM CDT Draft Agenda Moderator: Deborah McKenzie (AOAC INTERNATIONAL)

I. PRELIMINARY ITEMS (1:00PM – 1:15PM) Deborah McKenzie will welcome attendees, introduce speakers, and provide an overview and history of SPADA to date. II. WORKING GROUP II – OVERVIEW & NEXT STEPS (1 :15 PM – 1:45 PM) Dr. Shanmuga Sozhamannan - Technical Coordinator Joint Research and Development, Inc. (JRAD) supporting Defense Biological Product Assurance Office (DBPAO) at the JPEO, Joint Project Lead, CBRND Enabling Biotechnologies (JPL CBRND EB) Dr. Sozhamannan will discuss ASqMI and SMPR needs for both Amp Seq and Meta Seq (genomics) and review the successful completion of the Amp Seq SMPR. III. VALIDATION OF BIOTHREAT METHODS ( 1:45 PM – 2: 15 PM) Dr. Sharon Brunelle – Technical Consultant, AOAC INTERNATIONAL Dr. Brunelle will discuss the SMPR for the Detection of Multiple Biothreat Agent Organisms in Environmental Samples by Amplicon Sequencing (Amp Seq) and review related method validation guidelines. IV. HOW SPADA GOT US TO METAGENOMICS (2: 15P M – 2:45 PM) Dr. Michael Sussman - Senior Research Scientist, US Department of Agriculture, Agricultural Marketing Service, Agricultural Analytics Division Dr. Sussman will provide a summary on VNGS, FAIR, and Artificial Intelligence as it relates to the work that has been accomplished in SPADA.

V.

WRAP UP & ADJOURNMENT ( 2:45 PM – 3: 00 PM) Deborah McKenzie and Allison Baker (AOAC INTERNATIONAL) will discuss next steps.

This agenda is subject to change without notice.

SHANMUGA SOZHAMANNAN, Ph. D. Dr. Shanmuga Sozhamannan is the technical coordinator for the Defense Biological Product Assurance Office (DBPAO) at the JPEO, JPL CBRND Enabling Biotechnologies. In this role, he provides SME support for the development, fielding, and sustainment of reagents, assays, and biothreat detection systems used in national biodefense. He serves as the key interface between service laboratories, quality assurance test laboratories, and product end-users while maintaining positive government, industry, and interagency partnerships. He

brings in diverse and extensive experience in molecular genetics, molecular biology and genomics working in academia, private industry, and government laboratories. Shanmuga obtained his Ph. D degree in Bacterial molecular genetics and has published over 70 peer- reviewed, scientific publications and several book chapters and holds several patents. Over the years, his research work focused on bacterial and bacteriophage genetics, molecular aspects of pathogenesis, and creating phage-based and molecular diagnostic assays. He has also used next-generation sequencing technologies for various applications, ranging from pathogen detection and characterization to development of genomics tools for diagnostics.

Sharon Brunelle, Ph.D. Sharon Brunelle has worked with the AOAC Research Institute and AOAC INTERNATIONAL since January 2003 as a technical consultant designing and coordinating microbiology and chemistry method validation studies in the Performance Tested Methods SM , Reviewed and Recognized SM and Official Methods of Analysis SM programs. Sharon also served as the technical lead for various contract and standard development projects within AOAC, including best practices for microbiology methods, biothreat agents, veterinary drug residues, low lactose, and glycerol

esters of wood rosins. She played a major role in developing validation guidelines for biothreat agent and microbiology methods. In addition, Dr. Brunelle consults for industry in the areas of food microbiology and veterinary drugs in animal tissues and feeds. Prior to AOAC, Dr. Brunelle worked for 10 years in food microbiology and clinical oncology diagnostics development and validation. Sharon earned a B.S. in chemistry from the University of Delaware and a Ph.D. in biochemistry from Brandeis University. Her postdoctoral work was in medical biochemistry at Rockefeller University and the Picower Institute for Medical Research.

Michael Sussman, Ph.D Michael Sussman received his BS from Tulane University, MS in molecular microbiology from the University of Illinois at Chicago and Ph.D. in molecular biology and biochemistry in biomedical science from the University of Connecticut. He trained in two molecular virology post-doctoral fellowships at Plum Island Animal Disease Center and Michigan State University. Until 2003, Dr. Sussman was employed as a research scientist in animal vaccine development. From 2003 to 2009, Michael led the US

Department of Agriculture’s biotechnology testing program. From 2009 to 2013 he served as the Director of the USDA, Agricultural Marketing Service Laboratory Division. Since 2013, Dr Sussman is currently serving as a senior research scientist for the US Department of Agriculture, Agricultural Marketing Service’s Agricultural Analytics Division in Washington, DC. Dr. Sussman was the Chairperson for ISO TC 34/SC 16 from 2008 until 2018. He became the current committee manager of ISO/Technical committee 34 “Food products”/SC 16 “Horizontal methods for molecular biomarker analysis” in 2017 and also serves as an expert in a number of ISO, AOAC, and federal policy committees. He most recently has served the Chair for the SPADA working group to develop a SMPR for NGS reference sequences.

Deborah McKenzie Joining AOAC INTERNATIONAL in 2003, Deborah is Deputy Assistant Executive Director & Chief Standards Officer, including method approval processes. Her responsibilities include overseeing implementation and execution of voluntary consensus standards processes and the Official Methods SM and Performance Tested Methods SM programs. She and her team coordinate and administer associated activities with standards development and method approval programs. In support of her responsibilities, she is the lead staff liaison to the AOAC Official Methods Board and develops educational materials and orientations on

processes for standards development and method approval programs. Previously, she was responsible for daily management of and business development for the AOAC Research Institute and served as Senior Director, Standards and Official Methods SM . Most recently, McKenzie also served in an interim role of coordinating the AOAC Science Team which included working with AOAC Research Institute and the Laboratory Proficiency Testing Program in the coordination of the AOAC science programs and resources. Most recently, Deborah also served in an interim role of coordinating the AOAC Science Team which included working with AOAC Research Institute and the Laboratory Proficiency Testing Program in the coordination of the AOAC science programs and resources. Before joining AOAC, Deborah was a chemistry teacher and STEM educator and served as co- project director for state and federally funded grants for student and teacher development in STEM education with a focus on socioeconomically underserved communities. Deborah holds M.S. and B.A. degrees in Chemistry from SUNY Oswego and Syracuse University. She is also an

ordained rabbi, with a certificate of rabbinic ordination from the Jewish Spiritual Leaders Institute. Deborah is member of the American Chemical Society (ACS), International Association for Food Protection (IAFP), American Institute of Parliamentarians (AIP), Society for Standards Professionals (SES), National Society for Black Engineers (NSBE), and AOAC INTERNATIONAL.

Stakeholder Panel on Agent Detection Assays Program Meeting Saturday, August 26 | 1:00PM – 3:30PM CDT

Scan to sign into the meeting and get access to meeting materials!

AOAC INTERNATIONAL 137 th Annual Meeting & Exposition August 25 – 30, 2023 • New Orleans, LA

You are Cordially Invited…

AOAC Micro Community Meeting

Sunday, August 27 | 4:30PM – 5:30PM Studios 7-8

AOAC INTERNATIONAL 137 th Annual Meeting & Exposition August 25 – 30, 2023 • New Orleans, LA

OVERVIEW OF STAKEHOLDER PROGRAM ON AGENT DETECTION ASSAYS (SPADA) CELEBRATING 16 YEARS OF DEVELOPING STANDARDS IN SUPPORT OF PUBLIC & FIELD DEPLOYED SAFETY

August 26, 2023

Deborah McKenzie, AOAC INTERNATIONAL Deputy Assistant Executive Director & Chief Standards Officer

In Food & Agriculture, We Set the Standard

OVERVIEW

 SPADA Background and History

 Background on previous standards / Standard Method Performance Requirements (SMPRs) and recent standards approved

 Current Initiative / Meeting Goals

2

In Food & Agriculture, We Set the Standard

SPADA SETS STANDARDS 2007 - 2023

 SPADA represents a broad collaboration of stakeholders  Formed to develop public health actionable assays  Developed 26 standards in support of biothreat agent method development and validation for public health, public safety, and field deployed safety  All SPADA members volunteer their time and expertise

DHS

DoE

CDC

USPS

Academia

SPADA Stakeholders, Members & Experts

EPA

Industry

First Responders

DoJ

State/Local Dept of Health

DoD

FDA

NIST

In Food & Agriculture, We Set the Standard

AOAC SPADA JOURNEY 2007 - 2013

Original Objectives in 2007

•Under contract with DHS S&T, AOAC established SPADA whose scope was to develop public health actionable assays and standards to validate Polymerase Chain Reaction (PCR)-based technologies that detect

aerosolized Bacillus anthracis , Yersinia Pestis , or Francisella tularensis •Pilot the validation process with an assay that detects B. anthracis •Held Town Hall in support of the overall needs of the community 2008

•Develop standards to validate immunoassay-based Hand-Held Assays (HHAs) that detect B. anthracis or Ricin in suspicious powders •Test commercially-available HHAs 2009 •Develop standards to validate PCR-based technologies that detect aerosolized Burkholderia psuedomallei and Burkholderia mallei •Develop standards to validate PCR-based technologies that detect B. anthracis in suspicious powders 2010

•Develop recommendations on controls needed for field-based assays 2011

•Develop standards to validate PCR-based technologies that detect aerosolized Variola •Establish First Responder Working Group •Maintain a SPADA Executive Steering Committee 2013

In Food & Agriculture, We Set the Standard

SPADA SETS STANDARDS 2014 - 2019

Original Objectives in 2014

•Under Contract with Deputy Undersecretary of the Army- Test and Evaluation through The Johns Hopkins University, Applied Physics Laboratory, AOAC engaged SPADA to establish voluntary consensus standards with emphasis on the warfighter •Establishes Standard Method Performance Requirements (SMPRs) that include inclusivity/exclusivity panels SMPRs •SPADA streamlined this panel for to promote efficiency and modernity with the formation of the Environmental Organisms Panel •The Environmental Organisms Panel is an evaluation of potential cross-reactivity to the DNA from a wide variety of organisms that are in the environment Environmental Panel •Develop guidelines for the characterization and authentication of bacterial strains to provide confidence in the identification of material being used •Develop standard procedures for the use of in silico PCR analysis •Develop standard procedures for preparation and characterization of soils to be used in the evaluation of candidate biothreat detection assays Other General & Supporting Standards

5

In Food & Agriculture, We Set the Standard

SPADA WORKING GROUPS

B. anthracis (PCR) (DoD scope) Dr. Paul Jackson (LLNL) Dr. Ted Hadfield (Hadeco)

F. tularensis Dr. Peter Emanuel (DoD) Dr. Mark Wolcott (DoD)

F. Tularensis (DoD scope) Dr. Paul Keim (Northern Arizona University)

B. anthracis (PCR) Dr. Paul Jackson (LLNL)

Y. Pestis Dr. Luther Lindler (DHS)

Dr. Ted Hadfield (MRI)/Hadeco)

Y. Pestis (DoD scope) Dr. Luther Lindler (DHS)

Variola Victoria Olson (CDC) Ted Hadfield (MRI)

Burkholderia Paul Keim (NAU) Alex Hoffmaster (CDC)

Burkholderia (DoD Scope) Jay Gee (CDC)

Variola Victoria Olson (CDC)

Venezuelan Equine Encephalitis (DoD scope) James Samuel (U of Texas, A&M)

Brucella Suis (DoD scope) Frank Roberto (Idaho Natl. Laboratory)

C. Burnetti Eileen Ostlund (USDA, ARS)

In Food & Agriculture, We Set the Standard

SPADA WORKING GROUPS

B. Anthracis Handheld Assay (Marian McKee, BioReliance Corp.)

Assay Controls Dr. Christina Egan (NYDOH) Dr. Larry Blyn (Ibis)

Ricin Handheld Assay Mark Poli (DoD) Botulinum Neurotoxin A (DoD scope) Sashi Sharma (FDA)

Environmental Factors Dr. Steven Morse (CDC)

Public Health Actionable Assay Dr. Peter Estacio (LLNL)

SEB WG (DoD scope) Sandra Tallent (FDA)

Revision of the Sampling Standard for Nonporous Surfaces • Jayne Morrow (US NIST) • Christina Egan (NYSDOH) • David Ladd (Masss. FR)

Operational Guidance for Sampling • Jayne Morrow (US NIST) • Christina Egan (NYSDOH) • David Ladd (Masss. FR) Bacterial Strain Verification Dr. Linda Beck (JRAD) Dr. Shanmuga Sozhamannan (DOD JPEO)

Environmental Organisms Dr. Linda Beck (JRAD)

Insilico Analysis Dr. Linda Beck (JRAD) Dr. Shanmuga Sozhamannan (DOD JPEO)

Soils Analysis Dr. Linda Beck (JRAD) Dr. Morgan Minyard (DTRA)

Various Working Groups formed to develop handheld or toxin assays

Various Working Groups formed to develop general or supporting standards

In Food & Agriculture, We Set the Standard

PUBLICATION & AVAILABILITY

 An overview of the JHU APL/DoD/AOAC project and all 10 SMPRs together with the revised Environmental Organisms Panel was published in the 2018 Nov/Dec issue of the Journal of AOAC INTERNATIONAL.

 ALL SMPRs are available at AOAC at www.aoac.org.

8

In Food & Agriculture, We Set the Standard

SPADA CONSENSUS 2018

SPADA voted to accept three standard documents:  Guidelines for Verifying and Documenting the Relationships Between Microbial Cultures (AOAC OMA, Appendix R)  Recommendations for Developing Molecular Assays for Microbial Pathogen Detection Using Modern In Silico Approaches (AOAC OMA, Appendix Q)  Guidance for Soil Collection, Characterization, and Application for Biothreat Agent Detection Method and Site Evaluations (AOAC OMA, Appendix P) Published in the Journal of AOAC INTERNTIONAL and available as Appendices to the electronic version of the Official Methods of Analysis of AOAC INTERNATIONAL.

In Food & Agriculture, We Set the Standard

SPADA 2021 - NEW SCOPE OF WORK Builds from the previous SPADA work on in silico analysis and bacterial strain verification with the following goals

Advancing the utility of reference microbial sequencing databases to evaluate, develop and support detection capabilities using next generation sequencing applications for deployable

Developing standards for NGS biothreat agent detectors that encompass both PCR-based amplicon sequencing applications and metagenomic applications

Creating validation criteria and confidence parameters in reference genome databases

diagnostic kits and platforms/detectors

In Food & Agriculture, We Set the Standard

SPADA WORKING GROUPS – 2021-2022

SPADA

 WG1 – Detection Capabilities with NGS C hair: Dr. Tim Minogue, USAMRIID Chair: Dr. Michael Sussman, USDA AMS

WG1 – Detection Capabilities with NGS Working Group WG2-Standardsfor NGS Biothreat Agent Detectors Working Group

 WG2 – Standards for NGS Biothreat Agent Detectors: Chair: Dr. Shanmuga Sozhamannan, JRAD  Subgroup on Amplicon Sequencing Applications (Chair: Dr. Shanmuga Sozhamannan, JRAD)  Subgroup on Metagenomic Applications (Chair: TBD)

WG2a –Amplicon Sequencing

Applications Subgroup WG2b –Metagenomics Applications Subgroup

 WG3 – Reference Database and In Silico Processes Validation Chair: Dr. Michael Sussman, USDA AMS

WG3- Reference Database & In Silico Process Validation Working Group

In Food & Agriculture, We Set the Standard

SPADA CONSENSUS 2022

SPADA voted to approve two standard documents that are being published in the Journal of AOAC INTERNTIONAL or as an Appendix to the Official Methods of Analysis of AOAC INTERNATIONAL TM

Standard Requirements for Nucleotide Sequences used in Biothreat Agent Detection, Identification, and Quantification: Verified Next Generation Sequences (VNGS) In publication as an Appendix of the Official Methods of Analysis of AOAC INTERNATIONAL TM

Amplicon Sequencing Minimal Information (ASqMI): Quality and Reporting Guidelines for Actionable Calls in Biodefense Applications Advance publication in Journal of AOAC INTERNATIONAL (online)

In Food & Agriculture, We Set the Standard

TODAY’S MEETING

Development of Standard Method Performance Requirements (SMPRs®) for Amp Seq assay and meta seq assays •Dr. Shanmuga Sozhamannan (JRAD)

Performance Requirements and Validation of Biothreat Amp Seq Methods

•Dr. Sharon Brunelle (Brunelle Biotech/AOAC INTERNATIONAL)

Engage in stakeholder discussion about the SMPR

•All Attendess

VNGS, FAIR and Artificial Intelligence

•Dr. Michael Sussman (USDA AMS)

In Food & Agriculture, We Set the Standard

SUPPORT FOR CURRENT SCOPE OF WORK IN SPADA IS MADE POSSIBLE BY THE FOLLOWING:

 Contract through Johns Hopkins University Applied Physics Laboratory:  Deputy Undersecretary of the Army-Test and Evaluation (DUSA-TE)  Joint Program Executive Office- Chemical Biological Radiological Nuclear Defense (JPEO – CBRND) Joint Project Lead – Enabling Biotechnologies DBPAO

In Food & Agriculture, We Set the Standard

YOUR AOAC SPADA STAFF TEAM

David B. Schmidt, Executive Director Katerina Mastovska, Deputy Executive Director & Chief Science Officer Deborah McKenzie, Sr. Dir., Standards Development Alicia Meiklejohn, Director, Governance and Business Development

Allison Baker, Standards Coordinator Saliha Argubie, Standards Coordinator Sharon Brunelle, AOAC Technical Consultant

In Food & Agriculture, We Set the Standard

FOR QUESTIONS OR MORE INFORMATION ON SPADA

Please contact: Kate Mastovska at kmastovska@aoac.org Deborah McKenzie at dmckenzie@aoac.org Alicia Meiklejohn at ameiklejohn@aoac.org Allison Baker at abaker@aoac.org.

For Questions or Information on SPADA Supporting SPADA Joining SPADA and its active working groups Receiving updates on SPADA General inquiries

In Food & Agriculture, We Set the Standard

In Food & Agriculture, We Set the Standard

DEVELOPMENT OF STANDARD METHOD PERFORMANCE REQUIREMENTS (SMPRs®) FOR AMP SEQ ASSAY AND META SEQ ASSAYS

August 26, 2023

Shanmuga Sozhamannan, Ph. D. Technical Coordinator Defense Biological Product Assurance Office JPL CBRND Enabling Biotechnologies JPEO, DoD

Presented at the AOAC International Annual Meeting-SPADA session 2023

1

UNCLASSIFIED

OUTLINE- 5WH

AmpSeq SMPR

Precursor document: A mplicon S e q uencing M inimal I nformation (ASqMI): Quality and Reporting Guidelines for Actionable Calls in Biodefense Applications

UNCLASSIFIED

2

AOAC Annual 2023

• The Defense Biological Products Assurance Office (DBPAO) maintains a portfolio of forty-six PCR assays targeting twenty-one biological warfare agent organisms • These assays are produced under ISO 17034 accredited production standards by Naval Medical Research Center (NMRC). • The assays were developed by various Department of Defense (DoD) developer laboratories and their performance characteristics reported in a standardized DTRA CB56 data package. • The result is a set of well characterized and validated assays . These assays are available to any DoD laboratory and their academic or industry partners. • The assays are currently formulated for three widely used platforms during development: the RAPID/LightCycler, the ABI7900HT FastBlock, and the ABI7500 Fast platforms. • Additionally, these reagents have been shown to function with no loss of performance on platforms such as the ABI StepOne, the QuantStudio5, and the BioMolecular Systems MIC qPCR cycler. • The intent of the DBPAO was to standardize the assays available to the US Government and their partners in research and development efforts or operational use for detection of these agents.

REAL TIME PCR HAS BEEN THE GOLD STANDARD IN BIOSURVEILLANCE FOR DECADES

• Courtesy: Dr. Mark Munson (DBPAO’s Molecular portfolio Repository)

UNCLASSIFIED

3

AOAC Annual 2023

PCR - STATE OF THE ART- IN BIOSURVEILLANCE/BIODEFENSE

• Biodefense relied on PCR for three decades or more (at least since first Gulf war-1991) o Singleplex –

Biomeme Franklin

o Multiplex (recent) o High throughput? o Not portable

RAZOR

R.A.P.I.D.

The most we have seen is :

Quant Studio

Light Cycler

JBAIDS

BIOFIRE-FILM ARRAY

UNCLASSIFIED

4

AOAC Annual 2023

STANDARD METHOD PERFORMANCE REQUIREMENTS (SMPRS®)

AOAC has been the standard setter for many biothreat agent detection methods

• SMPRs are voluntary consensus standards, developed by stakeholders, that prescribe the minimum analytical performance requirements for classes of analytical methods. • SMPRs are unique to AOAC’s processes and were introduced in recognition of the fact that acceptance criteria are critical to evaluating the suitability of any testing protocol for its intended use. • Methods that meet the performance requirements can claim conformity to the SMPR

UNCLASSIFIED

5

AOAC Annual 2023

BTAM SMPR EXAMPLE – 2016.006

DNA-Based Methods of Detecting Bacillus anthracis in Field Deployable, Department of Defense Aerosol Collection Devices

(Accepted Minimum Detection Level)

UNCLASSIFIED

6

AOAC Annual 2023

BIOTHREAT AGENT METHOD (BTAM) SMPRS

SMPR Target

Technology Matrix

AMDL

2010.001 Francisella tularensis

PCR

Aerosol Collector

20,000 cells/filter; 2000 cells/mL; 2000 GE/mL FT Schu-S4

2010.002 Yersinia pestis

PCR

Aerosol Collector

20,000 cells/filter; 2000 cells/mL; 2000 GE/mL YP strain CO92

2010.003 Bacillus anthracis 2010.004 Bacillus anthracis

PCR

Aerosol Collector Visible powders Visible powders Aerosol Collector

20,000 spores/filter; 2000 spores/mL; 2000 GE/mL BAAmes

HHAs HHAs

10^7 CFU/mL BAAmes

2010.005 Ricin

25 ng/mL RCA 60

2014.006 Variola virus 2015.011 Coxiella burnettii

PCR

50,000 copies/mL Variola target DNA (viral genome or equivalent plasmid copies)

Nucleic acid Aerosol Collector

2000 GE/mL

Venezuelan Equine Enc. Virus (VEEV) Staph. Enterotoxin B (SEB)

2015.012

Molecular

Aerosol Collector

5000 genome copies/mL

2015.013

Protein

Liquid

0.25 ng/mL recovered SEB 2000 spores/mL BAAmes

2016.006 Bacillus anthracis 2016.007 Francisella tularensis

Nucleic acid Aerosol Collector Nucleic acid Aerosol Collector Nucleic acid Aerosol Collector Nucleic acid Aerosol Collector

2000 cells/mL

2016.008 Yersinia pestis 2016.009 Brucella suis

2000 cells/mL YP strain CO92

2000 GE/mL BS biovar 1, type strain 1330

Burkholderia pseudomallei

2016.010

Nucleic acid Aerosol Collector

2000 cells/mL BP 1026b

Botulinum Neurotoxins A1 and A2

2016.011

Protein

Aerosol Collector Aerosol Collector

1.25 ng/mL recovered Bot toxin A1 and A2 complexes

2016.012 Variola virus

PCR

50,000 copies/mL Variola target DNA (viral genome or equivalent plasmid copies)

UNCLASSIFIED

7

AOAC Annual 2023

BETTER FASTER CHEAPER- NEXT GEN SEQUENCING

Next gen sequencing based biodetection assays and the need for SMPRs Two types of assays • Targeted • Unbiased untargeted metagenome

ONT

UNCLASSIFIED

8

AOAC Annual 2023

SEQUENCE ANYTHING, ANYTIME, ANYWHERE BY ANYONE

August 2016 – NASA astronaut Kate Rubin sequences DNA in space for the first time.

Next Gen ‘Sequencing’

‘Next Gen’ Sequencing

In Antarctica, the MinION proved capable of sequencing well even under harsh conditions. https://www.nasa.gov/mission_pages/station/research/bene fits/dna-sequencing-in-space-timeline

HM Gan @DrGanHM CSO - GeneSeq Sdn Bhd NGS in SEA. For business inquiries : gan.gseq@gmail.com

9 Bukit Beruntung, Malaysia; 03212021

https://www.science.org/doi/10.1126/science.356.6338.572 AOAC Annual 2023

UNCLASSIFIED

‘RUMSFELDISM’ IN NGS APPLICATIONS

Donald Rumsfeld in a press briefing back in 2002 introduced the concept with respect to threats facing the country

Known Pathogen Unknown Target Usual suspects plus (SARS-CoV-2 variants) • Targeted AmpSeq • Isolate sequencing

Known Pathogen Known Target Usual suspects • Real time PCR • Targeted AmpSeq

Unknown Pathogen

Unknown Pathogen Known Target Newly

Unknown Target One that defies logic (Discovery) • MetaSeq • Isolate sequencing

emergent/engineered with usual attributes ( B. cereus var. anthracis ” CI) • Targeted AmpSeq (16S) • Isolate sequencing

Solving for complexity Posted on January 26, 2021 by Doug Belshaw

UNCLASSIFIED

10

AOAC Annual 2023

• PCR to Unbiased metagenome sequencing is a step too far for various reasons • Meta Seq • True unbiased sequencing to find known and unknowns in a sample • Metagenome seq- drawback- background data – needle in the haystack • Metagenome sequence analyses has some challenges (solutions exist e.g., EDGE) • Amp Seq • Amp Seq: More info than PCR • Possibility of High throughput, Multiplex • Potentially low cost or at least on par with PCR • Reduced time from sample to actionable call • Low abundance, high background, high error rate technologies can be addressed • Enriching for low abundance of targeted pathogens by PCR • Probing very limited seq space in a pathogen as opposed to whole genome sequencing (0.03 % of a 3Mb genome with 10 amplicons of 100 bps) • Rationale for us: Amplicon sequencing (PCR assays especially those that have been verified and validated in DoD, stood the test of time, have data on performance over a long period of time and in field testing) has the potential to bridge the gap between PCR- and NGS-based biosurveillance • What is the gap? • Existing NGS assay panels lack “validated analytical techniques” to adjudicate high background and error-prone NGS data. • Linking the data to pathogen with high confidence

AMP SEQ CAN BRIDGE THE GAP BETWEEN PCR AND META SEQ

UNCLASSIFIED

11

AOAC Annual 2023

How do you make actionable call from very few read data of a suspect pathogen?

Coincidence vs Causation

Data content vs context

Data without context is just useless! ;-)Pascal Bornet_tweet

UNCLASSIFIED

12

AOAC Annual 2023

CHANGING THE BIODEFENSE PATHOGEN DETECTION PARADIGM

Solution • ON KEA Tar HTS (pronounced as On Key Tarhits): Oxford Nanopore based Known Etiological Agents Targeted High Throughput Sequencing. Highly multiplexed, high throughput, amplicon sequencing based Oxford Nanopore assay for molecular detection of bio threat agents in far forward scenarios. • Mk1C: Evaluation of Portable Sequencer -Oxford Nanopore based Mk1C. Problem • Pathogen detection by PCR ( for almost 30 years ) has throughput limitations • Singleplex assays identify one target in one organism and one sample at a time • Field-forward detection assays needed to enable simultaneous screening of multiple targets, in multiple pathogens and in multiple samples • Full sample-to-answer pipeline should be <8-hour workday

UNCLASSIFIED

13

AOAC Annual 2023

ON-KEATARHTS-OXFORD NANOPORE AMPLICON SEQUENCING

Sample to Actionable Call Pipeline

Sample Preparation (~2 hours)

Extraction (~2 hours) PCR (1 hour)

Unlock the unknown

Matrices spiked with bacterial agents at different concentration

Library Preparation (~6 hours)

Sequencing (1-48 hours)

Sample 1

Load the flowcell

Sequencing

Data Analysis

Barcoding and Adapter Ligation

Pool Libraries

Player R, Verratti K, Staab A, et al. BMC Genomics . 2020;21(1):166. Published 2020 Feb 17. doi:10.1186/s12864-020-6557-5

Sample 2

UNCLASSIFIED

14

AOAC Annual 2023

LoD threshold

PCR DATA (LIMITED MULTIPLEX)

Spike concentrations Matrix

• Ct values plotted as heat map- intensity of color high = low Ct value = high spike level • Decreasing concentration series of spiked agents • Cut off Ct>40 • Left y axis: organism group and strains • Right y axis: assays • Top x axis: concentration gradient left to right: 5-1; blocks of CB- cocktail buffer, CF- Clean filter, DF- Dirty filter • Bottom x axis- replicates • All assays (agent specific multiplex) worked as expected) three exceptions: 2 FN and 1 FP)

Data from 405 samples/ 1440 reactions 4040

O r g a n i s m s

A s s a y s

Replicates

UNCLASSIFIED

15

AOAC Annual 2023

SEQUENCING RESULTS (10 MINUTES DATA)

Data from 405 samples/ 1440 reactions

Spike

UNCLASSIFIED

16

AOAC Annual 2023

14 PLEX PCR ASSAY (ALL STRAINS AND ALL PRIMERS AND PROBES)

10 min

48 hrs

Replicate

Replicate

Player R, Verratti K, Staab A, et al. BMC Genomics . 2020;21(1):166. Published 2020 Feb 17. doi:10.1186/s12864-020-6557-5

Assay

Assay

UNCLASSIFIED

17

AOAC Annual 2023

WHAT IS IN THE BACKGROUND? METAGENOME ANALYSES

UNCLASSIFIED

18

AOAC Annual 2023

Amplified Unamplified WHAT IS IN THE BACKGROUND? METAGENOME ANALYSES

UNCLASSIFIED

19

AOAC Annual 2023

AOAC- SPADA’S PAST AND RECENT EFFORTS IN BIODEFENSE

Past SPADA work- Generation of Standard Methods Performance Requirements (SMPR) Various SMPRs for PCR assay development for biodefense pathogens and sampling & testing methods Most recent: • Guidelines for PCR Assay designs in silico • Standards for Strain Characterization to establish provenance •Standards for soil matrices

Current SPADA effort we are focusing on NGS aspects

WG 1 – Detection Capabilities with NGS & Reference Databases (Dr. Minogue) WG 2 – Standards for NGS Biothreat Agent Detectors WG 3 – Validation of NGS and In Silico Processes (Dr. Sussman)

AOAC MYM 2023

UNCLASSIFIED

20

AOAC Annual 2023

SPADA WG 2 – STANDARDS FOR NGS BIOTHREAT AGENT DETECTORS

• Objective: To develop standards for NGS biothreat agent detectors that encompass both PCR-based amplicon sequencing (Amp Seq) and metagenomics (Meta Seq) applications • Scope • Scan end to end process to generate guidelines for platform agnostic adjudication of NGS results both for amp seq and meta seq data (WG2) and classify the samples for making actionable calls; follow the model of Minimum Information for Biological and Biomedical Investigations (a document like MIGS, MIMS, MIQE etc) • The best example: EMMI – qPCR, ddPCR quality and reporting guidelines • Topics Discussed during WG2 meetings • Why NGS for biothreat detection in biodefense/biosurveillance? • The current states of AmpSeq/MetaSeq for NGS biothreat agent detection • PCR-based sequencing application for Biodefense - AmpSeq • Metagenomic application perspectives-MetaSeq • Challenges • Questions to consider and discussion • End Product for this WG:

• Amplicon Sequencing Minimal Information (AsqMI): Quality and Reporting Guidelines for Actionable Calls in Biodefense Applications

UNCLASSIFIED

21

AOAC Annual 2023

EMMI- ENVIRONMENTAL MICROBIOLOGY MINIMUM INFORMATION

The Environmental Microbiology Minimum Information (EMMI) Guidelines: qPCR and dPCR Quality and Reporting for Environmental Microbiology | Environmental Science & Technology (acs.org)

UNCLASSIFIED

22

AOAC Annual 2023

CONTROLS..CONTROLS..CONTROLS…

The Environmental Microbiology Minimum Information (EMMI) Guidelines: qPCR and dPCR Quality and Reporting for Environmental Microbiology | Environmental Science & Technology (acs.org)

UNCLASSIFIED

23

AOAC Annual 2023

TESTING SCHEMES

24 The Environmental Microbiology Minimum Information (EMMI) Guidelines: qPCR and dPCR Quality and Reporting for Environmental Microbiology | Environmental Science & Technology (acs.org) UNCLASSIFIED AOAC Annual 2023

REPORTING ELEMENTS

The Environmental Microbiology Minimum Information (EMMI) Guidelines: qPCR and dPCR Quality and Reporting for Environmental Microbiology | Environmental Science & Technology (acs.org)

UNCLASSIFIED

25

AOAC Annual 2023

WHAT STANDARDS DO WE NEED FOR AMPSEQ AND METASEQ?

• Questions and points we considered when drafting guidelines • Upstream steps such as sample prep methods & PCR methods and suggest metrics to achieve? • Number of samples processed per experiment and sequencing run (cost vs quality) • Issues in processing multiple samples and how to control for spill over • What are appropriate controls for the experiment- process controls, PCR controls, positive and negative controls? • What are the appropriate reference materials for AmpSeq and MetaSeq? • Genomic DNA vs Whole cell • PCR step • The individual targets have varying efficiencies in PCR when run as a singleplex and as in multiplex- how do we account for this? • How do we set the threshold LoD? How do you make adjustments for target organism that is in low abundance with a weak assay?

UNCLASSIFIED

26

AOAC Annual 2023

• Sequencers/Sequencing/Sequences (WG 3)

WHAT STANDARDS DO WE NEED FOR AMPSEQ AND METASEQ?

• Sequencing platforms and associated variations • Workflows • Quality scores cut off for reads?--WG3 • What is the threshold number of reads for making a TP, FP calls? • What is the identify threshold for making FP, TN calls? • What is the threshold time for a sequencing run to make calls? • How do we treat the background organisms reads? • What do we do with un-mappable reads? • Reference Databases (WG1) • What should the blast database contain? Just the target amplicon sequences or more expanded sequences of organisms supposed to be contained in the matrices? What about the target organism and near neighbors WGS? • If we include everything what is quality of the sequences included? Does it matter to include incomplete genome sequences since we are looking at only amplicon sequences? • What are the matrix specific background organisms to include in the data base?

UNCLASSIFIED

27

AOAC Annual 2023

• The ASqMI: Amplicon Sequencing Minimal Information (AsqMI): Quality and Reporting Guidelines for Actionable Calls in Biodefense Applications WG2 GUIDELINES DOCUMENT

WG 1 and WG 3 lays the foundation for these 2 tasks (Mike Sussman) _Sequence quality from samples _Databases: targeted vs untargeted _Quality of sequences in DB ___VNGS _Amp Seq library methods _Target organisms_Inc _Target organisms_Exc _Matrix specific_organisms_Seq

UNCLASSIFIED

28

AOAC Annual 2023

CONTROLS IN PROCESSES

UNCLASSIFIED

29

AOAC Annual 2023

OVERALL PROCESS AND CONTROLS

UNCLASSIFIED

30

AOAC Annual 2023

ASQMI- PUBLICATION

Contributors • NIST • JHU-APL • Datirium, LLC • USDA • MRIGlobal • Neogen Food Safety • JPEO & • SPADA WG 2 members

UNCLASSIFIED

31

AOAC Annual 2023

1 - Introduction 2 - Challenges with Sample Matrices - & - The Need for Controls

3-Controls

3.1 - Negative controls 3.2 - PCR controls 3.3 - Inhibition control 4 - Library Preparation and Amplicon Sequencing

TABLE OF CONTENTS- OUTLINE

4.1 - Library preparation 4.2 - Amplicon sequencing

5 - Sequence Analysis

5.1 - Assessing read and base call quality 5.2 - Sequence cleanup and filtering 5.3 – Read classification and database selection

6 - Creating Response Actionable Calls

6.1 - Establishing thresholds for action

7 - Conclusions

8 – Supplementary files- mock data with various reporting elements and adjudication metrics

UNCLASSIFIED

32

AOAC Annual 2023

WHAT NEXT FOR SPADA?

• SMPRs (Standard Method Performance Requirements) for Amp Seq and Meta Seq Methods for assay developers • Follow the general paradigm of SMPRs • Provide metrics to be captured • For a specific amp seq method (Oxford Nanopore or Illumina) • With targets defined (target amplicons) • LoD defined • Reproducibility • Confidence defined • Threshold read counts defined • Threshold read quality according to VNGS • Database locked • Version control

AOAC MYM 2023

UNCLASSIFIED

33

AOAC Annual 2023

SMPRs FOR AMPLICON SEQUENCING ASSAYS

UNCLASSIFIED

34

AOAC Annual 2023

• Draft Version 23-8-11

DRAFT SMPR FOR AMP SEQ ASSAY

• AOAC SMPR  2023.XXX

• Standard Method Performance Requirements (SMPRs  ) for Detection of Multiple Biothreat Agent Organisms in Environmental Samples by Amplicon Sequencing

UNCLASSIFIED

35

AOAC Annual 2023

METHOD PERFORMANCE REQUIREMENTS

Parameter

Testing requirements

Performance criteria

Acceptable Minimum Detection Level (AMDL)

Co-inoculation of all target organisms as spores or vegetative cells a

2000 genome equivalents per organism per filter

POD AMDL

Target organisms at AMDL in matrix

≥0.95 with 95% confidence

POD 0

Near neighbor at 10 × AMDL and absence of target organisms in matrix

≤0.05 with 95% confidence

Inclusivity b

2x AMDL

All must test positive

Exclusivity b All must test negative a Every effort should be made to optimize the method to meet the criteria for co - inoculation of all target organisms. For any organisms that fail co - inoculation testing, test by individual inoculation. b Refer to Tables 4- 17 for Inclusivity and exclusivity panels. 10x AMDL

UNCLASSIFIED

36

AOAC Annual 2023

UNCLASSIFIED

37

AOAC Annual 2023

(mean NTC + 3*standard deviation) aligned read count

sample aligned read count

<

UNCLASSIFIED

38

AOAC Annual 2023

CONTACT

Shanmuga Sozhamannan, Ph.D Technical Coordinator DBPAO, CBRND Enabling Biotechnologies 240 529 8743 Shanmuga.Sozhamannan.ctr@army.mil Mr. Bryan Necciai Director, Defense Biological Product Assurance Office JPL CBRND EB 301 693 9386 bryan.d.necciai.civ@army.mil

Online jpeocbrnd.osd.mil

Public Affairs Office usarmy.apg.dod-jpeo-cbrnd.mbx.jpeo- cbd-public-affairs-office@army.mil Contracting Keith Batchelor| 240-586-1523 Lee Hess| 240-344-0462

@JPEOCBRND

Performance Requirements and Validation of Biothreat Amp Seq Methods Sharon L. Brunelle, Ph.D.

AOAC Technical Consultant sharon@brunellebiotech.com

AOAC INTERNATIONAL 137 th Annual Meeting & Exposition August 25 – 30, 2023 • New Orleans, LA

Standard Method Performance Requirements (SMPRs  ) for Detection of Multiple Biothreat Agent Organisms in Environmental Samples by Amplicon Sequencing

WG Draft August 23, 2023 In WG Review Public Comment Period Resolve Comments

Public Vote Publication

AOAC INTERNATIONAL 137 th Annual Meeting & Exposition August 25 – 30, 2023 • New Orleans, LA

• Address specific analytical needs o What is the purpose of the method? • Define the fitness-for-purpose characteristics of a method o Method design requirements o Method performance requirements • Provide guidance to method developers • Are the basis for method acceptance and approval

Standard Method

Performance Requirements (SMPRs)

AOAC INTERNATIONAL 137 th Annual Meeting & Exposition August 25 – 30, 2023 • New Orleans, LA

Description of Analytical Need

Intended Use: Use for analysis of aerosol collection dry filter unit (DFU) filters as part of concept of operation-based routine testing to make actionable calls. Applicability Simultaneous detection of Bacillus anthracis , Francisella tularensis , Yersinia pestis , Burkholderia mallei , Burkholderia pseudomallei , Brucella abortus , Coxiella burnetii , and Variola virus from DFU filters or polytetrafluoroethylene (PTFE) filters. A suitable method may target some or all of these agents as appropriate for the specific biodefense program or concept of operations. Analytical Technique Amplicon sequencing of double stranded DNA-containing organisms only. This assay does not target rRT-PCR assays that include a reverse transcription step.

Maximum Time to Results Maximum time for analysis of filters is 48 h.

AOAC INTERNATIONAL 137 th Annual Meeting & Exposition August 25 – 30, 2023 • New Orleans, LA

Key Definitions

AOAC INTERNATIONAL 137 th Annual Meeting & Exposition August 25 – 30, 2023 • New Orleans, LA Acceptable minimum detection level (AMDL) .—Predetermined minimum level of an analyte, as specified by consensus which must be detected by the candidate method at a specified probability of detection (POD). Dry filter unit (DFU).— System used to collect aerosolized particulates on a dry filter (typically 1 µm pore size, 3 mm thickness polyester felt, 47 mm in diameter) over a period of time. Polytetrafluoroethylene (PTFE) filter .—3 µm pore size, 47 mm diameter

System Suitability Tests and/or Analytical Quality Control

Suitable methods will include blanks and appropriate quality control measures as described in Table 1. Table 1. Method quality controls

Control Positive

Description

Implementation Must be analyzed for every sample or set of samples tested

Designed to demonstrate an appropriate test response. The positive control material should be included at a low but reliably detectable concentration and should monitor the performance of all assays in the panel. The purpose of using a low concentration of positive control is to demonstrate that the assays are performing at a previously determined level of sensitivity. It is recommended that a technique (i.e., unique distinguishable signature) is used to confirm whether the positive control is the cause of a positive signal generated by a sample. Designed to demonstrate that the amplicon sequencing panel does not produce any agent detection calls in the absence of target organisms. The purpose of this control is to rule out causes of false positives, such as contamination.

Negative

Must be analyzed for every sample or set of samples tested

Inhibition

AOAC INTERNATIONAL 137 th Annual Meeting & Exposition August 25 – 30, 2023 • New Orleans, LA Designed to specifically address the impact of a sample or sample matrix on the panel’s ability to produce amplicon sequences. This may be performed in the same reaction as target testing (e.g., by adding a non-target control to the sample) or can be performed in a separate reaction (e.g., by adding the target positive control).

Must be analyzed for every sample tested

System Suitability Tests and/or Analytical Quality Control

Sequencing and analysis workflow .— Detection thresholds .—The following thresholds are applied.

AOAC INTERNATIONAL 137 th Annual Meeting & Exposition August 25 – 30, 2023 • New Orleans, LA Criteria for amplicon and organism calls .—To make actionable calls, multiple targets per organism must be detected to achieve strain specificity. For example, Bacillus anthracis requires three targets with all three amplicon reads detected for an actionable call (see Table 18 for required targets per organism). (1) Per alignment threshold: using BLASTN, ≥90% identify across ≥90% alignment length is applied. (2) Per sample (bar code) threshold: a reference must have >2% of total aligned reads; i.e., it is highly unlikely for 50+ amplicons to be truly positive in one sample. (3) Per flow cell threshold: based on NTC aligned read counts per amplicon. Amplicon “detection” will be masked in a sample if sample aligned read count is < (mean NTC + 3 SD) aligned read count.

Validation Guidance

AOAC INTERNATIONAL 137 th Annual Meeting & Exposition August 25 – 30, 2023 • New Orleans, LA o For example, “Method X is applicable to the simultaneous detection of Bacillus anthracis , Francisella tularensis , Yersinia pestis , Burkholderia mallei and Burkholderia pseudomallei , and individual detection of Brucella abortus and Variola virus, on dry filter unit filters according to SMPR 2023.XXX criteria.” • Inclusivity and exclusivity panel organisms provided per organism target • In silico analysis may be used to guide the wet-lab inclusivity, exclusivity, and environmental organism testing. Specific bioinformatics requirements provided. • Validation is carried out according to OMA Appendix I. • Method developers should make an effort to optimize method parameters to meet the performance requirements in Table 1 when all target organisms are co-inoculated at the AMDL on filters. Any target organisms not detected at the AMDL under co-inoculation conditions should be validated under single inoculation conditions.

PARAMETER

TESTING REQUIREMENTS

PERFORMANCE CRITERIA

Acceptable Minimum Detection Level (AMDL)

Co-inoculation of all target organisms as spores or vegetative cells Target organisms at AMDL on filters Near neighbor at 10 × AMDL and absence of target organisms on filters

2000 genome equivalents per organism per filter

Method Performance Requirements

POD AMDL

≥0.95 with 95% confidence ≤0.05 with 95% confidence

POD 0

Inclusivity

2x AMDL

All must test positive All must test negative

Exclusivity

10x AMDL

AOAC INTERNATIONAL 137 th Annual Meeting & Exposition August 25 – 30, 2023 • New Orleans, LA

AOAC INTERNATIONAL 137 th Annual Meeting & Exposition August 25 – 30, 2023 • New Orleans, LA Guidance Documents for Biothreat Agent Method Validations OMA Appendix I – BTAM Validation Guidelines (2009) OMA Appendix O – Environmental Factors OMA Appendix P – Soil Collection and Characterization OMA Appendix Q – In Silico Guidelines OMA Appendix R – Verifying and Documenting Cultures VNGS – Verified Next Generation Sequences ASqMI – Amplicon Sequencing Quality and Reporting Guidelines

Inclusivity/Exclusivity

AOAC INTERNATIONAL 137 th Annual Meeting & Exposition August 25 – 30, 2023 • New Orleans, LA o Perform wet lab testing on inclusivity strains with the highest probability of non-detection and the exclusivity and environmental organisms with the highest probability of detection based on primary and secondary structure assessments. • Wet lab testing: o Inclusivity organisms tested at 2x AMDL o Exclusivity organisms tested at 10x AMDL o Environmental Factors according to OMA Appendix O • In silico analyses: o Primary sequence alignments of primers with inclusivity, exclusivity, and environmental genomes o Unimolecular folding of primers and their target regions o Hybridization thermodynamics of primers to their target regions

Probability of Detection

• Probability of Detection (POD) – the probability of a positive response for a qualitative method for a given matrix at a given analyte level or concentration. POD is concentration dependent and is measured as a proportion of positive outcomes. POD = / o If replicate test portion data are coded ”0” for negative and “1” for positive, then POD is the mean of the data set. • LPOD – the POD value obtained from combining all valid collaborator data sets for a method for a given matrix at a given analyte level or concentration. LPOD = � ∑ ∑

AOAC INTERNATIONAL 137 th Annual Meeting & Exposition August 25 – 30, 2023 • New Orleans, LA

• High impact situations • Low tolerance for false- negative results o ≥95% detection with 95% confidence at the AMDL • Low tolerance for false- positive results o ≤5% detection with 95%

Actionable Results for Biodefense

confidence in the absence of analyte

AOAC INTERNATIONAL 137 th Annual Meeting & Exposition August 25 – 30, 2023 • New Orleans, LA

POD Model Application for BTAM

• SMPRs define the Acceptable Minimum Detection Level (AMDL) o Predetermined concentration that must be detected with an estimated 5% lower confidence limit on POD of 0.95 or higher – one- sided 95% Confidence Interval! o Requires a sample size of 96 test portions at AMDL with at least 95/96 testing positive

AOAC INTERNATIONAL 137 th Annual Meeting & Exposition August 25 – 30, 2023 • New Orleans, LA