AOAC SPADA Meeting

Sign-In Here: http://bit.ly/spada-mym

WiFi: Marriott Conference Password: aoac2020

S TAKEHOLDER P ROGRAM ON A GENT D ETECTION A SSAYS Thursday, March 12, 2020 | 9:00AM – 11:00AM

Gaithersburg Marriott Washingtonian Center 9751 Washingtonian Blvd. Gaithersburg, Maryland, 20878 Room: Salon G

STAKEHOLDER PANEL AGENDA

I. Introductions and Call to Order (9:00am – 9:15am) Palmer Orlandi, AOAC INTERNATIONAL a. Review of SPADA Sponsors: JPEO JPL CBRND Enabling Biotechnologies Biological Product Assurance Office & CBRN Defense Division Deputy Under Secretary of the Army – Test & Evaluation II. Review of Recent SPADA Work (9:15am – 10:15am) a. Recommendations for Development of Molecular Assays for Microbial Pathogen Detection Using Next Gen In- Silico Analysis John SantaLucia, Wayne State University & DNA Software, Inc. & Shanmuga Sozhamannan, Logistics Management Institute Support Defense b. Strain Verification: Verifying and Documenting the Relationship Between Microbial Cultures Katalin Kiss, Senior Manager, Manufacturing Science and Technology, American Type Culture Collection & Dave Rozak, Deputy Director, Unified Culture Collection, Core Laboratory Services, USAMRIID c. Voluntary Consensus Standard for Collection and Use of Soils for Biothreat Agent Method Validation and Site Assessments Linda Beck, JRAD & Morgan Minyard, DTRA

III. Perspectives on Phenotype-Genotype Correlation (10:15am-10:30am)

a. Successes by the National Antimicrobial Resistance Monitoring System (NARMS) Gregory Tyson, Center for Veterinary Medicine, US Food and Drug Administration

IV. Need to Update Molecular Validation Criteria for Biothreat Agents (10:30am – 10:45am) Sharon Brunelle, AOAC INTERNATIONAL

V. Discussion: Analytical Needs of the Biothreat Community (10:45am – 11:00am)

NO GOVERNMENT FUNDS HAVE BEEN USED IN THE PROVISION OF FOOD FOR THIS MEETING

3/11/2020

SPADA- IN SILICO ANALYSIS STANDARDS FOR ASSAY DEVELOPMENT Presented at AOAC Mid Year Meeting on March 12, 2020

Shanmuga Sozhamannan Technical Coordinator, LMI supporting DBPAO, CBRND-JPL-EB, JPEO John SantaLucia President & CEO, DNA Software, Inc.

UNCLASSIFIED Distribution Statement A: Approved for public release; distribution is unlimited.

1

1

RECOMMENDATIONS FOR DEVELOPING MOLECULAR ASSAYS FOR MICROBIAL PATHOGEN DETECTION USING MODERN IN SILICO APPROACHES

Working Group Members • Linda Beck , DUSA TE; JPEO JPM Guardian DBPAO; (Co-Chair) • Shanmuga Sozhamannan , DoD, JPEO, JPMG, DBPAO (Co-Chair)

o Nancy Lin , NIST o Timothy Minogue, USAMRIID o Victoria Olson , CDC o Richard Ozanich, PNNL o Kristian Roth , FDA o John Santalucia, Jr. Wayne State University

o Jessica Appler, HHS o Trevor Brown, JPM MCS/JPEO o Sharon Brunelle, AOAC o Don Cronce, DTRA o Matthew Davenport, DHS o Jason Gans , LANL o Bruce Goodwin, DBPAO o Scott Jackson, NIST o Jeff Koehler , USAMRIID

o Sanjiv Shah, EPA o Ricky Soong , FDA o Randy Vines, ATCC

AOAC_SPADA_ISA_presentation

3/12/2020

2

2

1

3/11/2020

TABLE OF CONTENTS

• Outline of talk

• Background and Rationale • Assay Development Process: Traditional vs modern • Assay Design • Metrology for In Silico Analysis • Assay Development and Characterization • Regulatory Considerations for Nucleic Acid-based Clinical Assays • Glossary

AOAC_SPADA_ISA_presentation

3/12/2020

3

3

AVAILABILITY OF WHOLE GENOME SEQUENCES VS. ASSAY DESIGN TIME FRAME

 Databases are growing exponentially  We should use

this information in a modern design pipeline.

AOAC_SPADA_ISA_presentation

3/12/2020

4

4

2

3/11/2020

SIGNATURE SEQUENCE IDENTITIES OF THE TARGET SEQUENCES OF INC/EXC STRAINS IN SPADA PANELS

100% match between amplicon and target

99% match between amplicon and target  Unnecessary wet lab testing of assays in strains with perfect matches to the assays

3/12/2020

AOAC_SPADA_ISA_presentation

5

5

DELAYS IN OBTAINING REFERENCE MATERIALS FROM OUTBREAKS

Examples of the timeline for obtaining Ebola reference materials from Africa

 Delays in obtaining pathogen reference materials may hamper development of new and or ‘old and improved’ detection assays in response to discovery of signature erosion and make an impact in the field in a timely manner.

AOAC_SPADA_ISA_presentation

6

3/12/2020

6

3

3/11/2020

TRADITIONAL, LOW THROUGHPUT VS. MODERN HIGH THROUGHPUT ASSAY DEVELOPMENT PIPELINE

 Goal is to minimize the expensive and time ‐ consuming experimental work, by taking advantage of the sequence databases and doing thorough bioinformatics analysis.

AOAC_SPADA_ISA_presentation

7

3/12/2020

7

TRADITIONAL PRIMER DESIGN APPROACH

 The software tools depicted are only example suggestions and not an endorsement of specific tools. Any other software with an equivalent functionality can also be used for producing similar outputs.

AOAC_SPADA_ISA_presentation

8

3/12/2020

8

4

3/11/2020

TOTAL NUMBER OF BACTERIAL GENOMES IN GENBANK VS THE PERCENTAGE OF COMPLETE BACTERIAL GENOMES AS A FUNCTION OF YEAR

 Databases are growing exponentially, but more incomplete genomes.  Use complete genome sequences for assay design  Use all sequences for assay validation

AOAC_SPADA_ISA_presentation

9

3/12/2020

9

SUMMARY OF RECOMMENDATIONS FOR PCR PRIMER DESIGN

• Full length genomes or genes • Near Neighbor sequences • Gather contaminating sequences

 In the publication, we provide reasons for these recommendations and give alternatives.

AOAC_SPADA_ISA_presentation

10

3/12/2020

10

5

3/11/2020

RECOMMENDATIONS FOR PCR EXPERIMENTS

AOAC_SPADA_ISA_presentation

11

3/12/2020

11

MODERN CONSENSUS DESIGN PARADIGM FOR PATHOGEN DETECTION BY PCR

AOAC_SPADA_ISA_presentation

12

3/12/2020

12

6

3/11/2020

CASE STUDY: MODERNIZATION OF TESTS FOR BIOLOGICAL THREAT AGENTS • Project funded by DBPAO (Project managed by JHU-APL) o Previous PCR assays only available in singleplex. o Signature erosion has occurred with time. o Multiplexing would save money on large number of environmental tests. o Previously multiplex reactions were difficult to design and involved extensive experimental optimization. • DNA Software, Inc. used database driven approach described in the SPADA manuscript • Design of Bacterial 35-plex and Virus 18-plex • Wet-Lab testing done by MRI Global • Preliminary Results for bacterial panel: (used synthetic targets for first round of testing) o 35/35 assays worked in singleplex reactions with LOD below 50 molecules o 35/35 of the assays showed no amplification in the NTC o 30/35 of the targets detected in multiplex format (without any optimization, LOD <100)

AOAC_SPADA_ISA_presentation

13

3/12/2020

13

GLOSSARY

• Standards • “something set up and established by authority as a rule for the measure of quantity, weight, extent, value, or quality”- there is no ambiguity here. If we are adopting a standard, we follow the rules set forth here • Guidelines • a general rule, principle, or piece of advice. • a piece of information that suggests how something should be done - there is some inducement to follow these • Recommendations • a suggestion or proposal as to the best course of action, especially one put forward by an authoritative body and its beneficial to follow these • Considerations • Something to think about; no obligation to follow • Best Practices • Following these practices will potentially yield a better assay

14

14

7

3/11/2020

GLOSSARY (CONT.)

• Assay Development • Refers to the entire process from target identification/ selection, assay design, validation, optimization, matrix testing, preparation of the packages for DoD acceptance, FDA EUA, 510k etc. • Target Selection • Entails identifying suitable “unique regions” in the genome of interest for assay design. This may be longer than the actual PCR amplicon. • Assay (region that contains the assay components) design • In silico process (extensively described by John in his talk) • Assay Optimization (IS) • In silico optimization of parameters • Assay Optimization (Wet lab) • Refers to wet lab testing of the designs from In silico process under different conditions of PCR (concentrations of components, temperature etc.)

15

15

GLOSSARY (CONT.)

• Sensitivity- The true positive rate. In simple terms, the sensitivity of the assay is the number of sick people that are correctly identified as sick.

ൌ ൅ ൌ ൅ ൌ ൅ ൅ ൅ ൅

• Specificity- The true negative rate. In simple terms, the specificity of the assay is number of healthy people (i.e. without disease) that are correctly identified as healthy.

• Limit of detection (LOD) – Minimal quantity of target that can be detected by the assay • Accuracy – Rate of the assay giving the correct result

16

16

8

3/11/2020

GLOSSARY (CONT.)

• Inclusivity panel- A panel of strains of the intended PCR assay target organism. Ideally, the assay is expected to be positive for all the panel strains [exceptions i.e., false negatives) are expected] and need to be captured and ideally includes members of its entire genetic diversity • Exclusivity panel- A panel of near neighbors that are expected to be negative for the assay [exceptions i.e., false positives are expected] and need to be captured • Background panel – A panel of organisms found in the matrix of the assay. (e.g. human genome, human microbiome, soil microbes, etc.) • Assay Improvement- Any improvement of existing assays in response to a variety of outcomes from initial testing or field testing. E.g., signature erosion, failure of assays in certain conditions or specific matrices, or in multiplex formats

17

17

GLOSSARY (CONT.)

• Signature Erosion • Emergence of mutations in the sequences of the assay target regions that may lead to assay failure; this may happen due to natural course of evolution especially in virus due to genetic drift or shift or deliberate acts • Reference Materials • Assay template materials used in development, testing, validation, test and evaluation of the assay; e.g., live organisms, inactivated organisms, genomic materials (DNA or RNA), synthetic plasmids, or even synthetic amplicons • Assay Performance (LoD) • Establishing LoD in various matrices • Advance development • Entails validating the assay performance for specific use scenarios

18

18

9

3/11/2020

GLOSSARY (CONT.)

• Matrix testing • Testing the final assay in specific matrices relevant to the end user; e.g., clinically relevant matrices such blood, sputum etc. and environmentally relevant matrices such as soil. • Matrix testing • Mock clinical trials • Data packages • Refers to DoD specific CB56 or FDA specific EUA or 510K that contain all the required information and data • Assay performance monitoring (assay stewardship) • In silico validation of assay performance with the availability of new genomic sequences over time • Test and Evaluation • FDA Vetting

• EUA • 510k • FDA licensure • Matrix panel

19

19

CONTACT Shanmuga Sozhamannan, Ph. D. Shanmuga.Sozhamannan.ctr@mail.mil 301 619 8430 John SantaLucia, Ph. D. john@dnasoftware.com 734 282 1063

20

10

3/11/2020

MICROBIAL CULTURE VERIFICATION

Katalin Kiss, PhD, PMP® | ATCC David Rozak, PhD | USAMRIID 12 March 2020

1

THE ISSUE

Microbial cultures are dynamic systems, which can undergo significant changes as they are handled and propagated in laboratory settings. While storage, propagation (passing), and distribution is necessary to ensure the availability of the organism for basic and applied research, these processes can make it difficult and misleading to compare experimental results obtained from different locations and times. The careful handling and verification of microbial cultures is essential for comparing experimental results.

2

1

3/11/2020

A RECENT EXAMPLE

Strain

Isolate

Mouse LD50 (CFU)

F. tularensis SCHU S4 F. tularensis SCHU S4 F. tularensis SCHU S4 F. tularensis SCHU S4

FT10

60

FT4

106 110

FT12

FRAN016

1660

Dave Waag, USAMRIID 2006

3

WORKING GROUP OBJECTIVES

Develop a consensus standard to provide confidence and assurance in the identification of the strains being used in testing. A consensus standard would improve the credibility of validation test results since evaluators will have a standard procedure recognized by the community that can be used to authenticate the strain of the cultures used in assay validation.

4

2

3/11/2020

MEMBERS

Linda Beck, JRAD (Co-Chair) Shanmuga Sozhamannan, DoD, JPEO, JPMG, DBPAO (Co-Chair)

Nancy Lin, NIST Timothy Minogue, USAMRIID Jason Opdyke, JPEO David Rozak, USAMRIID Mark Scheckelhoff, Armed Forced Surveillance Branch Sanjiv Shah, EPA Chuck Young, JHU/APL

Brian Bennett, Dugway Proving Ground Cory Bernhards, ECBC Rick Blank, NBACC Trevor Brown, JPEO Ryan Cahall, Censeo Insight /DUSA T&E

Randy Hoffman, ECBC Paul Jackson, LLNL (Ret.)

Scott Jackson, NIST Katalin Kiss, ATCC

5

TIMELINE

 Working Group Launch (October 16-17, 2018)  Six teleconferences (October 2018 – April 2019)  Drafted “Guidelines for Verifying and Documenting the Relationships Between Microbial Cultures”  Public comment period (June 4, 2019 – July 12, 2019)  Document presented for SPADA review and approval (13 August 2019)

6

3

3/11/2020

SOLUTION

Develop a consensus standard for tracking and verifying the relatedness of microbial test and index cultures.

7

MAINTAINING INDEX CULTURES

Francisella tularensis SCHU S4 UCC AC120318A

National prototype kilogram K20

8

4

3/11/2020

VERIFYING RELATEDNESS

Index

0

Lot B

Lot C

1

1

Document Trail

Empirical Assays

Lot D

Lot E

Lot F

2

2

2

Lot G Test 3

3

9

CONCEPTS AND DEFINITIONS

An extensible study is a research program whose results and conclusions are expected to apply equally to test and index cultures. In an extensible study, the test culture is the microbial culture that is being evaluated. The index culture is the culture to which the results are to be applied. Both the test and index cultures must be traceable cultures , meaning that each has a unique identifier (e.g., lot/batch/subculture, etc., as appropriate) and well- documented propagation history. Culture verification is the process by which the organisms in a test culture are shown to be sufficiently related to those in an index culture to allow the meaningful extension of experimental results from one culture to the other.

10

5

3/11/2020

CONCEPTS AND DEFINITIONS (CONTINUED)

Propagation history describes a test culture’s step-by-step derivation from the index culture via a series of propagation events. Orthogonal testing is the use of functionally independent assays to verify the genotypic and phenotypic relatedness of test and index cultures. Application-oriented testing is designed to assess the relatedness of test and index cultures with respect to the specific genotypic or phenotypic phenomena being evaluated in the extensible study. Culture verification statements provide a convenient mechanism for documenting the relatedness of microbial cultures used in extensible studies.

11

ROLES AND RESPONSIBILITIES

1. The sponsor identifies the index cultures to which test cultures must relate. 2. It’s the performer prepare culture verification statements, which demonstrate that test cultures are sufficiently related to the index cultures. 3. The culture producer provides the performer with any nonproprietary information that can help them demonstrate the relatedness of the test and index cultures.

12

6

3/11/2020

CULTURE VERIFICATION STATEMENT

 Propagation History  Orthogonal Test Results  Application-Specific Test Results

13

PRODUCTION HISTORY o Producer o Lot Number

o Lineage o Method o Storage

14

7

3/11/2020

ORTHOGONAL TESTING o Morphology o Genotype o Metabolism o Immunological features o Molecular features o Microbial functions o Virulence

15

APPLICATION-ORIENTED TESTING

Application

Possible Assays

Molecular Assays

Target specific sequencing or PCR

Immunoassays

Target specific ELISA, Magpix, DFA, IFA

Therapeutics

Virulence, antimicrobial resistance and sensitivity Virulence, gene expression, host immune response

Vaccines

16

8

3/11/2020

CHALLENGES

 The working group had to focus on practices that could be maintained by a cross section of laboratories, from cGMP manufacturers to small academic laboratories.  The Guidelines are meant to work within the framework of existing standard practices that conform to ISO standards and even protect intellectual property.  Working with these boundaries- the guidelines stresses traceability between cultures using:

 Unique culture identifiers  Propagation history  Orthogonal testing

17

17

CHALLENGES: CREATING A UNIQUE IDENTIFIER.

Challenge was that the format of the unique identifier could not be mandatory. • The unique IDs can be made at random or have embedded data in them- like date of manufacture or individual growing it or future storage location. • Smaller production facilities, academic labs can resort to “smart numbers” that embed data. • Manufacturers may need to rely on system-generated, random lot numbers, to avoid duplicate lot numbers. Solution was to allow the culture producer to generate their own format.

18

18

9

3/11/2020

CHALLENGES: CULTURE VERIFICATION

Historical information may not be available, incomplete or incoherent:

History has to start with documentation at some point. Study sponsors can determine if information is adequate or if additional verification testing is needed.

Propagation protocols may be proprietary: Culture producer should have the step-by-step procedure recorded, but it is possible that some information needs to be redacted. Study sponsors can determine if this is acceptable or if additional information is needed Culture Verification and Orthogonal Testing: Culture verification requires the historical documentation and propagation protocols. There is not a list of assays that are required for verification. This could be cost prohibitive to many laboratories. Instead, assays are to be selected by the culture producers for their verification.

19

19

CHALLENGES: APPLICATION SPECIFIC TESTING Challenge to this specification was how to consolidate application specific testing results as part of a provenance statement or Culture Verification statement. This activity requires curation of the Culture Verification statements, which could be labor intensive.

This is a nice to have but not required for the Culture Verification statement. If it is included, it should demonstrate an alignment between the test sample and index cultures.

20

20

10

3/11/2020

VERIFICATION IS MORE THAN JUST IDENTIFICATION

Verification is more than just culture identification according to the Guidelines. Verification of strains in a timely manner has safety and legal implications. Culture producers always want to make sure you are working at the correct biosafety level, with the correct personal protective equipment, and you are in compliance with all regulations that apply to your culture.

21

THANK YOU!

22

11

Background

Cont… Recommendations to evaluate candidate assays using soil samples was added to Part 2 of the Environmental Factors appendix in 2017: 2.2: Soil Testing “Airborne soil particles may constitute a significant challenge to the analysis of collected aerosol samples by polymerase chain reaction (PCR) assays. Soils contain genomic materials or nucleic acid fragments of countless archaebacterial, bacterial, and eukaryotic organisms. Some of the more common soil organisms can be anticipated. Soils may also contain unanticipated components that interfere with extraction, denaturation, polymerization, or annealing reactions. Therefore, determining the effect of a variety of representative soils on the robustness of a PCR assay is an important first step.” However, instructions in the Soil Testing section are extremely limited because there was not a consensus on how to conduct soil testing , nor what kinds of soil samples to use (T&E; assay development; industry/gov’t)

3

Background

Soils are very complex and unique

In US alone, over 19,000 soils have been identified The study of soil science includes taxonomy, chemistry, physics and biology in addition to agricultural and conservation practices.

Soil are: •Plant growth medium •Regulator for water supply •Recycler of raw materials •Habitat for Soil Organisms •Engineering Medium •Environmental interface

https://www.nrcs.usda.gov/wps/portal/nrcs/detail/nj/soils/?cid=nrcs141p2_018867

4

Background

Then…

Soil Testing Working Group launched in order to attempt to address concerns using soil in assay development. Working group an interagency effort in order to include experiences from verity of scientists and to address as many concerns about soil as a matrix during Biothreat testing as possible.

5

Soil Testing Working Group Members

Linda Beck, JRAD – DUSA TE; JPEO JPM Guardian DBPAO (Co ‐ Chair) Morgan Minyard, DTRA (Co ‐ Chair) Brian Bennett, Dugway Proving Ground Sherry Blight, Battelle Chris Bradburne, JHU/APL Ryan Cahall, Censeo Insight/DUSA T&E

Don Cronce, DTRA Jason Gans, LANL Paul Jackson, LLNL (Ret.)

Scott Jackson, NIST Kevin Kearns, JPEO Jeff Koehler, USAMRIID Nancy Lin, NIST Richard Ozanich, PNNL Frank Schaefer, EPA (Ret.) Sanjiv Shah, EPA Shanmuga Sozhamannan, JPEO, JPMG, DBPAO Randy Vines, ATCC

6

6

Soil Testing Working Group

Scope of Work • Develop standards to aid both assay developers and evaluators • Provide an increased confidence level for the robustness of assays with regards to soil contamination • Develop the standard methodology for soil collection and characterization • Develop or identify protocols/methods for testing with soils • Identify a repository for the soil samples 7

7

Soil Testing Working Group Work to Date

• Working Group Launch (October 16 ‐ 17, 2018) • Five teleconferences (October 2018 – April 2019) • Drafted “ Voluntary Consensus Standard for Collection and Use of Soils for Biothreat Agent Method Validation and Site Assessments” • Public comment period (June 4, 2019 – July 12, 2019) • Comments reviewed and document prepared for SPADA review and approval

8

Soil Document Key Points

• Provide Background Information on Soil – Terms and Definitions – Soil as a matrix • How to select Soil to use for Testing • Processing and Using Soil • Additional Resources • Experimental Set ‐ up recommendations

9

10 Guidance for Soil Collection, Characterization and Application for Biothreat Agent Detection Method and Site Evaluations • Provides guidance on the standardization of practices for collection, application ‐ driven processing, characterization and use of soil as a sample matrix or potentially interfering substance in biothreat agent detection assays (validation and evaluation) • Provides guidance on the standards for soils used in site assessments, evaluation of biothreat agent decontamination and remediation procedures • Considerations for the development of standard reference materials • Special Considerations: Clay content/soil texture, pH, microorganism content and characterization, moisture, soil horizons, organic matter (carbon content)

10

Discussion?

11

3/11/2020

Genomics to Predict Antimicrobial Resistance: Successes by the National Antimicrobial Resistance Monitoring System (NARMS)

Gregory Tyson, Ph.D. Research Microbiologist FDA, Center for Veterinary Medicine

1

Outline

• Background on NARMS • Antimicrobial susceptibility testing • Genomics for resistance detection

• Remaining challenges • Future opportunities

2

2

1

3/11/2020

NARMS Surveillance

Retail Meats Chicken Turkey

Humans

Food Animals

Chickens Turkeys

Physician Visit

Pork Beef Seafood

Swine Cattle

Local Lab

Random sampling of national production via federally inspected processing plants

Random stratified sampling in 23 States

54 + State Lab

Centers for Disease Control and Prevention

Food and Drug Administration

US Department of Agriculture

Data Integration

NARMS bacteria Salmonella Campylobacter E. coli Enterococcus

3

Use of NARMS data

• Track consequences of antimicrobial usage in animals • Preapproval of antimicrobials in food animals • Assess interventions • Develop FDA policies • Identify and respond to outbreaks

Attribution

AMU/AMR

Burden of Illness

Trends

Baseline

Spread

Interventions/ Evaluations

4

4

2

3/11/2020

Antimicrobial Susceptibility Testing • Antimicrobial susceptibility testing by broth microdilution on 96 ‐ well plates – Uses standard methods of the Clinical and Laboratory Standards Institute (CLSI) • Susceptibility testing involves detecting minimum inhibitory concentrations (MIC) – High numbers indicate resistance (µg/L)

Growth No growth

Concentration (µg/mL)

16 32 2 4 8

MIC

5

5

Antimicrobial Resistance Mechanisms

• Resistance can occur by a number of mechanisms – Mutation of target (mutation of DNA gyrase for quinolones) – Efflux of the antibiotic (TetA ‐ D tetracycline efflux pumps) – Cleavage of the antibiotic ( β‐ lactamases) – Modification of antibiotic or its target (aminoglycoside phosphotransferase)

Wild ‐ type

Mutated

Gene acquired

efflux

modification

E. coli

E. coli

6

6

3

3/11/2020

Genotype ‐ Phenotype Correlation

Phenotypic susceptibility

NAL FIS CHL TET STR

Sequencing/ genotyping

GyrA (S83L)

sul2 floR tetA strA/strB

7

7

Why might genomics fail to detect AMR?

• Resistance genes not expressed • Lack of knowledge of mechanisms

• Gene disruption • Epigenetic effects • Efflux pumps

8

8

4

3/11/2020

Genotype ‐ phenotype correlations

0 10 20 30 40 50 60 70 80 90 100 Correlation (%)

E. coli

Salmonella Campylobacter

9

9

Genotype ‐ phenotype correlations

10

10

5

3/11/2020

NARMS in the age of genomics

• Old way: Emerging resistance phenotype (time scale=years) – PCR for known genes – Sanger sequencing • New way: single sequenced isolate can be signal – Have context for finding based on previous results – Can use data in public domain

11

11

NARMS Sequencing Strategy

• All Salmonella • All Campylobacter • All E. coli • Select Enterococcus • Do repeat testing when mismatches occur FDA ‐ CVM USDA ‐ FSIS CDC* Salmonella 6,668 6,629 113,680 Campylobacter 2,935 8,360 13,377 E. coli 2,356 2,724 35,882 *not all these genomes are from NARMS sampling

12

12

6

3/11/2020

NARMS Genomic Surveillance

Sequence isolates

Identify resistance genotypes

Compare to existing genotypic/phenotypic results

MDR plasmid

Increased prevalence of known mechanism

Unusual combination of resistance genes

New in the U.S. or in specific source

New mechanism

13

13

Resistance reporting

14

14

7

3/11/2020

National Center for Biotechnology Information (NCBI) Pathogens https://www.ncbi.nlm.nih.gov/pathogens/

15

15

Challenges of genomics for AMR detection

• Clinical hesitancy to adopt • Never have 100% correlation with phenotypes • Time to accept new technique • Potential to miss new mechanisms • Lack of knowledge for certain bacterial species

16

16

8

3/11/2020

Opportunities: the case of mcr ‐ 1 • Colistin resistance gene identified in late 2015 • FDA did rapid retrospective surveillance of this gene • Identified none in U.S. among 42,000 public sequences • Three weeks later it was identified in Denmark

17

17

Artificial intelligence/machine learning • Can identify new resistance mechanisms, overcome database limitations

Training datasets (phenotypic, sequencing data)

Model

New sequencing data

Predicted phenotypes

18

18

Nguyen et al. J Clin Microbiol. 2019 Jan 30;57(2).

18

9

3/11/2020

Summary

• WGS accurately predicts phenotype (and MIC) • Rapid detection of resistance emergence and spread

• Retrospective resistance surveillance • Improved public health decision ‐ making • Global comparison of resistance

19

19

Acknowledgements

• FDA/CVM • FDA/CFSAN

• CDC

• NIH/NCBI

• USDA/FSIS • USDA/ARS

• Public health laboratories and universities

20

10

AOAC STAKEHOLDER PANEL ON AGENT DETECTION ASSAYS (SPADA)

Updating Molecular Validation Criteria for Biothreat Agents

Sharon Brunelle, Ph.D. March 12, 2020

AOAC Midyear Meeting Gaithersburg Marriott Washingtonian Center

1

Validation Guidelines for Biothreat Agent Methods (BTAM) OMA Appendix I: AOAC INTERNATIONAL Methods Committee Guidelines for Validation of Biological Threat Agent Methods and/or Procedures http://www.eoma.aoac.org/appendices.asp • Approved in 2009 • Published in 2012 • Scope includes immunoassays and molecular assays for organisms and toxins

2

Validation Guidelines for Biothreat Agent Methods (BTAM) • Single Laboratory Validation (SLV) – Inclusivity/Exclusivity Study (Selectivity)*

– Laboratory Matrix Study* – Intended Use Matrix Study* – Environmental Interference Study* – Robustness

– Product Consistency and Stability – Instrument Variation (if applicable) • Multi ‐ laboratory Validation (MLV, Collaborative Study) – Intended Use Matrix Study *Repeated by one independent laboratory 3

3

New SPADA Recommendations and Standards

• Recommendations for Development of Molecular Assays for Microbial Pathogen Detection Using Next Gen In ‐ Silico Analysis • Affects inclusivity/exclusivity for molecular methods • Strain Verification: Verifying and Documenting the Relationship Between Microbial Cultures • Provides clarity for documentation of inclusivity/exclusivity strains • Voluntary Consensus Standard for Collection and Use of Soils for Biothreat Agent Method Validation and Site Assessments • Provides clarity for matrix and environmental interference studies

4

Revisions to BTAM Guidelines

• Inclusivity/Exclusivity Study Revisions – How to select strains for in ‐ silico vs. wet lab testing – Application to WGS • ISO TC34/SC9/WG25 – Draft Standard “Whole genome sequencing for typing and genomic characterization of foodborne bacteria – General Requirements and Guidance” Errol Strain, FDA CVM, WG25 Chair – Data outputs – Metadata – Sequence Repository

5

Revisions to BTAM Guidelines

• Inclusivity/Exclusivity Study Revisions (cont.) – Application to culture ‐ independent diagnostics and metagenomics – Include recommendations for strain verification for cultural testing – How to assess phenotypic information (e.g., antibiotic resistance)

6

Revisions to BTAM Guidelines

• Matrix and Environmental Interference Studies – Now

• Revise to refer to guidance for collection and use of soils – Future • Develop chemistry ‐ focused soil guidance • Develop guidances for other matrices (water, air, plants, marine organisms, etc.)

7

Appendix W

POLICY AND PROCEDURES ON VOLUNTEER CONFLICT OF INTEREST

Statement of Policy

While it is not the intention of AOAC INTERNATIONAL (AOAC) to restrict the personal, professional, or proprietary activities of AOAC members nor to preclude or restrict participation in Association affairs solely by reason of such activities, it is the sense of AOAC that conflicts of interest or even the appearance of conflicts of interest on the part of AOAC volunteers should be avoided. Where this is not possible or practical under the circumstances, there shall be written disclosure by the volunteers of actual or potential conflicts of interest in order to ensure the credibility and integrity of AOAC. Such written disclosure shall be made to any individual or group within the Association which is reviewing a recommendation which the volunteer had a part in formulating and in which the volunteer has a material interest causing an actual or potential conflict of interest. AOAC requires disclosure of actual or potential conflicts of interest as a condition of active participation in the business of the Association. The burden of disclosure of conflicts of interest or the appearance of conflicts of interest falls upon the volunteer. A disclosed conflict of interest will not in itself bar an AOAC member from participation in Association activities, but a three-fourths majority of the AOAC group reviewing the issue presenting the conflict must concur by secret ballot that the volunteer's continued participation is necessary and will not unreasonably jeopardize the integrity of the decision-making process. Employees of AOAC are governed by the provision of the AOAC policy on conflict of interest by staff. If that policy is in disagreement with or mute on matters covered by this policy, the provisions of this policy shall prevail and apply to staff as well. 1. A volunteer who is serving as a committee member or referee engaged in the evaluation of a method or device; who is also an employee of or receiving a fee from the firm which is manufacturing or distributing the method or device or is an employee of or receiving a fee from a competing firm. 2. A volunteer who is requested to evaluate a proposed method or a related collaborative study in which data are presented that appear detrimental (or favorable) to a product distributed or a position supported by the volunteer's employer. 3. A referee who is conducting a study and evaluating the results of an instrument, a kit, or a piece of equipment which will be provided gratis by the manufacturer or distributor to one or more of the participating laboratories, including his or her own laboratory, at the conclusion of the study. 4. Sponsorship of a collaborative study by an interest (which may include the referee) which stands to profit from the results; such sponsorship usually involving the privilege granted by the investigator to permit the sponsor to review and comment upon the results prior to AOAC evaluation. Illustrations of Conflicts of Interest

5. A volunteer asked to review a manuscript submitted for publication when the manuscript contains information which is critical of a proprietary or other interest of the reviewer.

The foregoing are intended as illustrative and should not be interpreted to be all-inclusive examples of conflicts of interest AOAC volunteers may find themselves involved in.

Do's and Don't's

Do avoid the appearance as well as the fact of a conflict of interest.

Do make written disclosure of any material interest which may constitute a conflict of interest or the appearance of a conflict of interest.

Do not accept payment or gifts for services rendered as a volunteer of the Association without disclosing such payment or gifts.

Do not vote on any issue before an AOAC decision-making body where you have the appearance of or an actual conflict of interest regarding the recommendation or decision before that body.

Do not participate in an AOAC decision-making body without written disclosure of actual or potential conflicts of interest in the issues before that body.

Do not accept a position of responsibility as an AOAC volunteer, without disclosure, where the discharge of the accepted responsibility will be or may appear to be influenced by proprietary or other conflicting interests.

Procedures

Each volunteer elected or appointed to an AOAC position of responsibility shall be sent, at the time of election or appointment, a copy of this policy and shall be advised of the requirement to adhere to the provisions herein as a condition for active participation in the business of the Association. Each volunteer, at the time of his or her election or appointment, shall indicate, in writing, on a form provided for this purpose by AOAC, that he or she has read and accepts this policy. Each year, at the spring meeting of the AOAC Board of Directors, the Executive Director shall submit a report certifying the requirements of this policy have been met; including the names and positions of any elected or appointed volunteers who have not at that time indicated in writing that they have accepted the policy. Anyone with knowledge of specific instances in which the provisions of this policy have not been complied with shall report these instances to the Board of Directors, via the Office of the Executive Director, as soon as discovered.

* * * * * *

Adopted: March 2, 1989 Revised: March 28, 1990 Revised: October 1996 Reviewed by outside counsel March 2000 (Fran Dwornik) and found to be current and relevant

Appendix U

ANTITRUST POLICY STATEMENT AND GUIDELINES

Introduction

It is the policy of AOAC INTERNATIONAL (AOAC) and its members to comply strictly with all laws applicable to AOAC activities. Because AOAC activities frequently involve cooperative undertakings and meetings where competitors may be present, it is important to emphasize the on-going commitment of our members and the Association to full compliance with national and other antitrust laws. This statement is a reminder of that commitment and should be used as a general guide for AOAC and related individual activities and meetings.

Responsibility for Antitrust Compliance

The Association's structure is fashioned and its programs are carried out in conformance with antitrust standards. However, an equal responsibility for antitrust compliance -- which includes avoidance of even an appearance of improper activity -- belongs to the individual. Even the appearance of improper activity must be avoided because the courts have taken the position that actual proof of misconduct is not required under the law. All that is required is whether misconduct can be inferred from the individual's activities. Employers and AOAC depend on individual good judgment to avoid all discussions and activities which may involve improper subject matter and improper procedures. AOAC staff members work conscientiously to avoid subject matter or discussion which may have unintended implications, and counsel for the Association can provide guidance with regard to these matters. It is important for the individual to realize, however, that the competitive significance of a particular conduct or communication probably is evident only to the individual who is directly involved in such matters. In general, the U.S. antitrust laws seek to preserve a free, competitive economy and trade in the United States and in commerce with foreign countries. Laws in other countries have similar objectives. Competitors (including individuals) may not restrain competition among themselves with reference to the price, quality, or distribution of their products, and they may not act in concert to restrict the competitive capabilities or opportunities of competitors, suppliers, or customers. Although the Justice Department and Federal Trade Commission generally enforce the U.S. antitrust laws, private parties can bring their own lawsuits. Penalties for violating the U.S. and other antitrust laws are severe: corporations are subject to heavy fines and injunctive decrees, and may have to pay substantial damage judgments to injured competitors, suppliers, or customers. Individuals are subject to criminal prosecution, and will be punished by fines and imprisonment. Under current U.S. federal sentencing guidelines, individuals found guilty of bid rigging, price fixing, or market allocation must be sent to jail for at least 4 to 10 months and must pay substantial minimum fines. Antitrust Guidelines

Since the individual has an important responsibility in ensuring antitrust compliance in AOAC activities, everyone should read and heed the following guidelines.

1. Don't make any effort to bring about or prevent the standardization of any method or product for the purpose or intent of preventing the manufacture or sale of any method or product not conforming to a specified standard 2. Don't discuss with competitors your own or the competitors' prices, or anything that might

affect prices such as costs, discounts, terms of sale, distribution, volume of production, profit margins, territories, or customers.

3. Don't make announcements or statements at AOAC functions, outside leased exhibit space, about your own prices or those of competitors.

4. Don't disclose to others at meetings or otherwise any competitively sensitive information.

5. Don't attempt to use the Association to restrict the economic activities of any firm or any individual.

6. Don't stay at a meeting where any such price or anti-competitive talk occurs.

7. Do conduct all AOAC business meetings in accordance with AOAC rules. These rules require that an AOAC staff member be present or available, the meeting be conducted by a knowledgeable chair, the agenda be followed, and minutes be kept.

8. Do confer with counsel before raising any topic or making any statement with competitive ramifications.

9. Do send copies of meeting minutes and all AOAC-related correspondence to the staff member involved in the activity.

10. Do alert the AOAC staff to any inaccuracies in proposed or existing methods and statements issued, or to be issued, by AOAC and to any conduct not in conformance with these guidelines.

Conclusion

Compliance with these guidelines involves not only avoidance of antitrust violations, but avoidance of any behavior which might be so construed. Bear in mind, however, that the above antitrust laws are stated in general terms, and that this statement is not a summary of applicable laws. It is intended only to highlight and emphasize the principal antitrust standards which are relevant to AOAC programs. You must, therefore, seek the guidance of either AOAC counsel or your own counsel if antitrust questions arise.

Adopted by the AOAC Board of Directors: September 24, 1989 Revised: March 11, 1991 Revised October 1996

Appendix V

POLICY ON THE USE OF THE ASSOCIATION NAME, INITIALS, IDENTIFYING INSIGNIA, LETTERHEAD, AND BUSINESS CARDS

Introduction

The following policy and guidelines for the use of the name, initials, and other identifying insignia of AOAC INTERNATIONAL have been developed in order to protect the reputation, image, legal integrity and property of the Association. The name of the Association, as stated in its bylaws, is "AOAC INTERNATIONAL". The Association is also known by its initials, AOAC, and by its logo, illustrated below, which incorporates the Association name and a representation of a microscope, book, and flask. The AOAC logo is owned by the Association and is registered with the U.S. Patent and Trademark Office.

6JG HWNN #UUQEKCVKQP KPUKIPKC KNNWUVTCVGF DGNQY KU EQORTKUGF QH VJG NQIQ CPF VJG VCINKPG 6JG 5EKGPVKHKE #UUQEKCVKQP &GFKECVGF VQ #PCN[VKECN 'ZEGNNGPEG UJQYP DGNQY 6JG V[RGHCEG WUGF KU .CTIQ 6JG #1#% VCINKPG KU QYPGF D[ VJG #UUQEKCVKQP CPF KU TGIKUVGTGF YKVJ VJG 7 5 2CVGPV CPF 6TCFGOCTM QHHKEG

Policy

Policy on the use of the Association's name and logo is established by the AOAC Board of Directors as follows:

“The Board approves and encourages reference to the Association by name, either as AOAC INTERNATIONAL or as AOAC; or reference to our registered trademark, AOAC®, in appropriate settings to describe our programs, products, etc., in scientific literature and other instances so long as the reference is fair, accurate, complete and truthful and does not indicate or imply unauthorized endorsement of any kind. The insignia (logo) of AOAC INTERNATIONAL is a registered trade and service mark and shall not be reproduced or used by any person or organization other than the Association, its elected and appointed officers, sections, or committees, without the prior written permission of the Association. Those authorized to use the AOAC INTERNATIONAL insignia shall use it only for

the purposes for which permission has been specifically granted.

The name and insignia of the Association shall not be used by any person or organization in any way which indicates, tends to indicate, or implies AOAC official endorsement of any product, service, program, company, organization, event or person, endorsement of which, has not been authorized by the Association, or which suggests that membership in the Association is available to any organization.”

The Executive Director, in accordance with the above stated policy, is authorized to process, approve, fix rules, and make available materials containing the Association name and insignia.

It should be noted that neither the Association's name nor its insignia nor part of its insignia may be incorporated into any personal, company, organization, or any other stationery other than that of the Association; nor may any statement be included in the printed portion of such stationery which states or implies that an individual, company, or other organization is a member of the Association.

Instructions

1. Reproduction or use of the Association name or insignia requires prior approval by the Executive Director or his designate.

2. Association insignia should not be altered in any manner without approval of the Executive Director or his designate, except to be enlarged or reduced in their entirety.

3. Artwork for reproducing the Association name or insignia, including those incorporating approved alterations, will be provided on request to those authorized to use them (make such requests to the AOAC Marketing Department). Examples of the types of alterations that would be approved are inclusion of a section name in or the addition of an officer's name and address to the letterhead insignia.

4. When the Association name is used without other text as a heading, it should, when possible, be set in the Largo typeface.

5. Although other colors may be used, AOAC blue, PMS 287, is the preferred color when printing the AOAC insignia, especially in formal and official documents. It is, of course, often necessary and acceptable to reproduce the insignia in black.

6. Do not print one part of the logo or insignia in one color and other parts in another color.

7. The letterhead of AOAC INTERNATIONAL shall not be used by any person or organization other than the Association, elected and appointed officers, staff, sections, or committees; except by special permission.

Correspondence of AOAC official business should be conducted using AOAC letterhead. However, those authorized to use AOAC letterhead shall use it for official AOAC business only.

Copies of all correspondence using AOAC letterhead or conducting AOAC official business,

whether on AOAC letterhead or not, must be sent to the appropriate office at AOAC headquarters.

8. AOAC INTERNATIONAL business cards shall not be used by any person or organization other than the Association, its staff, and elected officials, except by special permission.

Those authorized to use AOAC business cards shall use them for official AOAC business only and shall not represent themselves as having authority to bind the Association beyond that authorized.

Sanctions

1. Upon learning of any violation of the above policy, the Executive Director or a designate will notify the individual or organization that they are in violation of AOAC policy and will ask them to refrain from further misuse of the AOAC name or insignia.

2. If the misuse is by an Individual Member or Sustaining Member of the Association, and the misuse continues after notification, the Board of Directors will take appropriate action.

3. If continued misuse is by a nonmember of the Association or if a member continues misuse in spite of notification and Board action, ultimately, the Association will take legal action to protect its property, legal integrity, reputation, and image.

* * * * * *

Adopted by the AOAC Board of Directors: September 24, 1989 Revised: June 13, 1991; February 26, 1992; March 21, 1995; October 1996