AOAC SPDS ERP DECEMBER 15, 2017

AOAC SPSFAM ERP DECEMBER 15, 2017

BOOK OF REVIEWS FOR:

CANNABIS QUANTITATION

AOAC INTERNATIONAL HQ 2275 Research Blvd., Suite 300 Rockville, Maryland, 20850 USA

contact: spsfam@aoac.org

Expert Review Panel for SPSFAM Cannabis in Plant Material & Extracts Methods Friday, December 15, 2017 8:30 am – 11:00 am ET AOAC INTERNATIONAL Headquarters 2275 Research Boulevard, Suite 300 Rockville, Maryland 20871, USA Main Conference Room A G E N D A 1. Welcome and Introductions Melissa Phillips, US National Institute of Standards and Technology (ERP Chair) and Dawn Frazier, AOAC INTERNATIONAL 2. Review of AOAC Volunteer Policies & ERP Process Overview and Guidelines Deborah McKenzie, AOAC INTERNATIONAL 3. Brief Overview of AOAC SPSFAM Cannabis SMPR Development Dawn Frazier & Scott Coates, AOAC INTERNATIONAL 4. Review of Methods For each method, the assigned ERP membres will present a review of the methods and manuscripts, after which the ERP will discuss the method and reach consensus on the status for each method. A. Method CAN-001* a. CAN-001 Reviews led by Hudalla and Schaffer and followed by ERP Discussion and Consensus B. Method CAN-002* a. CAN-002 Reviews led by Johnson and Sweeney and followed by ERP Discussion and Consensus 5. Final Action Requirements for Adopted Methods (if applicable) 6. Adjourn

ITINERARY Breakfast served: 8:30am 9:45am – 10:00am - Morning Break 11:00am– Lunch Served

SPSFAM Cannabis ERP 12/15/2017 – v1.0 Draft meeting agenda subject to change without notice *Items requiring a vote by ERP

Cannabis Expert Review Panel, December 15, 2017 1. Melissa Phillips, NIST (Chair) 2. Yohei Arao, Shimadzu 3. Ramkumar Dhandapani, Phenomenex 4. Nour Eddine ES-SAFI, Mohammed V University, Rabat

5. Heather Harris, Hills Beaver Creek Schools 6. Christopher Hudalla, Waters Corporation 7. Peter Indick, Microbac Laboratories 8. Holly Johnson, Alkemist

9. Janan Marcu, Greed Standard Diagnostics 10. Katerina Mastovska, Covance Laborator 11. Elizabeth Mudge, BCIT 12. Curtis Phinney, Curtis S. Phinney, CNS 13. Paul Reibach, Smithers Vincent 14. Kate Rimmer, NIST 15. Markus Roggen, OutCo 16. Michael Schaffer, Psychemedics 17. Christian Sweeney, Cannibistry Labs 18. Tomasz Tuzimski, Medical University of Lublin 19. Rodger Voelker, OG Analytical 20. Sudhakar Yadlapalli, First Source Laboratory Solutions

TWO THIRDS OF THIS EXPERT REVIEW PANEL MUST BE PRESENT (14)

AOAC SPSFAM CANNABIS EXPERT REVIEW PANEL

METHODS AND SMPR ACCESS

• AOAC SMPR 2017.001 (Concentrates) • AOAC SMPR 2017.002 (Plant Material) • METHOD ACCESS (ERP ONLY – PASSWORD REQUIRED)

Candidate Method CAN-01

Submitter Name Submitter Email

Katerina Mastovska

katerina.mastovska@covance.com

Organization Covance Method Type Cannabis Method Name

Quantitation of Cannabinoids in Cannabis Dried Plant Materials and Concentrates Using Liquid Chromatography–Diode Array Detection Technique with Optional Mass Spectrometric Detection Lukas Vaclavik, Frantisek Benes, Ales Krmela, Veronika Svobodova, Jana Hajslova and Katerina Mastovska Applicable to quantification and identification of cannabidiol, cannabidiolic acid, cannabinol, ∆9-tetrahydrocannabinol, tetrahydrocannabinolic acid, cannabichromene, cannabidivarinic acid, cannabigerol, cannabigerolic acid, cannabidivarin, Δ8- tetrahydrocannabinol, and tetrahydrocannabivarin.

Method Author(s)

Method Applicability

Primary Reviewer: Peter Indick, Microbac Secondary Reviewer(s):

Mike Schaffer, Psychemedics and Christopher Hudalla, Waters

AOAC DECEMBER 2017 ERPs - METHOD REVIEW FORM

Submission Date

2017-12-05 17:26:21

AOAC Expert Review Panel: Method Review

Reviewer Information Required to validate your review.

Name

Pete Indick

E-mail

peter.indick@microbac.com

Organization

Microbac Laboratories

Remember, the main purpose of your review is to ensure the method conforms to the applicable SMPR. View and download SMPRs here:

SPDS SMPRs SPSFAM SMPRs

Method Review

Title of Method

Quantitation of Cannabinoids in Dried Plant Materials

AOAC Candidate Method Number (e.g. ALN-01)

CAN-001

Applicable SMPR

2017.001/.002

I. Summary of the Method

I. Summary of the Method

Method for the identification and quantitation of twelve cannabinoids in dried plant material and cannabis concentrated. Method utilizes HPLC-Diode array detector and provides an option for mass spectrometer detector.

II. Review of the Method Only:

II. Review of the Method Only 1. Does the applicability of the method support the applicability of the SMPR? If not, please explain what is missing.

The method does support the applicability of the SMPR with the exception that two compounds were not included due to the lack of a reference standard (cannabichromeric acid and tetra hydrocannabivarin acid)

2. Does the analytical technique(s) used in the method meet the SMPR? If not, please specify how it differs from what is stated in the SMPR. 3. Are the definitions specified in the SMPR used and applied appropriately in the method? If no, please indicate how the terms are used. 4. Does the method, as written, contain all appropriate precautions and warnings related to the method's reagents, components, instrumentation, or method steps that may be hazardous? If no, please suggest wording or option(s). III. Review of Supporting Information 1. Are the definitions specified in the SMPR used and applied appropriately in the supporting documentation (manuscripts, method studies, etc...)? If not, please explain the differences and if the method is impacted by the difference. 2. Is there information demonstrating that the method meets the SMPR Method Performance Requirements using the Reference Materials stated in the SMPR? If not, then specify what is missing and how this impacts demonstration of performance of the method.

The analytical technique meets the SMPR.

The method does not include the definition of "Quantitative Method" The method utilizes "Precision" in place of "Repeatability" The method does not include the definition of "Reproducibility" The method defines "Recovery" as "Trueness". Neither definition describes the SMPR definition of "Recovery"

General reference to AOAC Appendix B:Lab Safety is included. No additional precautions are specified.

III. Review of Information in Support of the Method

The methods determination of "Recovery (Trueness)" is not consistent with the SMPR definition. The method does not provide for a spiked sample due to the lack of adequate 'clean' sample matrix. The methods review of "Recovery" is closer to a check on extraction efficiency.

Reference materials acquired from Cerilliant. Two compounds were not available (CBCA & THCVA).

3. Is there information demonstrating that the

Performance data is not provided for cannabichromenic acid tetrahydrocannabivarin acid due to a lack of available commercial standards.

method performs within the SMPR Method Performance REquirements table specifications for all analytes in the SMPR applicability statement? If not, please specify what is missing and whether or not the method's applicability should be modified. IV. General Submission Package 1. Based on the supporting information, were there any additional steps in the evaluation of the method that indicated the need for any additional precautionary statements in the method? 2. Does the method contain system suitability tests or controls as specified by the SMPR? If not, please indicate if there is a need for such tests or controls and which ones. 3. Is there information demonstrating that the method system suitability tests and controls as specified in the SMPR worked appropriately and as expected? If no, please specify. 4. Based on the supporting information, is the method written clearly and concisely? If no, please specify the needed revisions.

IV. General Submission Package

Should there be language included regarding the sampling/storage/disposal of cannabis materials? Both from a safety and legal perspective.

The method does not include a matrix spike procedure to evaluate the extraction/analytical efficiency. Some type matrix spike should be included either in 'clean' equivalent matrix or sample matrix spike.

Overall the method seems to demonstrate that it worked appropriately with the exception of the two compounds not included. While the supporting data give the general impression that the method is appropriate there are several sections that are incomplete or were only performed on a single varietal.

In sections F.b.7 and F.c.4 it is not clear that the extract should be filtered prior to the dilution preparation.

Section F.a discusses sample preparation but greater detail could be provided to insure the greatest possible homogenization and reduction of cross- contamination between samples. How much sample should be processed to be representative? How should the grinder be cleaned between samples?

5. Based on the supporting information, what are the pros/strengths of the method?

The method did a good job with an investigation of the appropriate analytical conditions to meet the SMPR requirements.

The analytes can be determined from a single preparation and analysis.

The method provides options for the utilization of different detectors (QMS, MS/MS) to improve positive identification.

6. Based on the supporting information, what are the cons/weaknesses of the method?

No demonstration of the extraction efficiency of a single solvent preparation. They did show that the prep appears to remove all the analytes of interest with one or two extractions but there is not indication that the extraction removed all of the compound from the sample matrix. Could some percentage be bound and require a different solvent/procedure to more thoroughly extract the compounds of interest? There is also no discussion of possible extract cleanup (aside from filtration) prior to analysis. Given the potential for non-target compounds to be extracted in this procedure some thought should be given to a potential cleanup proceure. Overall it seems like it just meets the requirements of the SMPR. I would like to see additional data supporting the extraction efficiency, inclusion of matrix spiked samples, and completion of the repeatability data (Table 9).

7. Any general comments about the method?

As the method did not include data for CBCA and THCVA these compounds should not reportable under this method.

Recommendation for the Method

V. Final Recommendation Do you recommend this method be adopted as a First Action and published in the Official Methods of Analysis of AOAC INTERNATIONAL? Please specify rationale.

I do not recommend that this method be adopted at this time. I would like to see the inclusion of some type of sample spiking to demonstrate the analytical & preparation efficiency. The supporting data appears to provide some support that the procedure will work but is incomplete for some species varieties.

Submitter Name Elizabeth Mudge Submitter Email emudge@bcit.ca Organization BCIT Method Type Cannabis Method Name

Leaner and Greener Analysis of Cannabinoids

Method Author(s)

E Mudge, SJ Murch, PN Brown

Method Applicability

This method is applicable to the quantitation of 10 cannabinoids in cannabis dried plant material: THC, THCA, CBD, CBDA, CBN, CBG, THCV, CBC, CBGA, CBDVA.

Primary Reviewer: Holly Johnson, Alkemist Labs Secondary Reviewer: Christian Sweeney, Cannibistry

Candidate Method #

CAN-002

AOAC DECEMBER 2017 ERPs - METHOD REVIEW FORM

Submission Date

2017-12-14 11:51:17

AOAC Expert Review Panel: Method Review

Reviewer Information Required to validate your review.

Name

Holly Johnson

E-mail

holly@alkemist.com

Organization

Alkemist Labs

Remember, the main purpose of your review is to ensure the method conforms to the applicable SMPR. View and download SMPRs here:

SPDS SMPRs SPSFAM SMPRs

Method Review

Title of Method

Leaner and Greener Analysis of Cannabinoids

AOAC Candidate Method Number (e.g. ALN-01)

CAN-002

Applicable SMPR

2017.002

I. Summary of the Method

I. Summary of the Method

The method submitted is an SLV for an optimized HPLC analysis of dried Cannabis flower using reversed phase separation with UV detection.

II. Review of the Method Only:

II. Review of the Method Only 1. Does the applicability of the method support the applicability of the SMPR? If not, please explain what is missing.

Yes.

2. Does the analytical technique(s) used in the method meet the SMPR? If not, please specify how it differs from what is stated in the SMPR. 3. Are the definitions specified in the SMPR used and applied appropriately in the method? If no, please indicate how the terms are used. 4. Does the method, as written, contain all appropriate precautions and warnings related to the method's reagents, components, instrumentation, or method steps that may be hazardous? If no, please suggest wording or option(s). III. Review of Supporting Information 1. Are the definitions specified in the SMPR used and applied appropriately in the supporting documentation (manuscripts, method studies, etc...)? If not, please explain the differences and if the method is impacted by the difference. 2. Is there information demonstrating that the method meets the SMPR Method Performance Requirements using the Reference Materials stated in the SMPR? If not, then specify what is missing and how this impacts demonstration of performance of the method.

Yes.

Yes.

Yes.

III. Review of Information in Support of the Method

Yes.

Yes.

3. Is there information demonstrating that the

LOQ and Range criteria from table 3 in the SMPR are met. Applicable criteria for Recovery & Repeatability are met for required analytes. Reproducibility data are not presented as this was an SLV.

method performs within the SMPR Method Performance REquirements table specifications for all analytes in the SMPR applicability statement? If not, please specify what is missing and whether or not the method's applicability should be modified. IV. General Submission Package 1. Based on the supporting information, were there any additional steps in the evaluation of the method that indicated the need for any additional precautionary statements in the method? 2. Does the method contain system suitability tests or controls as specified by the SMPR? If not, please indicate if there is a need for such tests or controls and which ones. 3. Is there information demonstrating that the method system suitability tests and controls as specified in the SMPR worked appropriately and as expected? If no, please specify. 4. Based on the supporting information, is the method written clearly and concisely? If no, please specify the needed revisions. 5. Based on the supporting information, what are the pros/strengths of the method?

IV. General Submission Package

No.

Yes.

Yes.

Yes.

This method was optimized with a thorough extraction study and separation conditions. The method has been fully validated (SLV) and data presented confirm that the method produces repeatable accurate results. The time to result is quick, and the method is overall robust.

6. Based on the supporting information, what are the cons/weaknesses of the method?

A matrix extension study to look at more oils, extracts, or concentrates would be good to broaden the applicability.

7. Any general comments about the method?

A very well done and thoroughly validated method for straightforward analysis of Cannabis flower.

Recommendation for the Method

V. Final Recommendation Do you recommend this method be adopted as a First Action and published in the Official Methods of Analysis of AOAC INTERNATIONAL? Please specify rationale.

Yes. Recommended for approval as a first action method. The submission has ample supportive data and meets all criteria in the SMPR 2017.002.

AOAC DECEMBER 2017 ERPs - METHOD REVIEW FORM

Submission Date

2017-12-07 12:29:21

AOAC Expert Review Panel: Method Review

Reviewer Information Required to validate your review.

Name

Christian Sweeney

E-mail

csweeney@cannabistry.com

Organization

Cannabistry Labs

Remember, the main purpose of your review is to ensure the method conforms to the applicable SMPR. View and download SMPRs here:

SPDS SMPRs SPSFAM SMPRs

Method Review

Title of Method

Leaner and Greener Analysis of Cannabinoids

AOAC Candidate Method Number (e.g. ALN-01)

CAN-002

Applicable SMPR

AOAC SMPR 2017.002

I. Summary of the Method

I. Summary of the Method

The method is a straightforward chromatographic analysis for the identification and quantification of a number of cannabinoids in dried plant material. Sample prep involves dry grinding a large sample (5g) of dried plant material and pulling a sub-sample (~200mg) to be extracted in a single solvent (80% methanol), followed by filtration, dilution, and injection. The removal of chloroform from sample preparation is a key point of differentiation for the method. The chromatography is a gradient method carried out on a standard C18 column with detection via DAD at a single wavelength (220nm). Quantification is based on external standards, no internal standards were used. A number of method performance indicators are provided including linearity of external standards and selectivity. Most critically to the SMPR; LOQs, repeatability, and recovery are also provided for most of the required cannabinoids. Additionally, the results of a stability test and a partial factorial optimization were reported to provide justification for many of the method steps. The method also contains information about analysis of cannabis oil which is not being considered in this review.

II. Review of the Method Only:

II. Review of the Method Only 1. Does the applicability of the method support the applicability of the SMPR? If not, please explain what is missing. 2. Does the analytical technique(s) used in the method meet the SMPR? If not, please specify how it differs from what is stated in the SMPR. 3. Are the definitions specified in the SMPR used and applied appropriately in the method? If no, please indicate how the terms are used. 4. Does the method, as written, contain all appropriate precautions and warnings related to the method's reagents, components, instrumentation, or method steps that may be hazardous? If no, please suggest wording or option(s).

This method is capable of identifying, via retention time of external standards, and quantifying, via external calibration curve, cannabinoids in dried plant material.

Yes

Reproducibility as defined in the SMPR is not carried out, but Intermediate Precision data is presented. LOQ is calculated based on an EPA MDL procedure which appears to be quite similar to the AOAC recommended method.

The use of liquid nitrogen is proposed for sample prep in the supporting data section of the submission with no methodology or accompanying safety precautions. If this is the recommended process it deserves a clear description and incorporation in the body of the method.

III. Review of Information in Support of the Method

III. Review of Supporting Information 1. Are the definitions specified in the SMPR used and applied appropriately in the supporting documentation (manuscripts, method studies, etc...)? If not, please explain the differences and if the method is impacted by the difference. 2. Is there information demonstrating that the method meets the SMPR Method Performance Requirements using the Reference Materials stated in the SMPR? If not, then specify what is missing and how this impacts demonstration of performance of the method.

Reproducibility as defined in the SMPR is not carried out, but Intermediate Precision data is presented. LOQ is calculated based on an EPA MDL procedure which appears to be quite similar to the AOAC method.

All reference materials used conform to Tables 1 and 2 in the SMPR, with CBL being an additional cannabinoid from outside of the list.

3. Is there information demonstrating that the

The method provides limit of quantitation (LOQ) data for the 5 required cannabinoids, with 4 of the 5 required cannabinoids achieving the SMPR specification for LOQ. CBD was determined to have an LOQ of 0.17% whereas the LOQ SMPR specification was = 0.1%. LOQs were provided for 3 of the “additional, desirable cannabinoids”; THCV, CBD, and CBC. All three of these LOQs were below the LOQ SMPR specification. The method provides recovery data for 4 of the 5 required cannabinoids, CBN was not included. Of the 4 cannabinoids tested, 2 were tested in the midrange (>1-25%) with CBDA meeting the recovery requirements and THCA falling just outside of the 97-103% range at 96.1% recovery. All 4 cannabinoids were tested in the low range (0.1- 1%) and at the lowest level 0.1% all 4 met the recovery SMPR*. The lower ranges tested are understandable due to challenges associated with procuring high concentration CRMs, but it is important to point out that the recovery of CBD, THCA, and THC technically do not meet the recovery SMPR at the highest concentrations tested (1%, 3.5%, and 1% respectively). The supporting info provides an extraction study suggesting that a single methanol extraction is exhaustive. This helps bolster the recovery data already provided in the method. Repeatability data was provided for all 5 of the required cannabinoids as well as 3 additional cannabinoids (THCV, CBG, and CBC). The Repeatability data, reported as %RSD, was generally within the SMPR specification* with the notable exception of THCA & CBDA in samples CAN008 & CAN009. It is unclear if this inhomogeneity is an artifact of an inhomogeneous sample or the method. In the supporting info, additional repeatability data is provided for samples with a higher level of cannabinoids. RSDs below the SMPR specification are achieved which adds more confidence to the overall method and may further suggest the prior repeatability outliers are due to sample inhomogeneity and not the method. Additional repeatability data is provided for CBDVA and CBGA and is on par with repeatability data previously shown for the minor cannabinoids. Reproducibility data is not provided as defined by the SMPR. Instead, intermediate precision was determined on 3 consecutive days. It is not specified, but the assumption is that this was performed by the same analyst on the same instrument. Intermediate precision showed similar % RSD when compared to Repeatability in most cases, but is likely not a sufficient replacement for Reproducibility data as defined in the SMPR. Given the nature of the analyte, “among-laboratory” data will be challenging prior to first action and intermediate precision may be considered acceptable especially if it represents multiple analysts and multiple instruments. *n.b. AOAC SMPR 2017.002 Tables 3&4 refer only to Table 2, the reviewer assumes these refer to both Tables 1&2.

method performs within the SMPR Method Performance REquirements table specifications for all analytes in the SMPR applicability statement? If not, please specify what is missing and whether or not the method's applicability should be modified.

IV. General Submission Package

IV. General Submission Package 1. Based on the supporting information, were there any additional steps in the evaluation of the method that indicated the need for any additional precautionary statements in the method?

No

2. Does the method contain system suitability tests or controls as specified by the SMPR? If not, please indicate if there is a need for such tests or controls and which ones. 3. Is there information demonstrating that the method system suitability tests and controls as specified in the SMPR worked appropriately and as expected? If no, please specify. 4. Based on the supporting information, is the method written clearly and concisely? If no, please specify the needed revisions.

The system suitability test is provided in the form of instrument precision on standards at the midrange point of the analytical range and achieved RSDs ranging from 2.8-3.8%.

The system suitability test suggests the instrumental repeatability is fairly close to the SMPR repeatability requirements, representing a large challenge.

The method as submitted does have some clarity challenges.

No column temperature is provided in the description of the separation, though AOAC Appendix K does allow for adjustments to column temperature during column lifetime. The extracted sample dilution is left to the reader to determine. It is challenging to predict the correct dilution as the analytes have a wide range of concentrations, but multiple dilutions and injections will result in decreased sample throughput.

Additionally the use of liquid nitrogen is mentioned in the supporting info but not truly incorporated into the method.

As this is a submission for the dried plant material method, it may be beneficial to remove the sections about analysis of oil for clarity. The ability of this method to analyze oil was not reviewed per the method applicability statement supplied by the author.

Lastly, it would be ideal to integrate the supplemental data into the body of the submission to make the entire method more clear.

5. Based on the supporting information, what are the pros/strengths of the method?

The use of methanol as an extraction solvent is a major improvement over 9:1 Methanol:Chloroform for laboratories and represents a major positive of this method. The method overall shows a reasonable adherence to the SMPRs tested for especially when the “non-analytical” challenges of the analyte are considered (i.e. difficulty gathering among-lab data for reproducibility, CRM concentration limits making recovery challenging).

The method is simple and could be adopted by most analysts and labs simply without major training or capital expenditure required.

6. Based on the supporting information, what are the cons/weaknesses of the method?

The method does not meet all of the SMPRs as discussed above.

The validation guidance around homogeneity is clear that grinding, while a reasonable option for homogenizing samples, may result in degradation and clumping of resin resulting in less accurate results and highly variable samples. This method employs grinding of a 5g sample, with 200mg used for extraction, but does not address how to ensure the 200mg sample is representative. Some alternative methods successfully achieve homogeneity by “wet-grinding” a sample in the extraction solvent. The use of liquid nitrogen for grinding could necessitate more rigorous safety requirements. Additionally, incorporating a major material change to the sample preparation in the supplemental info could be quite confusing to potential adopters. 5g is a large sample requirement for a crop of this value. One of the key challenges for the adoption of a method will be sample size. While it may be that 5g does produce a far more accurate and precise sample for analysis, it would be valuable to prove that this sample size is an improvement beyond the current status quo of 0.5-2g samples. The separation time is quite long and could represent a barrier to adoption. This is exacerbated by the use of a gradient method and a 7 minute equilibration. Shorter isocratic methods have been employed successfully for the separation of cannabinoids with similar columns and mobile phases.

The addition of vortexing every 5 minutes during the sonication adds a fair bit of labor without showing a statistically significant improvement.

7. Any general comments about the method?

This method comes close to achieving most of the SMPR specifications. The submission represents a published article with some additional supporting information. This format has made reading and review challenging.

Recommendation for the Method

V. Final Recommendation Do you recommend this method be adopted as a First Action and published in the Official Methods of Analysis of AOAC INTERNATIONAL? Please specify rationale.

The method does not meet SMPR specifications for all analytes. The method is quite close to achieving SMPR specifications in many cases, though no among- laboratory reproducibility data is provided. Additionally, the submission as presented is challenging to read due to being a previously published paper with nonessential material (i.e. oil method) remaining as well as supplemental info added, but not incorporated. As previously discussed the clarity challenges in my opinion would represent a barrier to adoption. I firmly believe with some minor optimization this method could achieve the SMPR specifications and better serve the stakeholder community. Additionally, with some minor editing the method could be clearer, more concise, and ideal for First Action. I hope there is an opportunity for this optimization and editing to take place as the method has many strengths that would serve the stakeholder community quite well.

AOAC INTERNATIONAL

First Action Official Methods of Analysis SM Guidance Documents

I.

Process Flowchart

II. Process Guidelines III. Expert Review Panel – Policies & Procedures

Alternate Pathway to Official First Action Method Status

Funded Stakeholder Panel

x x x

Managed by AOAC HQ Properly vetted by OMB

Carefully documented and transparent

Working Groups

Standard Method

x Managed by AOAC HQ x Properly vetted by OMB x Carefully documented and transparent

Performance Requirements

Expert Review Panels

Call for Methods & Literature Search

x Managed by AOAC HQ x Properly vetted by OMB x Carefully documented and transparent

Official First Action Method

JAOAC OMA Web ILM

x ERPs continue to monitor for two years, until method is either advanced or removed from system (period is extendable for active data collection) x ERP recommends Final Action to OMB x OMB grants Final Action status

Updated 2011Ͳ5Ͳ11

AOAC INTERNATIONAL (updated 2011-0ϱ-ϭϭ by APOFAMS Task Force) ALTERNATIVE PATHWAY to OFFICIAL FIRST ACTION METHOD STATUS REQUIREMENTS Expert Review Panels -Must be supported by relevant stakeholders. -Constituted solely for the ERP purpose, not for Standard Method Performance Requirements (SMPR) purposes or as an extension of an SMPR. -Consist of a minimum of seven members representing balance of key stakeholders. -ERP constituency must be approved by the Official Methods Board (OMB). -Holds transparent public meetings only. -Remains in force as long as method in First Action Status. Official First Action Method Status decision -Must be made by an ERP constituted or reinstated post 2011-03-28 for Official First Action Status Method Approval (OFASMA). -Must be made by an ERP vetted for OFASMA purposes by OMB post 2011-03-28. -Method adopted by ERP must ƉĞƌĨŽƌŵ ĂĚĞƋƵĂƚĞůLJ ĂŐĂŝŶƐƚ the SMPR set forth by the stakeholders. -Method must be adopted by unanimous decision of ERP on first ballot, If not unanimous, negative votes must delineate scientific reasons. -Negative voter(s) can be overridden by 2/3 of non-negative voting ERP members after due consideration - Method becomes Official First Action on date when ERP decision is made. -Methods to be drafted into AOAC format by a knowledgeable AOAC staff member or designee in collaboration with the ERP and method author. -Report of OFAMS decision complete with ERP report regarding decision including scientific background (references etc) to be published concurrently with method in traditional AOAC publication venues. Method in First Action Status and Transitioning to Final Action Status -Further data indicative of adequate method reproducibility (between laboratory) performance to be collected. Data may be collected via a collaborative study or by proficiency or other testing data of similar magnitude. -Two years maximum transition time (additional year(s) if ERP determines a relevant collaborative study or proficiency or other data collection is in progress). -Method removed from Official First Action and OMA if no evidence of method use available at the end of the transition time. -Method removed from Official First Action and OMA if no data indicative of adequate method reproducibility is forthcoming as outlined above at the end of the transition time. -ERP to recommend Method to Official Final Action Status to the OMB. -OMB decision on First to Final Action Status

EXPERT REVIEW PANELS --Policies and Procedures—

Introduction Expert Review Panels (ERP) are created to provide stakeholders with an expert resource to evaluate analytical solutions to identified needs and concerns. The ERP will be tasked to search for appropriate methods, issue a “Call for Methods” in the ILM and other avenues, and critically evaluate all collected methods. The ERP will then recommend appropriate methods (as submitted or modified) for adoption as Official First Action methods or for further validation. The ERP, if requested by the Committee/Topic Advisor, would be expected to assist in identifying appropriate materials to be used in the validation studies and in reviewing the protocols for such studies. Outline of ERP establishment process An Expert Review Panel is established as follows: A stakeholder or stakeholder body submits a request for the creation of an ERP to the AOAC staff. The request includes a description of the subject area, the desired outcome, and should include a list of recommended subject experts with supporting documentation (see "Qualifications of Expert Reviewers"). Included with this list of recommended subject experts could be a recommendation for an ERP Chair. The request is forwarded to the appropriate AOAC Chief Science Officer (CSO) who identifies potential members for the ERP from a recognized Pool of Experts, a Call for Experts on the AOAC website, and from the stakeholder recommendations. The candidate list and supporting documentation are forwarded to the Chair of the OMB who will assign the review to at least two OMB members. The OMB reviewers will review the candidates for expertise and perceived conflicts of interest and the OMB may then approve the members of the ERP. A Chair for the ERP is also selected. The Chair of the ERP will organize meetings of the ERP to discuss and make recommendations relative to method recommendations, the method(s) to be further validated, and the materials to be used in the validation studies. The conclusions and recommendations of the ERP will be transmitted by the ERP Chair to the OMB and stakeholder body. The stakeholder body will proceed with implementation of the ERP's recommendations by organizing the appropriate SLV study and other items needed for application. Pool of Potential Expert Reviewers : Candidates for ERPs are pulled from the following sources. Upon acceptance of the request for the formation of an ERP, a Call for Experts is posted on the AOAC website for a minimum of two weeks. Candidates can then contact AOAC with their interest and credentials. Also, AOAC maintains a Pool of Experts database containing a list of

Approved by Official Methods Board, November 13, 2008 Approved by AOAC Board of Directors, December 9, 2008 Appeals Process Appended – September 2009 Revisedby AOAC Board of Directors, May 25, 2011

Page 1 of 6

AOAC members willing to serve as experts and cataloging their education, experience, and other applicable credentials. Candidates can also be recommended by the stakeholder(s). Note: Candidates (except for the chair) do not need to be members of AOAC. The appointment of experts to an ERP will be for a minimum of 3 years. Qualification of Expert Reviewers: To qualify as an Expert Reviewer, the candidate must meet one of the following requirements: (1) Demonstrated knowledge in the appropriate scientific disciplines. (2) Demonstrated knowledge regarding data relevant to adequate method performance. (3) Demonstrated knowledge of practical application of analytical methods to bona fide diagnostic requirements. These qualifications must be clearly described in a CV submitted to the CSO and kept on file at AOAC headquarters. Duties: Members of the Pool of Experts will be called upon to serve on ERPs as needed, and to review documents prepared in the course of the project. These documents may include: (1) procedural documents on how methods will be selected and how single laboratory validation studies will be done; (2) methods submitted for consideration as Official First Action Methods; (3) methods submitted for selection for further validation studies; (4) protocols to be used for single laboratory validation studies; (5) the selection of methods to be considered for full collaborative studies; and (6) validation study reports. Expert Review Panel: The CSO selects candidates for an ERP from the Pool of Experts database, the Call for Experts on the AOAC website, and from candidates recommended by the stakeholders. Selection of ERP candidates is based upon their knowledge and experience to adequately evaluate the scope of the study and the anticipated number of submitted methods. The size of the ERP will be sufficient to assure the necessary expertise is present. The CSO may recommend one of the Panel members to serve as Chair. The CSO submits the following to the OMB Chair: The original submission package, a list of all candidates considered for inclusion on the ERP, the slate of recommended candidates, and a list of possible alternates. Explanations for the ERP choices may be included by either the CSO or a stakeholder if desired. The OMB Chair will delegate two members of the OMB to perform a review. The reviewers submit their recommendations in writing to the OMB. The OMB then votes on the reviewers’ recommendations. This vote can be either by email or during an OMB meeting. The OMB may choose not to select one or more individuals on the Panel as submitted and may or may not accept the recommendation of the CSO for the panel Chair. A majority of those voting will be required for approval. The vote of the Chair will break any tie. The CSO, ERP members, and stakeholder body are notified of the vote within one week. Conflict of Interest: It is incumbent upon each ERP member to avoid any known or potential conflicts of interest and make these known to the CSO and OMB Chair. Each pool member chosen for an ERP will be asked to agree to the AOAC Policies and Procedures on Conflicts of Interest evidenced by completing a Conflict of Interest Form.

Approved by Official Methods Board, November 13, 2008 Approved by AOAC Board of Directors, December 9, 2008 Appeals Process Appended – September 2009 Revisedby AOAC Board of Directors, May 25, 2011

Page 2 of 6

If a Pool member being considered to serve on any particular panel is an author, or his/her laboratory is the source of a method under consideration by the Panel, they must so indicate to the CSO or OMB Chair. At the discretion of the CSO or OMB, the names of such Pool members may be removed from consideration, or they may be considered to serve on the ERP with the understanding that a deliberate effort will be required to avoid any known or potential conflicts of interest. In these latter cases, assignments of individual methods for peer review will be made in such a way by the Chair that ERP members will not review any method for which they are an author or co-author, or for which their laboratory is the source; and, most importantly, the Chair will require that they abstain from voting on such a method during the final method selection process. The CSO or OMB may also allow Pool members that qualify under the requirements of expert reviewers, but for whom there is a known or potential conflict of interest to be present as an observer on any particular Panel. In these cases, and only at the discretion of the Chair, observers may provide comments, but only if and when called upon by the Chair to do so. Non-disclosure Statement: All members of an ERP must have signed the AOAC Volunteer Acceptance Form. For certain contracts, each Pool member or observer chosen may be asked to sign a non-disclosure statement agreeing not to discuss or disclose confidential information presented and discussed during meetings of the ERP. Meetings of the ERP: The ERP Chair will organize meetings of the ERP, to review the methods and accompanying validation data, score them numerically, and prepare a summary report. Meetings of the ERP can include voting members of the Panel, and non-voting members (AOAC staff, stakeholder members, and observers). The CSO may assist the Panel Chair in facilitating meetings. The members of the Panel are to review distributed documents before the meeting. To facilitate the process, the Chair may assign primary and secondary reviewers for each method. The primary and secondary reviewers prepare a short critique of the method that is distributed or presented to the ERP. If both the primary and secondary reviewers conclude that the method should not be considered further, the ERP Chair may call for a vote by the Panel; if a unanimous vote to drop a method without further discussion results, the Chair removes the method from further consideration. The Panel then discusses each of the remaining methods in turn. Method Selection Process: The ERP will evaluate all of the methods in a scientifically unbiased manner. Occasionally, a large number of analytical methods of variable quality are encountered. When this occurs, the following “pre-screening” procedure is suggested to eliminate methods that are not satisfactory. The Chair of the ERP with the assistance of at least one other member of the ERP may review all of the methods and remove unsatisfactory methods from consideration. The remainder of the methods would be sent to the ERP members for review.

Approved by Official Methods Board, November 13, 2008 Approved by AOAC Board of Directors, December 9, 2008 Appeals Process Appended – September 2009 Revisedby AOAC Board of Directors, May 25, 2011

Page 3 of 6

The basic requirements for selection of methods for further validation studies will be: fitness for purpose, applicability to the scope needed, clarity of method description, satisfactory performance characteristics, and single laboratory validation data. To assist the Panel, the AOAC will provide a “Methods Selection Worksheet,” which may be modified at the discretion of the ERP. ERP members will identify the best method(s) for further validation, and identify any modifications to be made to the method. An example of the Method Selection Worksheet is attached. Samples: The ERP will be asked to recommend the specific materials (matrices) to be included in the subsequent validation studies, along with detailed justifications. Summary Report: The Chair of the ERP prepares a Summary Report clearly enunciating the recommendations of the Panel, the manner in which these conclusions were reached, any modifications of the method(s) chosen, and the materials (matrices) to be included in the validation studies. The report is to be submitted to the ERP in a timely fashion after the concluding ERP meeting. Comments are also due back to the ERP Chair in a timely fashion. The report is then sent to the stakeholders and a copy is forwarded to the Chair of the OMB. Post-ERP Activities: AOAC retains the right to call on the panelists, as well as members of the Industry Groups, for continued assistance in the subsequent validation studies. This may include (1) help in obtaining the required samples for use in the subsequent validation studies, as well as participating laboratories; (2) help in developing and reviewing the validation study protocols; and (3) help in reviewing the data resulting from the validation studies and reviewing the manuscript describing the results. These activities will be coordinated by the CSO.

Method Selection Worksheet

Method Title: Method Number: Overall evaluation score (1being lowest, 10 being highest): Additional Factors to Consider:

Recommendation: Signature (date):

Approved by Official Methods Board, November 13, 2008 Approved by AOAC Board of Directors, December 9, 2008 Appeals Process Appended – September 2009 Revisedby AOAC Board of Directors, May 25, 2011

Page 4 of 6

Expert Review Panel Selection Criteria: 1. AOAC paid consultants and AOAC staff should not act as Chairs of ERPs. 2. Members of the BoD may act as voting members but it is recommended that they sit as non-voting members of the panel, unless the CSO can demonstrate that there are so few experts in the field available to the community that they are needed to move the project forward.

3. Paid consultants of AOAC and AOAC staff may not serve as voting members on ERPs.

4. If a single business location is represented by more than one person on an ERP, that location shall have only one vote.

5. The Chair of the ERP must be a member of AOAC INTERNATIONAL.

Appeals Process: ERP - Openness of Process and Appeals:

The entire ERP review process is fully open. Any interested party (person, agency, organization, association, company, Chief Scientific Officer (CSO), or group) shall have the right to comment. Appeals or comments are sent to the AOAC Staff. Technical decisions by the ERP are final and are not subject to review or appeal. Other questions or issues regarding procedures, conflict of interest, or impropriety may be

appealed to the President of the AOAC INTERNATIONAL. All written concerns will be considered and given a response.

If there is disagreement between the CSO and the Official Methods Board reviewers, the CSO may appeal to the Chair of the Official Methods Board for consideration. The Official Methods Board can select an impartial panel to review the issue, which must report to the Official Methods Board with a resolution within 21 days of its assignment.

Approved by Official Methods Board, November 13, 2008 Approved by AOAC Board of Directors, December 9, 2008 Appeals Process Appended – September 2009 Revisedby AOAC Board of Directors, May 25, 2011

Page 5 of 6

Appeal From CSO or any interested party

President of AOAC Int’l

Chair of the Official Methods Board

AOAC Staff

From CSO

From interested party

Minor Resolutions to CSO

Response to Appeal Determined

Assign >2 reviewers from OMB

AOAC Staff

Response to OMB within 21 days

Notify interested party

Unresolved

OMB vote (majority vote)

Resolved

Unresolved

Approved by Official Methods Board, November 13, 2008 Approved by AOAC Board of Directors, December 9, 2008 Appeals Process Appended – September 2009 Revisedby AOAC Board of Directors, May 25, 2011

Page 6 of 6

$SSHQGL[ * 3URFHGXUHV DQG *XLGHOLQHV IRU WKH 8VH RI $2$& 9ROXQWDU\ &RQVHQVXV 6WDQGDUGV WR (YDOXDWH &KDUDFWHULVWLFV RI D 0HWKRG RI $QDO\VLV

WKDW DUH UHTXLUHG WR EH UHVROYHG SULRU WR DGRSWLRQ DV D )LQDO $FWLRQ Official Method. 0HWKRGV DGRSWHG E\ DQ (53 DV )LUVW $FWLRQ Official Methods PD\ QRW EH LQ $2$& Official Methods IRUPDW 0HWKRG GHYHORSHUV DXWKRUV DUH DVNHG WR DVVLVW $2$& WR UHZULWH WKH PHWKRG DQG DFFRPSDQ\LQJ PDQXVFULSW LQWR DQ $2$& DFFHSWDEOH IRUPDW Two-Year First Action Evaluation Period 8QGHU WKH QHZ SDWKZD\ D PHWKRG PD\ EH GHVLJQDWHG DV D )LUVW $FWLRQ Official Method EDVHG RQ WKH FROOHFWLYH MXGJPHQW RI DQ (53 Official Methods UHPDLQ DV )LUVW $FWLRQ IRU D SHULRG RI DERXW \HDUV 'XULQJ WKH )LUVW $FWLRQ SHULRG WKH PHWKRG ZLOO EH XVHG LQ ODERUDWRULHV DQG PHWKRG XVHUV ZLOO EH DVNHG WR SURYLGH IHHGEDFN RQ WKH SHUIRUPDQFH RI WKH PHWKRG $V SUHYLRXVO\ GHVFULEHG WZR RU PRUH (53 PHPEHUV DUH DVVLJQHG WR OHDG WKH UHYLHZ RI FDQGLGDWH PHWKRGV IRU DGRSWLRQ DV )LUVW $FWLRQ Official Methods $IWHU D PHWKRG KDV EHHQ DGRSWHG DV )LUVW$FWLRQ WKHVH OHDG UHYLHZHUV DUH H[SHFWHG WR NHHS WUDFN RI WKH XVH RI DQG H[SHULHQFH ZLWK WKH )LUVW $FWLRQ Official Method $W WKH FRQFOXVLRQ RI WKH \HDU HYDOXDWLRQ SHULRG RQH RU ERWK RI WKH OHDG UHYLHZHUV ZLOO UHSRUW EDFN WR WKH (53 RQ WKH H[SHULHQFH RI WKH )LUVW $FWLRQ Official Method. 7KH SUHVLGLQJ (53 ZLOO PRQLWRU WKH SHUIRUPDQFH RI WKH PHWKRG DQG DW WKH FRPSOHWLRQ RI WKH \HDU )LUVW $FWLRQ HYDOXDWLRQ SHULRG GHWHUPLQH ZKHWKHU WKH PHWKRG VKRXOG EH UHFRPPHQGHG WR WKH 20% IRU DGRSWLRQ DV DQ $2$& )LQDO $FWLRQ Official Method ,W LV DOVR SRVVLEOH WKDW )LUVW $FWLRQ Official Methods DUH QRW UHFRPPHQGHG IRU )LQDO $FWLRQ 7KHUH DUH WZR SRVVLELOLWLHV IRU DQ (53 WR GHFLGH QRW WR SURFHHG ZLWK D )LUVW $FWLRQ PHWKRG 1 IHHGEDFN IURPPHWKRG XVHUV LQGLFDWHV WKDW D )LUVW $FWLRQ PHWKRG LV QRW SHUIRUPLQJ DV ZHOO LQ WKH ¿HOG DV ZDV H[SHFWHG RU 2 DQRWKHU PHWKRG ZLWK EHWWHU SHUIRUPDQFH FKDUDFWHULVWLFV KDV EHHQ GHYHORSHG DQG UHYLHZHG ,Q HLWKHU FDVH WKH (53PD\ FKRRVH WR UHSHDO WKH )LUVW $FWLRQ VWDWXV RI D PHWKRG OMB Review 7KH 20% ZLOO UHYLHZ DOO PHWKRGV UHFRPPHQGHG IRU )LQDO $FWLRQ RU UHSHDO E\ WKH (53 DQG ZLOO FRQVLGHU D QXPEHU RI IDFWRUV LQ WKHLU GHFLVLRQ $JXLGDQFH GRFXPHQW IRU IDFWRUV WR FRQVLGHU LV SURYLGHG RQ WKH $2$& ZHEVLWH DW KWWS ZZZDRDF RUJ YPHWK 20%B(53B*XLGDQFH SGI 6RPH RI WKH IDFWRUV LGHQWL¿HG E\ WKH JXLGDQFH GRFXPHQW IRU 20% FRQVLGHUDWLRQ DUH 1 IHHGEDFN IURP PHWKRG XVHUV 2 FRPSDULVRQ WR WKH DSSURSULDWH 6035 3 UHVXOWV IURP VLQJOH ODERUDWRU\ YDOLGDWLRQ 4 UHSURGXFLELOLW\ XQFHUWDLQW\ DQG SUREDELOLW\ RI GHWHFWLRQ 5 DYDLODELOLW\ RI UHIHUHQFH PDWHULDOV DQG 6 VDIHW\ FRQFHUQV Conclusion 7KH QHZ SDWKZD\ WR Official Methods SM LV GHOLEHUDWHO\ GHVLJQHG WR DYRLG FUHDWLRQ RI HODERUDWH UHYLHZ V\VWHPV 7KH LQWHQW RI WKH PRGHO LV IRU PHWKRG H[SHUWV WR XVH WKHLU VFLHQWL¿F NQRZOHGJH H[SHULHQFH DQG JRRG MXGJPHQW WR LGHQWLI\ DQG DGRSW WKH EHVW PHWKRGV SRVVLEOH IRU WKH DQDO\WLFDO QHHG

([SHUW 5HYLHZ 3DQHOV 2I¿FLDO 0HWKRGV %RDUG )LUVW DQG )LQDO $FWLRQ Official Methods SM In early 2011, an AOAC Presidential Task Force recommended that AOAC use Expert review panels (ERPs) to assess candidate methods against standard method performance requirements (SMPRs) to ensure that adopted First Action Official Methods SM are fit for purpose. Formation of an ERP $2$& (53V DUH DXWKRUL]HG WR DGRSW FDQGLGDWH PHWKRGV DV )LUVW $FWLRQ Official Methods DQG WR UHFRPPHQG DGRSWLRQ RI WKHVH PHWKRGV WR )LQDO $FWLRQ Official Methods VWDWXV 6FLHQWLVWV DUH UHFUXLWHG WR VHUYH RQ (53V E\ D YDULHW\ RI ZD\V 1RUPDOO\ D FDOO IRU H[SHUWV LV SXEOLVKHG DW WKH VDPH WLPH DV D FDOO IRU PHWKRGV LV SRVWHG ,QWHUHVWHG VFLHQWLVWV DUH LQYLWHG WR VXEPLW WKHLU curriculum vitae &9 IRU FRQVLGHUDWLRQ $GYLVRU\ SDQHO VWDNHKROGHU SDQHO DQG ZRUNLQJ JURXS PHPEHUV PD\ PDNH UHFRPPHQGDWLRQV WR$2$& IRU (53 PHPEHUV $OO &9V DUH UHYLHZHG DQG HYDOXDWHG IRU H[SHUWLVH E\ WKH $2$& &KLHI 6FLHQWL¿F 2I¿FHU &62 7KH &9V DQG &62 HYDOXDWLRQV DUH IRUZDUGHG WR WKH 20% IRU IRUPDO UHYLHZ %RWK WKH &62 DQG 20% VWULYH WR HQVXUH WKDW WKH FRPSRVLWLRQ RI D SURSRVHG (53 LV ERWK TXDOL¿HG DQG UHSUHVHQW WKH YDULRXV VWDNHKROGHU JURXSV 7KH UHFRPPHQGHG (53 PHPEHUV DUH VXEPLWWHG WR WKH $2$& SUHVLGHQW ZKR WKHQ DSSRLQWV WKH (53 PHPEHUV Review of Methods 0HWKRGV VXEPLWWHG WR $2$& LQ UHVSRQVH WR D FDOO IRU PHWKRGV DUH FROOHFWHG DQG FRPSLOHG E\ $2$& VWDII 7KH $2$& &62 DQG ZRUNLQJ JURXS FKDLU SHUIRUP D SUHOLPLQDU\ UHYLHZ RI WKH PHWKRGV DQG FODVVLI\ WKHP LQWR WKUHH FDWHJRULHV 1 IXOO\ GHYHORSHG DQG ZULWWHQ PHWKRGV WKDW DSSHDU WR PHHW 6035V 2 IXOO\ GHYHORSHG DQG ZULWWHQ PHWKRGV WKDW PD\ RU PD\ QRW PHHW 6035V DQG 3 LQFRPSOHWH PHWKRGV ZLWK QR SHUIRUPDQFH GDWD 0HWKRG VXEPLWWHUV DUH DSSULVHG RI WKH HYDOXDWLRQ RI WKHLU PHWKRGV 0HWKRG GHYHORSHUV ZLWK VXEPLVVLRQV WKDW DUH FODVVL¿HG DV &DWHJRU\ RU DUH HQFRXUDJHG WR SURYLGH DGGLWLRQDO LQIRUPDWLRQ LI DYDLODEOH $ OLVW RI DOO WKH VXEPLWWHG PHWKRGV DQG WKHLU FODVVL¿FDWLRQV DUH SRVWHG IRU SXEOLF UHYLHZ 8VXDOO\ WZR (53 PHPEHUV VRPHWLPHV PRUH DUH DVVLJQHG WR OHDG WKH UHYLHZ RI HDFK &DWHJRU\ PHWKRG $Q (53 PHHWLQJ LV FRQYHQHG WR UHYLHZ WKH PHWKRGV (53 PHHWLQJV DUH RSHQ WR DOO LQWHUHVWHG SDUWLHV DQG DUH XVXDOO\ ZHOO DWWHQGHG HYHQWV ZLWK DERXW ± DWWHQGHHV FRPPRQ (DFK &DWHJRU\ PHWKRG LV UHYLHZHG DQG GLVFXVVHG E\ WKH (53 ,I VWDNHKROGHUV KDYH GHVLJQDWHG WKH PHWKRG WR EH D GLVSXWH UHVROXWLRQ PHWKRG DV VWDWHG LQ WKH 6035 WKHQ WKH (53 LV DVNHG WR LGHQWLI\ WKH VLQJOH EHVW FDQGLGDWH PHWKRG WR EH DGRSWHG DV D )LUVW $FWLRQ Official Method ,I WKH 6035 GRHV QRW VSHFLI\ WKH QHHG IRU D GLVSXWH UHVROXWLRQ PHWKRG WKHQ WKH (53 PD\ FKRRVH WR DGRSW DOO PHWKRGV WKDW PHHW WKH 6035V RU PD\ FKRRVH WR DGRSW WKH VLQJOH EHVW PHWKRG LQ WKHLU FROOHFWLYH H[SHUW RSLQLRQ ,Q DGGLWLRQ DQ (53PD\ FKRRVH WR UHTXLUH FKDQJHV WR D FDQGLGDWH PHWKRG DV SDUW RI LWV )LUVW $FWLRQ DGRSWLRQ DQG RU LGHQWLI\ LVVXHV

‹ $2$& ,17(51$7,21$/