OMB Winter Mtg.-February 5-6, 2015

96

Additional Considerations:

The Official Methods Board should be aware that under section 2 above, repeatability may not be estimated, as within-laboratory replication in a PT study may not meet the ISO definition of repeatability conditions. We recommend calculating within-laboratory variance as an intermediate step to estimating reproducibility standard deviation, but the within-laboratory variance term should not be called “repeatability.”

The above recommendations do not supersede the requirements set out in OMA Appendix D or ISO5725 regarding number of matrices, number of levels, or levels per matrix.

Statistical estimates derived from PT studies should be used cautiously – any attempt to extrapolate reproducibility estimates to different concentration levels or to different matrices must be discouraged. Proficiency data is used to certify laboratories to ISO 17025 standards. ISO 13528 is the standard used for analyzing and reporting proficiency data for laboratory accreditation purposes. The advantage of using PT data when possible is that it will allow estimation of reproducibility on existing data sets. In addition, laboratories may be motivated to participate in proficiency studies in order to satisfy the needs of accreditation bodies and to give external feedback on laboratory performance. This is something that CS do not normally provide to participating collaborators. Unfortunately, many laboratories around the world have been forced to look to cut costs and volunteering to help validate a consensus method may be seen as an unnecessary cost to a laboratory. In effect this proposal will allow laboratories to contribute to validation efforts and be rewarded for their contributions by obtaining a z-score for the proficiency study. The main problem with using proficiency data to estimate reproducibility is the problem of method control in proficiency test regimes. In the past, Proficiency Test (PT) data was all about accrediting the laboratory, and not as much emphasis was put on the method. If PT data is to be used, it can only come from laboratories that confirm they are using the candidate method without modification. Other laboratories in the proficiency study may use other methods, but only data from compliant labs will be used to calculate validation statistics for the candidate method. Another common issue when using PT data is that matrix and level decisions are generally not made for PT studies the way they would be made for randomized, controlled Collaborative Studies (CS). In a CS situation, the study director will meet with a group of advisors and plan out the number of matrices, concentration levels, and design the experiment to cover a planned design space. Proficiency systems very often use convenience samples commonly obtained at