Statistics Meeting Book (June 20, 2017)

Qualitative Micro Validation Proposed Plan

Then for the other people, just assign them a page or 2 to do, and have them build the sheet from scratch in their favorite software – WITHOUT LOOKING AT THE CODES IN THE EXCAL SHEET UNDER TEST. It is very critical that they do it from the Appendix. Have them call me or email if they get stuck. The sheets were:

1. 2. 3. 4. 5.

QUAL SLV – No Ref method comparison

QUAL SLV UNPAIRED QUAL SLV PAIRED QUAL COLLAB UNPAIRED QUAL COLLAB PAIRED

The calculations for confidence intervals are the tricky parts, although we also need to check all the calcs for mean PODs and dPODs. In addition, people may have difficulty finding the correct formulae for these confidence intervals, due to the fact that there are several, and also due to the fact that we have added some since publishing the Appendix. Here’s my best shot at identifying where the formula are:

1.

QUAL SLV – No Ref method comparison CI calcs are given in Appx J, Annex C, very first section on the left of the page.

2. QUAL SLV UNPAIRED Calcs are Appx J, Annex C, immediately following the above section, starting at lower left, it gives calcs for dPODs then on the upper right, gives the calcs for the CIs for dPODs. For this sheet, you have 2 methods, so you calc POD values (and CIs) for both methods as done in sheet 1, then calculate dPOD and CIs on the dPOD as given here in this section of Annex C. QUAL SLV PAIRED This one is not in the annex and is discussed in the ppt file we presented at the meeting in Dallas. The section of Annex C called “dPOD for Paired Studies” will need to be changed to incorporate the info in the ppt file. 4. QUAL COLLAB UNPAIRED This one is in the Annex F – we have modified it slightly though in 2 areas. We have found in our work in the ISO group that the very complicated Welch-Satterthwaite estimation for df for the t-based interval is not necessary and the formula df = L-1 works just as well for the collab sizes we are using. So p. 17 of the Appendix J, Annex F, that df calculation can be simplified to just df = L-1, or degrees of freedom is number of labs minus one. The second thing we found was better was to change how we switch from the t-based CI to the binomial-based CI calculator. Originally, we decided to do this based on the POD value, but in simulations, it seems to provide a more seamless transition if it transitions not on POD, but on x, the number of positive hits. For this new version, the Appendix will need to be changed. See ISO Draft standard or call Paul on this. 3.

Then in addition, you will need to calculate s(R) and s(r) and s(L) for the data. So this will be