zero_response
0 likes204 views
This document discusses statistics for usability tests when zero non-conforming items are found (i.e. a zero response). It explains that even when zero issues are observed, there is still a probability that non-conforming items exist in the sample. The document provides equations and tables to calculate the upper limit of non-conforming items in a sample for a given confidence level, based on the sample size and a binomial distribution. It also shows how to determine the necessary sample size needed for a given confidence level and expected fraction of non-conforming items.
![Statistics for usability tests with zero response
Zero response
If n tested items of the LOT show a zero response, i.e. zero non-conforming items are found,
a probability exists that non-conforming items are still in the LOT. For n/M ≤ 0.05, i.e. large
LOT size compared to the sample size, the binomial distribution can be used to calculate the
upper limit of non-conforming items for a given confidence level C: [1]
P(k) =
n
k
pk
(1 − p)n−k
(1)
with n the sample size, k the number of non-conforming items and p the fraction of
non-conforming items in a LOT. For k = 0:
P(0) = (1 − p)n
. (2)
One can ask for the largest p the makes P(0) reasonably small. Therefore, we set
P(0) = 1 − C, where C is the confidence level for the upper limit of non-conforming items pu:
1 − C = (1 − pu)n
(3)
or
pu = 1 − n
√
1 − C (4)
Table 1: Maximal fraction of non-conformance items in a LOT in percent as a function of
samples n taken and zero non-compliance items k are found for confidence intervals
between 75% and 99%.
Samples Upper limit in percent
n C: 0.75 0.90 0.95 0.975 0.99
1 75.00 90.00 95.00 97.50 99.00
3 37.00 53.58 63.16 70.76 78.46
5 24.21 36.90 45.07 52.18 60.19
7 17.97 28.03 34.82 40.96 48.21
9 14.28 22.57 28.31 33.63 40.05
11 11.84 18.89 23.84 28.49 34.21
17 7.83 12.67 16.16 19.51 23.73
21 6.39 10.38 13.29 16.11 19.69
25 5.39 8.80 11.29 13.72 16.82
Sometimes it’s very convienient to calculate the sample size n for a given confidence level C
and a upper fraction of non-conforming items 1 − C. This leads to
Cn
+ C − 1 = 0. (5)
Solved for n:
n =
ln(1 − C)
ln(C)
. (6)
Title: Usability stats
Revision: 0
Effective: 2007-10-25
Page 1 of 3
ID-Number:
Author: Dr. Peter Drechsler
Developed: 2007-10-25
Phone: +49 89 5008-4163
Fax: +49 89 5008-4163
peter.drechsler@tuev-sued.de
TÜV R
TÜV SÜD Product Service GmbH
Ridlerstr. 65
80339 Munich
Germany](https://image.slidesharecdn.com/5e04d42c-de95-4245-95fe-9d59d067c81e-160117100610/85/zero_response-1-320.jpg)
![Statistics for usability tests with zero response
0.5 0.6 0.7 0.8 0.9 1.0
0.00.10.20.30.4 n = 11
Confidence level
Fractionofnon−conformingitems
Figure 1: Confidence level versus upper fraction of non-conforming items for n = 11.
References
[1] ASTM E 2334. Standard practice for setting an upper confidence bound for a fraction or
number of non-conforming items, or a rate of occurence for non-conformities, using
attribute data, when there is a zero response in the sample, 2003.
Title: Usability stats
Revision: 0
Effective: 2007-10-25
Page 2 of 3
ID-Number:
Author: Dr. Peter Drechsler
Developed: 2007-10-25
Phone: +49 89 5008-4163
Fax: +49 89 5008-4163
peter.drechsler@tuev-sued.de
TÜV R
TÜV SÜD Product Service GmbH
Ridlerstr. 65
80339 Munich
Germany](https://image.slidesharecdn.com/5e04d42c-de95-4245-95fe-9d59d067c81e-160117100610/85/zero_response-2-320.jpg)
zero_response
- 1. Statistics for usability tests with zero response Zero response If n tested items of the LOT show a zero response, i.e. zero non-conforming items are found, a probability exists that non-conforming items are still in the LOT. For n/M ≤ 0.05, i.e. large LOT size compared to the sample size, the binomial distribution can be used to calculate the upper limit of non-conforming items for a given confidence level C: [1] P(k) = n k pk (1 − p)n−k (1) with n the sample size, k the number of non-conforming items and p the fraction of non-conforming items in a LOT. For k = 0: P(0) = (1 − p)n . (2) One can ask for the largest p the makes P(0) reasonably small. Therefore, we set P(0) = 1 − C, where C is the confidence level for the upper limit of non-conforming items pu: 1 − C = (1 − pu)n (3) or pu = 1 − n √ 1 − C (4) Table 1: Maximal fraction of non-conformance items in a LOT in percent as a function of samples n taken and zero non-compliance items k are found for confidence intervals between 75% and 99%. Samples Upper limit in percent n C: 0.75 0.90 0.95 0.975 0.99 1 75.00 90.00 95.00 97.50 99.00 3 37.00 53.58 63.16 70.76 78.46 5 24.21 36.90 45.07 52.18 60.19 7 17.97 28.03 34.82 40.96 48.21 9 14.28 22.57 28.31 33.63 40.05 11 11.84 18.89 23.84 28.49 34.21 17 7.83 12.67 16.16 19.51 23.73 21 6.39 10.38 13.29 16.11 19.69 25 5.39 8.80 11.29 13.72 16.82 Sometimes it’s very convienient to calculate the sample size n for a given confidence level C and a upper fraction of non-conforming items 1 − C. This leads to Cn + C − 1 = 0. (5) Solved for n: n = ln(1 − C) ln(C) . (6) Title: Usability stats Revision: 0 Effective: 2007-10-25 Page 1 of 3 ID-Number: Author: Dr. Peter Drechsler Developed: 2007-10-25 Phone: +49 89 5008-4163 Fax: +49 89 5008-4163 peter.drechsler@tuev-sued.de TÜV R TÜV SÜD Product Service GmbH Ridlerstr. 65 80339 Munich Germany
- 2. Statistics for usability tests with zero response 0.5 0.6 0.7 0.8 0.9 1.0 0.00.10.20.30.4 n = 11 Confidence level Fractionofnon−conformingitems Figure 1: Confidence level versus upper fraction of non-conforming items for n = 11. References [1] ASTM E 2334. Standard practice for setting an upper confidence bound for a fraction or number of non-conforming items, or a rate of occurence for non-conformities, using attribute data, when there is a zero response in the sample, 2003. Title: Usability stats Revision: 0 Effective: 2007-10-25 Page 2 of 3 ID-Number: Author: Dr. Peter Drechsler Developed: 2007-10-25 Phone: +49 89 5008-4163 Fax: +49 89 5008-4163 peter.drechsler@tuev-sued.de TÜV R TÜV SÜD Product Service GmbH Ridlerstr. 65 80339 Munich Germany
- 3. Statistics for usability tests with zero response Table 2: Sample size n for a given confidence level C. C/% pu n 99.99 0.0001 92099 99.90 0.0010 6904 99.5 0.0050 1057 99 0.0100 458 97 0.0300 115 95 0.0500 58 93 0.0700 37 90 0.1000 22 70 0.3000 4 50 0.5000 1 Title: Usability stats Revision: 0 Effective: 2007-10-25 Page 3 of 3 ID-Number: Author: Dr. Peter Drechsler Developed: 2007-10-25 Phone: +49 89 5008-4163 Fax: +49 89 5008-4163 peter.drechsler@tuev-sued.de TÜV R TÜV SÜD Product Service GmbH Ridlerstr. 65 80339 Munich Germany