Determination of a reference value, associated standard uncertainty and degrees of equivalence

Transcription

1 Determnaton of a erence value, assocated standard uncertanty and degrees of equvalence for CCRI(II key comparson data Stefaan Pommé 01 Report EUR 5355 E

2 European Commsson Jont Research Centre Insttute for Reference Materals and Measurements Contact nformaton Stefaan Pommé Address: IRMM, Reteseweg 111, 440 Geel, Belgum E-mal: Tel.: Fax: Ths publcaton s a Reference Report by the Jont Research Centre of the European Commsson. Legal otce ether the European Commsson nor any person actng on behalf of the Commsson s responsble for the use whch mght be made of ths publcaton. Europe Drect s a servce to help you fnd answers to your questons about the European Unon Freephone number (*: (* Certan moble telephone operators do not allow access to numbers or these calls may be blled. A great deal of addtonal nformaton on the European Unon s avalable on the Internet. It can be accessed through the Europa server JRC 7185 EUR 5355 E ISB ISS do:10.787/61338 Luxembourg: Publcatons Offce of the European Unon, 01 European Unon, 01 Reproducton s authorsed provded the source s acknowledged. Prnted n Belgum

3 Determnaton of a erence value, assocated standard uncertanty and degrees of equvalence for CCRI(II key comparson data Stefaan Pommé Insttute for Reference Materals and Measurements, 440 Geel, Belgum stefaan.pomme@ec.europa.eu Aprl 01 Executve summary CCRI(II key comparson data consst of a measured value of actvty concentraton, ndependently obtaned, and the assocated standard uncertanty for each laboratory partcpatng n a key comparson. A method s proposed for calculatng a key comparson erence value (KCRV, ts assocated standard uncertanty, and degrees of equvalence (DoE for the laboratores. The method allows for techncal scrutny of data, correcton or excluson of extreme data, but above all uses an estmator (power-moderated mean, PMM that can calculate an effcent and robust mean from any data set. For mutually consstent data, the method approaches a weghted mean, the weghts beng the recprocals of the varances (squared standard uncertantes assocated wth the measured values. For data sets suspected of nconsstency, the weghtng s moderated by augmentng the laboratory varances by a common amount and/or decreasng the power of the weghtng factors. The PMM has the property that for ncreasngly dscrepant data sets there s a smooth transton of the KCRV from the weghted mean to the arthmetc mean. It s a good compromse between effcency and robustness, whle provdng also a relable uncertanty. Before applyng the method, the data provded by the key comparson partcpants should be scrutnsed to see whether any appear to be dscrepant. Extreme data may also be dentfed subsequently by the applcaton of a sutable statstcal crteron. Such data should be consdered for excluson from the calculaton of the KCRV on relevant techncal grounds. Then the KCRV, ts assocated uncertanty and DoEs can meanngfully be obtaned. DoEs are calculated usng the uncertantes provded by the key comparson partcpants, not the augmented uncertantes.

4 1 Introducton The CCRI(II organses key comparsons n whch each of partcpatng laboratores ndependently provdes a measured value of an actvty concentraton x and an assocated standard uncertanty u. Untl now n the CCRI(II, the uncertantes u have generally been dsregarded for the calculaton of a KCRV, the KCRV beng calculated as an arthmetc (unweghted mean. The CCRI(II s now consderng calculatng a KCRV usng a method, such as the weghted mean, that accounts for the u. However, the CCRI(II takes nto consderaton that these uncertanty values are generally mperfect estmates of the combned effect of all sources of varablty, and theore also prone to error. A method s proposed for calculatng a key comparson erence value (KCRV, ts assocated standard uncertanty, and degrees of equvalence (DoEs for the laboratores. The method s based on a few fundamental prncples: - The estmator should be effcent, provdng an accurate KCRV on the bass of the avalable data set (x, u and techncal scrutny. - The estmator should gve a realstc standard uncertanty on the KCRV. - The data are treated on an equal footng, albet that relatve weghtng may vary as a functon of stated uncertanty. The method shall optmse the use of nformaton contaned n the data. - Before evaluaton, all data s scrutnsed n an ntal data screenng by (representatves of the CCRI(II, whch may choose to exclude or correct data on techncal grounds from the calculaton of the KCRV. - Extreme data can be excluded from the calculaton of the KCRV on statstcal grounds as part of the method. The CCRI(II s always the fnal arbter regardng excludng any data from the calculaton of the KCRV. - The estmator should be robust aganst extreme data, n case such data have not been excluded from the data set. It should also adequately cope wth dscrepant data sets. - The method s perably not complex and convenently reproducble. The estmator of choce s the power-moderate weghted mean (PMM, an upgrade of the well-establshed Mandel-Paule mean [1-], ncorporatng deas by Pommé-Spasova [3]. Its results are generally ntermedate between arthmetc and weghted mean. Annex A gves a resume of the ratonale behnd the choce of estmator on the bass of conclusons drawn from smulatons. Annex B contans an overvew of relevant formulae for the arthmetc mean, the classcal weghted mean, the Mandel-Paule mean and the PMM. In Annex C, formulae are derved for degree of equvalence and outler dentfcaton. Secton dscusses the use of the PMM estmator for the KCRV and ts uncertanty. Secton 3 consders the dentfcaton and treatment of data regarded as statstcally extreme. Secton 4 descrbes the determnaton of degrees of equvalence. Secton 5 summarses the proposed method. Secton 6 gves conclusons.

5 The PMM estmator The PMM estmator combnes aspects of the arthmetc mean, the weghted mean and the Mandel-Paule mean. The logcal steps leadng to ths procedure can be read n Annex B. In ths secton, the mathematcal steps are shown n order of executon: 1 Calculate the Mandel-Paule mean x mp 1 u x s 1 u 1 s usng s =0 as ntal value, conform to the weghted mean. (1 Calculate the modfed reduced erved ch-squared value 1 ( x xmp ~ ( 1 1 u s 3 If ~ >1, ncrease the varance s and repeat steps 1- untl ~ =1 s obtaned. 4 Assess the relablty of the uncertantes provded. Choose a value for the power of the uncertantes n the weghtng factors: power = 0 = 0 relablty of uncertantes unnformatve uncertantes (arthmetc mean uncertanty varaton due to error at least twce the varaton due to metrologcal reasons (arthmetc mean = -3/ nformatve uncertantes wth a tendency of beng rather underestmated than overestmated (ntermedately weghted mean = = nformatve uncertantes wth a modest error; no specfc trend of underestmaton (Mandel-Paule mean accurately known uncertantes, consstent data (weghted mean 5 Calculate a characterstc uncertanty per datum, based on the varance assocated wth the arthmetc mean, x, or the Mandel-Paule mean, x mp, whchever s larger 1. 1 Both varances are equal when ~ =1. 3

6 n whch S max( u ( x, u ( x (3 mp ( x x 1 u ( x, x 1 ( 1 1 x and 1 ( mp 1 u x (4 1 u s 6 Calculate the erence value and uncertanty from a power-moderated weghted mean 1 x w x u s S 1 u ( x 1 n whch the normalsed weghtng factor s w 1 ( u x u s S (6 1 (5 3 Treatment of extreme data Statstcal tools may be used to ndcate data that are extreme. An extreme datum s such that the magntude of the dfference e between a measured value x and a canddate KCRV x exceeds a multple of the standard uncertanty u(e assocated wth e : e > ku(e, e = x x, (7 where k s a coverage factor, typcally between two and four, correspondng to a specfed level of confdence. Irrespectve of the type of mean, the varance of the dfference s convenently calculated from the modfed uncertantes through the normalsed weghtng factors (see Annex C: 1 u ( e u ( x ( 1 (x ncluded n mean (8 w 1 u ( e u ( x ( 1 (x excluded from mean (9 w Perably, the dentfcaton and rejecton of extreme data s kept to a mnmum, so that the mean s based on a large subset of the avalable data. A default coverage factor of k=.5 s recommended. After excluson of any data, a new canddate KCRV and ts assocated uncertanty are calculated, and on the bass of test (7 possbly further extreme values are dentfed. The process s repeated untl there are no further extreme values to be excluded. The CCRI(II s always the fnal arbter regardng excludng any data from the calculaton of the KCRV. In ths way, the KCRV can be protected aganst extreme values that are asymmetrcally dsposed wth respect to the KCRV, and the standard uncertanty assocated wth the KCRV s reduced. The approach of usng the modfed uncertantes lmts the number of values for whch the nequalty n expresson (7 holds. 4

7 4 Degrees of equvalence The degrees of equvalence, DoE, for the th laboratory has two components (d, U(d, where, assumng normalty, d = x x, U(d = u(d. (9 u(d s the standard uncertanty assocated wth the value component d, and U(d, the uncertanty component, s the expanded uncertanty at the 95 % level of confdence. Gven a KCRV x and the assocated standard uncertanty u(x obtaned from expressons (5-6, the correspondng DoEs are determned from the generally vald expresson for any knd of weghted mean (see Annex C: u ( d (1 w u u ( x. (10 The DoEs for partcpants whose data were excluded from the calculaton of the KCRV are gven by essentally the same expresson, applyng w = 0: u ( d u u ( x. (11 The varance u assocated wth x s not augmented by s for the calculaton of the DoE, snce t s the measurement capablty of laboratory, ncludng proper uncertanty statement, that s beng assessed. 5 Summary of the method 1. Carry out a caul examnaton of the partcpants data. If necessary, correct or exclude erroneous data on techncal grounds.. Form the weghted mean and the assocated standard uncertanty of the remanng data, (x, u, = 1,,, usng Eq. (1 wth s =0. 3. Test for consstency of the data wth the weghted mean by calculatng ~ Eq. (, regardng the data as consstent f 1. ~ usng ~ 4. If 1, calculate the Mandel-Paule mean of the remanng data. That s, the varance s n the weghted mean (Eq. 1 s chosen such that ~ (Eq. s unty. 5. Choose a value for the power based on the relablty of the uncertantes and the sample sze. 6. Calculate the PMM and ts standard uncertanty from Eqs. ( Use the statstcal crteron n Eq. (7 to dentfy any further extreme values, applyng the normalsed weghtng factors (Eqs Should the CCRI(II exclude such data from the calculaton of the KCRV, repeat steps -6 on the remanng data set. 8. Take the PMM as the KCRV and ts assocated standard uncertanty as the standard uncertanty assocated wth the KCRV. 9. Form the DoEs for all partcpatng laboratores (Eqs In the partcular case of the classcal weghted mean, the expresson for u (d s dentcal to u u (x, because w = u (x /u. The more general expresson has to be used for the Mandel-Paule and PMM (s > 0 or <. 5

8 6 Conclusons The method proposed here for calculatng a KCRV and ts uncertanty s based on a weghted mean, n whch the relatve weghtng factors are adjusted to the level of consstency n the data set. It apples when the measured values provded by the partcpants n the key comparson are mutually ndependent. The method foresees n protecton aganst erroneous and extreme data through the possblty for correcton or excluson of data. Further dscrepancy of the data, most typcally caused by understatement of the uncertanty, s generally well taken nto account by the estmator. Ths s establshed by augmentng the uncertantes and reducng the power of uncertantes n the weghtng factors. Ths s done purely for the calculaton of the KCRV, as the laboratory data reman unaltered when obtanng degrees of equvalence. For consstent data wth correctly determned uncertantes, the KCRV approaches the classcal weghted mean. For hghly dscrepant data wth unnformatve uncertantes, the KCRV approaches the arthmetc mean. There s a smooth transton from the weghted mean to the arthmetc mean as the degree of data nconsstency ncreases. For CCRI(II ntercomparson results, typcally slghtly dscrepant data wth nformatve but mperfectly evaluated uncertantes, the KCRV s ntermedate between the Mandel-Paule mean and the arthmetc mean. The resultng KCRV should n general combne good effcency and robustness propertes. The approach wll not perform well f a majorty of the data values have sgnfcant bas of the same sgn. The assocated uncertanty wll n most cases be realstc. The method remans vulnerable to manly small, seemngly consstent data sets wth systematcally understated (or overstated uncertantes. Acknowledgements The author s ndebted to Prof. Dr. Maurce Cox and Dr. Peter Harrs of PL, who coauthored the frst proposal submtted to the CCRI(II n ovember 010. References 1. J. Mandel, R.C. Paule, Interlaboratory evaluaton of materal wth unequal number of replcates. Anal Chem 4 ( R.C. Paule, J. Mandel, Consensus values and weghtng factors. J Res at Bur Std 87 ( S. Pommé, Y. Spasova, A practcal procedure for assgnng a erence value and uncertanty n the frame of an nterlaboratory comparson. Accred Qual Assur 13 (

9 Annex A. Ratonale behnd the choce of estmator Measurement results show devatons from the "true" value of the measurand. The same s also true for the reported uncertanty, whch n general s only a rough estmate of the combned effect of all sources of varablty. Computer smulatons were performed to evaluate the performance of estmators of the mean of data sets, n partcular of dscrepant data sets for whch the varaton of the data x exceeds expectaton from the stated uncertantes u. Important crtera were effcency, a measure for the accuracy by whch the true value was approached, robustness aganst extreme data, and relablty, a measure for the accuracy of the uncertanty value provded. Some conclusons were as follows: 1. If none of the u s nformatve, the arthmetc mean s the most effcent. In practce, the arthmetc mean s a good choce for data wth poorly known uncertanty,.e. f the varaton of uncertantes u due to error n the uncertanty assessment s twce or more the varaton due to metrologcal orgn.. If the uncertantes u are nformatve, an estmator that uses them can be employed to mprove the effcency. Some approaches usng the u are better than others. 3. The classcal weghted mean, usng the recprocal of the varance as weghtng factor, s the most effcent estmator for normally dstrbuted data, only n absence of unrepresentatve data. Even modest contamnaton by such data, n partcular those havng extreme values and/or understated uncertantes, results n a too low uncertanty estmate. 4. The Mandel-Paule mean provdes a good combnaton of effcency and robustness for dscrepant data that are approxmately symmetrcally dsposed wth respect to the KCRV. It degrades lttle wth ncreasng contamnaton, s only slghtly dependent on the level of nformaton carred by the u, and s superor to the classcal weghted mean when the u are not very nformatve. 5. The PMM yelds more relable uncertantes for dscrepant data sets n whch uncertantes are lkely to be underestmated. It uses moderate weghtng for small data sets, thus beng less nfluenced by undentfed outlers wth underestmated uncertantes. 6. In the presence of extreme values, the M-P and PMM estmators approach the result of the arthmetc mean. They can easly be complemented wth an outler rejecton mechansm, whch mproves ther effcency. For a consstent data set, the M-P mean and PMM approach the classcal weghted mean. 7

10 Annex B. Formulae for KCRV estmators and assocated standard uncertantes 1. Arthmetc mean The arthmetc mean s calculated from 1 (B.1 x x 1 and ts uncertanty, applyng the propagaton rule, s 1 (B. u( x u. 1 As the arthmetc mean s of partcular nterest when the u are not nformatve, one can replace the ndvdual varances u by an estmate of the sample varance u 1 ( x x (B resultng n 1/ 1 u ( x u( x u = x (B.4 1 ( 1 As the dsperson of data s determned by chance, the calculated uncertanty of the mean can sometmes be unrealstcally low, n partcular wth small data sets showng almost no scatter. If the u are nformatve wth respect to the uncertanty scale, one could take the maxmum value from both approaches: 1 ( x x (B.5 u( x max u, 1 1 ( 1. Weghted mean The classcal weghted mean uses the recprocal varances as weghtng factor. The weghted mean x of the data set and the assocated standard uncertanty u(x are obtaned from x 1 x x or x u ( x (B.6 1 u 1 u 1 u and 1/ or u ( x (B.7 u ( x 1 u 1 u The weghted mean and ts uncertanty are partcularly nadequate when appled to dscrepant data wth understated uncertantes. One can look for ndcatons of dscrepancy by calculatng the reduced erved ch-squared value 1 ( x x ~ (B u 8

11 A ~ -value (sgnfcantly hgher than unty (s may be an ndcaton of nconsstency Mandel-Paule mean The M-P mean was desgned to deal wth dscrepant data sets, havng a reduced erved ch-squared value ~ larger than unty. For the purpose of establshng a more robust mean, the laboratory varances u are ncremented by a further varance s to gve augmented varances u (x = u + s. The value of the unexplaned varance s s chosen such that the modfed reduced erved ch-squared value, 1 ( x x ~, (B u s equals one. The calculaton of the M-P mean and ts uncertanty proceeds through the same equatons as for the weghted mean, replacng the stated varances u by the augmented varances u (x x 1 1 x u ( x,. (B.10 1 u s u ( x 1 u s As the M-P mean x occurs n the equaton for ~, an teratve procedure s appled to fnd the approprate value of the varance s. For data sets wth ~ smaller than 1, the varances are not augmented, s = 0, and the result s dentcal to the weghted mean. For an extremely nconsstent set, s wll be large compared wth the u and the Mandel-Paule mean wll approach the arthmetc mean. For ntermedate cases, the nfluence of those laboratores that provde the smallest uncertantes wll be reduced and the standard uncertanty assocated wth the KCRV wll be larger compared wth that for the weghted mean. Though much more robust than the weghted mean, the M-P procedure tends to underestmate ts uncertanty for data sets wth predomnantly understated uncertantes. 4. The PMM The M-P does not counteract possble errors n the relatve uncertantes when the data set appears to be consstent, ths s when ~ s not (much larger than unty. Data wth understated uncertanty have a negatve effect on the robustness and the calculated uncertanty. The PMM estmator allows moderatng the relatve weghtng also for seemngly consstent data sets. For the M-P mean as well as the classcal weghted mean, uncertantes u are used wth a power of n the weghtng factor. By lowerng ths power, the nfluence of understated uncertantes can be moderated. A smooth transton from weghted to arthmetc mean can be realsed by ntermxng the uncertantes assocated wth both. Lke wth the M-P mean, the varances are ncreased by an unexplaned amount s to ascertan that ~ s not larger than one. Then a varance per datum s calculated for an 3 The mean (or expectaton of a varable havng a ch-squared dstrbuton wth 1 degrees of freedom s 1. Under normalty condtons, the expected value of ~ s unty. 9

12 unweghted mean, takng the larger value between the sample varance and the combned augmented uncertantes: S 1 1 ( x x 1 max, 1 u s ( 1 In the expressons (B.10, the weghtng factor 1/(u + s s replaced by (B.11 1 u s S, (B.1 n whch the power α (0 α s the leverage by whch the mean can be smoothly vared between arthmetc mean (α=0 and M-P mean (α=. The Mandel-Paule method can be regarded as a subset of the PMM method. Reducng has a smlar effect to the KCRV and ts uncertanty as does augmentng the laboratory varances n the M-P method. The choce of can be made to lect the level of trust n the stated uncertantes. For data sets wth a predomnance of understated uncertantes, one obtans a more realstc uncertanty on the KCRV by reducng the power. Ths s partcularly recommended wth small data sets. Larger data sets have a better defned ~, facltatng the dentfcaton of extreme data and the level of relablty of the u. As a practcal rule, for data sets n whch the uncertantes u are nformatve but frequently understated, one can make the power depend on the number of data va a heurstc formula 3 (B.13 Table 1. Estmators of mean x Estmator 1 w x ormalsed weght w Mean x Assocated standard uncertanty u(x Arthmetc mean 1 1 x 1 ( x x 1 ( 1 1/ Weghted u ( x mean u u ( x 1 x u 1/ 1 1 u Mandel-Paule u ( x u ( x mean u s u PMM u u ( x s S u ( x 1 x s x 1 u s S 1 1 u 1 u s s 1/ S 1/ 10

13 Annex C. Formulae for 'degree of equvalence' and 'outler dentfcaton' 1. Degrees of equvalence The degrees of equvalence between pars of MIs are not nfluenced by the estmator used for the KCRV. The degree of equvalence of laboratory data wth respect to the KCRV nvolves calculaton of the dfference d x x x wjxj (C.1 j1 and ts expanded uncertanty. In the expresson C.1, the factor w s: - the normalsed weghtng factor (see Table 1 for data ncluded n the mean - zero for data excluded from the calculaton of the KCRV Data excluded from the calculaton of the mean are not correlated wth t and the varance assocated wth d s, n ths case, the sum of two varances: u d u u ( x (x excluded from mean (C. ( The data that have been ncluded n the calculaton of the mean are correlated wth t, and the varance of the dfference s calculated from u ( d (1 w u w u (x ncluded n mean j 1 j j j 1 w u u ( x (x ncluded n mean (C.3 (. Outler dentfcaton For a consstent data set wth relable standard uncertantes, one could apply a weghted mean and use the recprocal of the varances as weghtng factors. Data not complyng wth ths consstent set can be recognsed f ther dfference e from the mean exceeds the assocated uncertanty by a factor k or more. u e (1 w u u ( x (x ncluded n mean ( u ( x ( 1 w u ( x (weghted mean w 1 u ( x ( 1 (weghted mean (C.4 w A smlar equaton, wth opposte sgn, holds for data not ncluded n the mean. u e u u ( x (x excluded from mean ( 1 u ( x ( 1 (weghted mean (C.5 w Typcal CCRI(II ntercomparson data contan understated uncertantes, and normal crtera for outler dentfcaton would reject many data as extreme. Perably, the dentfcaton and rejecton of extreme data s kept to a mnmum, so that the mean s based 11

14 on a large subset of the avalable data. Ths s easly acheved n the phlosophy of the M-P mean, even for dscrepant data sets, by usng the augmented uncertantes u e (1- w ( u s - u (x (Mandel-Paule mean (C.6 ( Smlarly, one can apply the power-moderated uncertanty for the PMM method. In all cases, the varance equatons reduce to the same elegant solutons as n Eqs. (C.4-5, expressed as a functon of the weghtng factor. If the method reduces to an arthmetc mean (=0, the weghtng factors are equal for all data, rrespectve of the stated uncertanty. Extreme data are then dentfed from ther dfference wth the mean only. 1

15 European Commsson EUR Jont Research Centre -- Insttute for Reference Materals and Measurements Ttle: Determnaton of a erence value, assocated standard uncertanty and degrees of equvalence Author: Stefaan Pommé Luxembourg: Publcatons Offce of the European Unon pp x 9.7 cm EUR -- Scentfc and Techncal Research seres - ISS (onlne, ISS (prnt ISB do:10.787/61338 Abstract CCRI(II key comparson data consst of a measured value of actvty concentraton, ndependently obtaned, and the assocated standard uncertanty for each laboratory partcpatng n a key comparson. A method s proposed for calculatng a key comparson erence value (KCRV, ts assocated standard uncertanty, and degrees of equvalence (DoE for the laboratores. The estmator has the property that for ncreasngly dscrepant data sets there s a smooth transton of the KCRV from the weghted mean to the arthmetc mean. It s a good compromse between effcency and robustness, whle provdng also a relable uncertanty. A sutable statstcal crteron s provded to dentfy extreme data.

16 LA-A-5355-E- As the Commsson s n-house scence servce, the Jont Research Centre s msson s to provde EU polces wth ndependent, evdence-based scentfc and techncal support throughout the whole polcy cycle. Workng n close cooperaton wth polcy Drectorates-General, the JRC addresses key socetal challenges whle stmulatng nnovaton through developng new standards, methods and tools, and sharng and transferrng ts know-how to the Member States and nternatonal communty. Key polcy areas nclude: envronment and clmate change; energy and transport; agrculture and food securty; health and consumer protecton; nformaton socety and dgtal agenda; safety and securty ncludng nuclear; all supported through a cross-cuttng and mult-dscplnary approach.

17 Errata On page 11, the equaton before C. 3 should read: ( ( On page 1, the equaton C.6 should read: ( ( ( (