arxiv: v1 [physics.data-an] 3 Oct 2016

Transcription

1 EPJ Web of Confeences will be set by the publishe DOI: will be set by the publishe c Owned by the authos, published by EDP Sciences, 6 axiv:6.v [physics.data-an] Oct 6 Combination of measuements and the BLUE method Luca Lista,a INFN - Istituto Nazionale di Fisica Nucleae - Sezione di Napoli Abstact. The most accuate method to combine measuement fom diffeent expeiments is to build a combined likelihood function and use it to pefom the desied infeence. This is not always possible fo vaious easons, hence appoximate methods ae often convenient. Among those, the best linea unbiased estimato (BLUE) is the most popula, allowing to take into account individual uncetainties and thei coelations. The method is unbiased by constuction if the tue uncetainties and thei coelations ae known, but it may exhibit a bias if uncetainty estimates ae used in place of the tue ones, in paticula if those estimated uncetainties depend on measued values. In those cases, an iteative application of the BLUE method may educe the bias of the combined measuement. Measuement combination The most igoous and accuate method to combine measuements fom diffeent expeiments elies on the combination of the individual likelihood functions that have been used fo each measuements. Imagine that an expeiment A povides a measued data sample x A, whose likelihood function, chaacteized by a set of paametes θ A, is L( x A ; θ A ), () and an expeiment B povides a sample x B, whose likelihood function, chaacteized by a set of paametes θ B, is L( x B ; θ B ). () The paamete sets θ A and θ B contain some expeiment-specific paametes and some paametes common to both expeiments. Among the latte, thee ae physical paametes of inteest, nuisance paametes elated to common souce of systematic uncetainties, such as theoy uncetainties, acceleatospecific component of luminosity uncetainty, etc. The global likelihood function, if A and B ae independent expeiments, is: L( x A, x B ; θ) = L( x A ; θ A ) L( x B ; θ B ), () whee θ is the set of all common and non-common paametes. Any statistical method can be applied at this level to detemine a combined measuement, uppe limit and/o significance: a Bayesian o fequentis infeence, pofile likelihood, modified fequentist uppe limit, etc. a luca.lista@na.infn.it

2 EPJ Web of Confeences An example of such appoach is povided by the combination of measuements of single top-quak poduction in the s-channel at Tevaton pefomed by the CDF and D expeiments [ ]. CDF and D measuements of single-top poduction coss section wee combined using as input the binned distibutions of multivaiate disciminato outputs fo each individual measuement. Each bin in each data sample was used in a Bayesian analysis, assuming a Poisson distibution, to extact a cental value of the coss-section estimate. A likelihood-atio analysis using asymptotic fomulae [] was instead used to detemine the combined significance of the obseved s-channel signal. Appoximate appoaches: the BLUE method In many cases, unless expeiments, o even analysis goups within the same expeiment, agee in advance, individual likelihood functions may not be available, o ae available unde diffeent softwae famewoks, etc. Appoximate methods can be used to combine the individual esults, which ae usually povided in tems of a cental value and an uncetainty: θ = ˆθ ± σ, θ = ˆθ ± σ. () (5) Coelation between uncetainties, which is elated to the non-diagonal elements of the covaiance matix V ρ = V σ σ, (6) must be popely taken into account. The most popula method to combine coelated measuements is the best linea unbiased estimate (BLUE). The method was fomulated initially in the s [5] and poposed in high-enegy physics in the 8s [6]. By definition, it is the unbiased linea estimato that povides the smallest possible vaiance assuming the tue uncetainties and thei coelation ae known. The estimato is equivalent to a χ minimization which, fo Gaussian distibutions, is also equivalent to a maximum-likelihood estimate. Given two measuements, as in eq. () and (5), the BLUE estimate is given by the following linea combination of the individual measuements: ˆθ = ˆθ (σ ρσ σ ) + ˆθ (σ ρσ σ ) σ ρσ σ + σ (7) with vaiance: σ ˆθ = σ σ ( ρ ) σ ρσ. (8) σ + σ Unlike the usual weighted aveage, fo some values of the coelation coefficient ρ, the coefficients of the linea combination that appea in eq. (7) may be negative. Moe in geneal, fo n measuements ˆθ ± σ,, ˆθ n ± σ n, with a covaiance matix V, the BLUE combination in eq. (7) can be genealized as: with vaiance: ˆθ = n w i ˆθ i, (9) i= σ ˆθ = wt V w, ()

3 CONF The weights in eq. (9) can be computed as: w = V u u T V u () whee u = (,, ) is the vecto with all elements equal to unity. The nomalization condition fo weights w holds: n w i =. () i= In case the weights w i ae positive, an intepetation of the BLUE combination in eq. (7) in tem of weighted aveage is possible by intoducing the common eo [7], defined as: σ c = ρσ σ. () The two measuements in eq. () and (5) can be ewitten with uncetainties given by the sum in quadatue of fully uncoelated contibutions σ and σ and a % coelated contibution σ C: θ = ˆθ ± σ ± σ C, θ = ˆθ ± σ ± σ C, () (5) whee the uncoelated uncetainty contibutions ae defined by: σ = σ σ C, σ = σ σ C, (6) (7) (8) The BLUE combination in eq. (7) achieves an expession simila to a egula weighted aveage: ˆθ = ˆθ σ σ + ˆθ σ + σ, (9) with weights that only take into account the uncoelated uncetainty contibutions, but in this case the uncetainty achieves an additional contibution due to the coelated uncetainty, with espect to the uncetainty of the usual weighted aveage, and is given by: σ ˆθ = + σ σ + σ C. () Cases with negative weights have a less intuitive intepetation than eq. (9), as will be moe evident in section. Quantifying the impotance of individual measuements In ode to quantify the impotance of each individual measuement used in a combination, the fist appoach adopted in liteatue was to quote the so-called elative impotance (RI) of each individual

4 EPJ Web of Confeences measuement, popotional to the absolute value of the coesponding weight w i, and defined as: RI i = w i n. () w i i= The definition is chosen in ode to have a nomalization condition: i RI i =. This appoach was, fo instance, used in combinations of top-quak mass measuements at Tevaton and at LHC [8, 9]. This choice is questionable, as obseved in ef. []. In fact, imagine we have thee measuement, say A, B and B. The RI of measuement A changes whethe the thee measuements ae combined all togethe o if B and B ae fist combined into B, and then A and the patial combination B ae combined togethe. Ref. [] poposes altenatives to RI based on the Fishe infomation, which is defined as: ln L J i j = θ i ln L θ j, () the aveage being pefomed ove all possible measuements, hence J does not depend on a specific measuement, but only on the fom of the likelihood function and on the paametes choice. Fishe infomation sets a lowe bound to the vaiance of an unbiased estimato [, ]: and fo a single paamete, the Fishe infomation is given by: σ ˆθ i J ii, () J = J = u T V u = σ ˆθ. () The altenative quantities to RI poposed in ef. [] ae the intinsic infomation weight (IIW), defined as: IIW i = /σ i = /σ i /σ ˆθ J and the maginal infomation weight (MIW), defined as follows: MIW i = J J = J J {,,n} {i} J (5), (6) i.e.: the elative diffeence of the Fishe infomation of the combination and the Fishe infomation of the combination excluding the i th measuement. Both IIW and MIW don t obey a nomalization condition. Fo IIW the quantity IIW co can be defined such that: n IIW i + IIW co =. (7) i= IIW co epesents the weight assigned to the coelation inteplay, not assignable to individual measuements, and is given by: IIW co = /σˆθ n i= /σ i /σ ˆθ = J n i= /σ i J. (8) Table summaizes the popeties of the diffeent poposed quantities. IIW and MIW ae epoted, instead of RI, in ecent papes about combination of LHC and Tevaton measuements elated to top-quak physics [ 5].

5 CONF Table. Popeties of the diffeent quantities poposed fo the BLUE combination Weight type i = Consistent with patial combination BLUE coefficient w i Relative impotance w i / n i= w i Intinsic Infomation Weight IIW i Maginal Infomation Weight MIW i Negative weights Negative weights ae always sign of a high-coelations egime. The maximum value of the atio σˆθ /σ (eq. (8)) as a function of ρ is obtained fo ρ = σ /σ. Fo ρ > σ /σ, an incease in coelation implies a decease of the uncetainty and a negative weight, and uncetainty becomes stongly dependent on ρ. Fo ρ = σ /σ, in paticula, the weight w becomes equal to zeo, as well as MIW =. But this doesn t imply that the measuement θ is not used in the combination. Figue, fom ef. [], shows how the BLUE coefficient w and the atio of uncetainties σ ˆθ /σ vay as a function of the coelation ρ fo diffeent fixed values of the atio σ /σ (note that the figue uses a diffeent notation with espect to this text, as specified in the caption). Figue. BLUE coefficient fo the second measuement w (left; λ B in the oiginal figue notation) and combined BLUE vaiance σ ˆθ (ight; σ in the oiginal figue notation) as a function of the coelation ρ between Ŷ two measuements and fo vaious fixed values of the atio σ /σ (σ B /σ A in the oiginal figue notation). The figue is fom ef. []. When the coelation coefficient ρ is not well know, assuming ρ = is not always a consevative choice. The assumption ρ = yields the lagest possible uncetainty only if the uncoelated contibutions to the total uncetainty dominate. ρ should be accuately detemined in case of negative weights in ode to avoid the isk of undeestimating uncetainties. Assume that the two measuements A and B have total uncetainties given by the sum in quadatue of uncoelated contibutions, σ A (unc) and σ B (unc) espectively, and coelated contibutions, σ A (co) and σ B (co) espectively, whose coelation coefficient is ρ(co). Figue shows the most consevative value of the coelation coefficient ρ(co), which is equal to only fo σ B (co)/σ A (co)< (σ A /σ A (co)).

6 EPJ Web of Confeences Figue. The most consevative value of an unknown coelation ρ(co) between σ A (co) and σ A (co) as a function of σ A (co)/σ A, fo seveal values of σ B (co)/σ A (co). The figue is fom ef. []. 5 Bias with the BLUE method The BLUE method povides an unbiased estimate only if uncetainties and thei coelation ae known exactly. This is not always a ealistic scenaio, since estimates of uncetainties and thei coelation, and not thei tue values, ae available in most of the cases. Moeove, the uncetainty estimates may depend on the assumed cental value. One example of such a case in which the BLUE method povides a bias is the combination of two Poissonian estimates, each of which has an uncetainty estimate that depends on the cental value though a squae-oot elation: ˆn = ˆn ± ˆn, ˆn = ˆn ± ˆn. (9) () A maximum-likelihood estimate, using Poissonian distibutions, would poduce the following unbiased combined estimate: ˆn = (ˆn + ˆn ± ˆn ) + ˆn. () The BLUE combination, instead, gives weights popotional to /ˆn i, which coespond to a hamonic aveage: ˆn = ˆn ˆn ˆn ˆn ±. () ˆn + ˆn ˆn + ˆn Equation (), compaed to eq. (), exhibits a bias because, due to the dependence of uncetainties on the measued values, downwad measuement fluctuations achieve lage weights pulling down the combination, while upwad fluctuations poduce a smalle opposite effect. In many cases, elative uncetainty estimates ae available, like fo uncetainty contibutions due to luminosity, efficiencies, etc. When pefoming a combination, the best estimate of the cental value impoves the individual measuements, hence one may ague whethe the assumed uncetainties should change accodingly, e-evaluating them using the cental-value estimate fom the BLUE combination [6, 7]. The method can be applied iteatively until the combination conveges to a stable

7 CONF value. Namely: ˆσ () i = ˆσ i (ˆθ i ), ˆρ () i j = ˆρ(ˆθ i, ˆθ j ) ˆσ () i = ˆσ i (ˆθ () ), ˆρ () i j = ˆρ(ˆθ (), ˆθ () ) ˆσ () i = ˆσ i (ˆθ () ), ˆρ () i j = ˆρ(ˆθ (), ˆθ () ) BLUE ˆθ () () BLUE ˆθ () () BLUE ˆθ () (5) In pactice, convegence only needs few iteations in most of the cases. Let s assume that a contibution to the total uncetainty is known as elative uncetainty. In this case, uncetainties can be witten as the sum in quadatue of a contibution that does not depend on the cental value and anothe contibution that is popotional to the cental value : θ = ˆθ ± σ ± ˆθ, θ = ˆθ ± σ ± ˆθ. The covaiance matix estimate can be witten as follows: ( σ ˆV = + ( ˆθ ) ρ σ σ + ρ ˆθ ˆθ ρ σ σ + ρ ˆθ ˆθ σ + ( ˆθ ) ) (6) (7) (8) whee ρ and ρ epesent the coelation coefficients of the uncetainty contibutions that do not depend on the cental value and of the ones that ae popotional to the cental value, espectively. A special case is when uncetainties ae fully popotional to the cental values, i.e.: σ = and σ =. In that case, the iteative BLUE method conveges in two iteations to the following cental value: ˆθ = ( ρ )ˆθ + ( ρ )ˆθ q ρ +, (9) which is simila to eq. (7) but elative uncetainties ae used in place of the absolute ones. A numeical Monte Calo study [7] shows that in most of the cases the bias in the combination can be mitigated by applying the iteative pocedue. Assuming fo simplicity, and without loss of geneality, a tue value θ =, uncetainties and thei coelations ae chosen andomly by spanning a wide anges of values in the boundaies σ, σ,,, < and < ρ, ρ <. Fo each set of andomly-extacted uncetainties and coelation values, 5 andom values of ˆθ and ˆθ ae geneated using the pope two-dimensional coelated Gaussian distibution. The BLUE method is then applied in its standad fomulation and iteatively. Pulls distibutions allow to study the bias and the coectness of uncetainty estimates. Bias is in geneal mitigated with the iteative BLUE method while the standad BLUE method tends to undeestimate the cental value, as shown in figue. The iteative BLUE application may povide oveestimates in few of the cases with lage uncetainties. Moe detailed compaisons of the two method ae available in ef. [7]. In paticula, the combination uncetainty may be oveestimated (and in fewe cases undeestimated) in both standad and iteative BLUE, but the iteative BLUE estimates tend to be moe consevative, as visible in fig.. A dedicated toy Monte Calo may be useful in case of lage individual uncetainties in ode to achieve a pope combined uncetainty estimate. This was not the case fo the combination of Poissonian measuements fom eq. (9), (), whee the dependence on cental value was not linea, but it is a convenient assumption in many ealistic cases.

8 EPJ Web of Confeences Standad BLUE aveage, ρ and ρ Iteative BLUE aveage, ρ and ρ <. <. <. < <x> <x it > Standad BLUE aveage, ρ < o ρ < Iteative BLUE aveage, ρ < o ρ < 5 <. <. <. < <x> Bias diffeence, ρ and ρ <. < <x it > Bias diffeence, ρ < o ρ < <. < <x-> - <x it -> <x-> - <x it -> Figue. Distibution of the aveage value of the standad (top and middle, left) and iteative (top and middle, ight) BLUE estimates fo diffeent limits on, and fo ρ and ρ (top) and fo ρ < and ρ < (middle). Bottom plots show the diffeence of measued absolute value of the bias fo the standad and iteative BLUE estimates fo diffeent limits on, and fo ρ and ρ (left) and fo ρ < and ρ < (ight). Positive values indicate a smalle bias in the iteative method compaed to the standad method. Undeestimate of the uncetainty (negative values) occu in fewe cases with the iteative BLUE method than with the standad BLUE method. The figue is fom ef. [7].

9 CONF Standad BLUE pull st. dev. Iteative BLUE pull st. dev. <. <. <. < σ(p) σ(p ) it Figue. Distibution of the aveage value of the pull standad deviation fo the standad (left) and iteative (ight) BLUE estimates fo diffeent limits on,. The figue is fom ef. [7]. 6 Conclusions Combination of measuements is a cucial task to impove the pecision on the knowledge of impotant paametes. Combining individual likelihood functions befoe applying any statistical method is the most igoous and pecise appoach, but often the full likelihood function of individual measuements is not available. When uncetainties and thei coelations ae available, the BLUE method is a simple and poweful tool to combine measuements. But BLUE may have counteintuitive behavios, like negative weights o uncetainties that may decease fo inceasing coelation. Coelation needs to be accuately evaluated in those cases, and an assumption of a % coelation is not necessay a consevative choice. In case elative uncetainty contibutions ae known, o moe in geneal when uncetainty estimates depend on the cental values, the BLUE method may exhibit a bias that can be mitigated, in most of the cases, with an iteative application of the method whee uncetainties ae escaled at each iteation to the combined cental value. Refeences [] CDF Collaboation, Phys. Rev. Lett, 8 () [] CDF Collaboation, Phys. Rev. Lett, 85 () [] D Collaboation, Phys. Lett B 76, 656 () [] G. Cowan et al., Eu. Phys. J. C 7, 55 () [5] J. Aitken, Poc. Roy. Soc. Edinbugh 55, (95) [6] L. Lyons et al. Nucl. Inst. Meth. A 7, (988) [7] H. Geenlee, Femilab Wokshop on Confidence Limits () [8] CDF and D Collaboations, axiv:7.555 () [9] ATLAS and CMS Collaboations, ATLAS-CONF--95, CMS-PAS-TOP-- () [] A. Valassi, R. Chieici, Eu. Phys. J. C 7 77 () [] H. Camé, Mathematical Methods of Statistics (Pinceton Univ. Pess, Pinceton, NJ, 96) [] C. R. Rao, Bulletin of the Calcutta Mathematical Society, 7, 8 89 (95)

10 EPJ Web of Confeences [] ATLAS and CMS Collaboations, ATLAS-CONF--, CMS-PAS-TOP--5 () [] ATLAS and CMS Collaboations, ATLAS-CONF--, CMS-PAS-TOP--6 () [5] ATLAS, CMS, CDF and D Collaboations, ATLAS-CONF--8, CDF-NOTE-7, CMS-PAS-TOP--, D-NOTE-66, FERMILAB-TM-58, axiv:.7 () [6] L. Lyons, A. J. Matin, D. H. Saxon, Phys. Rev. D (99) [7] L. Lista, Nucl. Inst. Meth. A (); coig. ibid. A (5)