How To Calculate A Radom Umber From A Probability Fuctio

Iteral Report SUF PFY/96 Stockholm, December 996 st revisio, 3 October 998 last modificatio September 7 Had-book o STATISTICAL DISTRIBUTIONS for experimetalists by Christia Walck Particle Physics Group Fysikum Uiversity of Stockholm (e-mail: walck@physto.se)

Cotets Itroductio. Radom Number Geeratio.............................. Probability Desity Fuctios 3. Itroductio........................................ 3. Momets......................................... 3.. Errors of Momets................................ 4.3 Characteristic Fuctio................................. 4.4 Probability Geeratig Fuctio............................ 5.5 Cumulats......................................... 6.6 Radom Number Geeratio.............................. 7.6. Cumulative Techique.............................. 7.6. Accept-Reject techique............................. 7.6.3 Compositio Techiques............................. 8.7 Multivariate Distributios................................ 9.7. Multivariate Momets.............................. 9.7. Errors of Bivariate Momets.......................... 9.7.3 Joit Characteristic Fuctio...........................7.4 Radom Number Geeratio.......................... 3 Beroulli Distributio 3. Itroductio........................................ 3. Relatio to Other Distributios............................. 4 Beta distributio 3 4. Itroductio........................................ 3 4. Derivatio of the Beta Distributio........................... 3 4.3 Characteristic Fuctio................................. 4 4.4 Momets......................................... 4 4.5 Probability Cotet................................... 4 4.6 Radom Number Geeratio.............................. 5 5 Biomial Distributio 6 5. Itroductio........................................ 6 5. Momets......................................... 6 5.3 Probability Geeratig Fuctio............................ 6 5.4 Cumulative Fuctio................................... 7 5.5 Radom Number Geeratio.............................. 7 5.6 Estimatio of Parameters................................ 7 5.7 Probability Cotet................................... 8 6 Biormal Distributio 6. Itroductio........................................ 6. Coditioal Probability Desity............................. 6.3 Characteristic Fuctio................................. 6.4 Momets......................................... 6.5 Box-Muller Trasformatio............................... 6.6 Probability Cotet................................... 3 i

6.7 Radom Number Geeratio.............................. 4 7 Cauchy Distributio 6 7. Itroductio........................................ 6 7. Momets......................................... 6 7.3 Normalizatio....................................... 7 7.4 Characteristic Fuctio................................. 7 7.5 Locatio ad Scale Parameters............................. 7 7.6 Breit-Wiger Distributio................................ 8 7.7 Compariso to Other Distributios........................... 8 7.8 Trucatio........................................ 9 7.9 Sum ad Average of Cauchy Variables......................... 3 7. Estimatio of the Media................................ 3 7. Estimatio of the HWHM................................ 3 7. Radom Number Geeratio.............................. 33 7.3 Physical Picture..................................... 34 7.4 Ratio Betwee Two Stadard Normal Variables.................... 35 8 Chi-square Distributio 36 8. Itroductio........................................ 36 8. Momets......................................... 36 8.3 Characteristic Fuctio................................. 37 8.4 Cumulative Fuctio................................... 38 8.5 Origi of the Chi-square Distributio.......................... 38 8.6 Approximatios...................................... 39 8.7 Radom Number Geeratio.............................. 39 8.8 Cofidece Itervals for the Variace.......................... 4 8.9 Hypothesis Testig.................................... 4 8. Probability Cotet................................... 4 8. Eve Number of Degrees of Freedom.......................... 4 8. Odd Number of Degrees of Freedom.......................... 4 8.3 Fial Algorithm...................................... 43 8.4 Chi Distributio..................................... 43 9 Compoud Poisso Distributio 45 9. Itroductio........................................ 45 9. Brachig Process.................................... 45 9.3 Momets......................................... 45 9.4 Probability Geeratig Fuctio............................ 45 9.5 Radom Number Geeratio.............................. 46 Double-Expoetial Distributio 47. Itroductio........................................ 47. Momets......................................... 47.3 Characteristic Fuctio................................. 47.4 Cumulative Fuctio................................... 48.5 Radom Number Geeratio.............................. 48 ii

Doubly No-Cetral F -Distributio 49. Itroductio........................................ 49. Momets......................................... 49.3 Cumulative Distributio................................. 5.4 Radom Number Geeratio.............................. 5 Doubly No-Cetral t-distributio 5. Itroductio........................................ 5. Momets......................................... 5.3 Cumulative Distributio................................. 5.4 Radom Number Geeratio.............................. 5 3 Error Fuctio 53 3. Itroductio........................................ 53 3. Probability Desity Fuctio.............................. 53 4 Expoetial Distributio 54 4. Itroductio........................................ 54 4. Cumulative Fuctio................................... 54 4.3 Momets......................................... 54 4.4 Characteristic Fuctio................................. 54 4.5 Radom Number Geeratio.............................. 55 4.5. Method by vo Neuma............................ 55 4.5. Method by Marsaglia.............................. 55 4.5.3 Method by Ahres................................ 56 5 Extreme Value Distributio 57 5. Itroductio........................................ 57 5. Cumulative Distributio................................. 58 5.3 Characteristic Fuctio................................. 58 5.4 Momets......................................... 58 5.5 Radom Number Geeratio.............................. 6 6 F-distributio 6 6. Itroductio........................................ 6 6. Relatios to Other Distributios............................ 6 6.3 /F............................................ 6 6.4 Characteristic Fuctio................................. 6 6.5 Momets......................................... 6 6.6 F-ratio........................................... 63 6.7 Variace Ratio...................................... 64 6.8 Aalysis of Variace................................... 65 6.9 Calculatio of Probability Cotet........................... 65 6.9. The Icomplete Beta fuctio.......................... 66 6.9. Fial Formulæ.................................. 67 6. Radom Number Geeratio.............................. 68 iii

7 Gamma Distributio 69 7. Itroductio........................................ 69 7. Derivatio of the Gamma Distributio......................... 69 7.3 Momets......................................... 7 7.4 Characteristic Fuctio................................. 7 7.5 Probability Cotet................................... 7 7.6 Radom Number Geeratio.............................. 7 7.6. Erlagia distributio.............................. 7 7.6. Geeral case................................... 7 7.6.3 Asymptotic Approximatio........................... 7 8 Geeralized Gamma Distributio 73 8. Itroductio........................................ 73 8. Cumulative Fuctio................................... 73 8.3 Momets......................................... 74 8.4 Relatio to Other Distributios............................. 74 9 Geometric Distributio 75 9. Itroductio........................................ 75 9. Momets......................................... 75 9.3 Probability Geeratig Fuctio............................ 75 9.4 Radom Number Geeratio.............................. 75 Hyperexpoetial Distributio 77. Itroductio........................................ 77. Momets......................................... 77.3 Characteristic Fuctio................................. 77.4 Radom Number Geeratio.............................. 78 Hypergeometric Distributio 79. Itroductio........................................ 79. Probability Geeratig Fuctio............................ 79.3 Momets......................................... 79.4 Radom Number Geeratio.............................. 8 Logarithmic Distributio 8. Itroductio........................................ 8. Momets......................................... 8.3 Probability Geeratig Fuctio............................ 8.4 Radom Number Geeratio.............................. 8 3 Logistic Distributio 83 3. Itroductio........................................ 83 3. Cumulative Distributio................................. 83 3.3 Characteristic Fuctio................................. 84 3.4 Momets......................................... 84 3.5 Radom umbers..................................... 85 iv

4 Log-ormal Distributio 86 4. Itroductio........................................ 86 4. Momets......................................... 86 4.3 Cumulative Distributio................................. 87 4.4 Radom Number Geeratio.............................. 87 5 Maxwell Distributio 88 5. Itroductio........................................ 88 5. Momets......................................... 88 5.3 Cumulative Distributio................................. 89 5.4 Kietic Theory...................................... 89 5.5 Radom Number Geeratio.............................. 9 6 Moyal Distributio 9 6. Itroductio........................................ 9 6. Normalizatio....................................... 9 6.3 Characteristic Fuctio................................. 9 6.4 Momets......................................... 9 6.5 Cumulative Distributio................................. 93 6.6 Radom Number Geeratio.............................. 93 7 Multiomial Distributio 95 7. Itroductio........................................ 95 7. Histogram......................................... 95 7.3 Momets......................................... 95 7.4 Probability Geeratig Fuctio............................ 96 7.5 Radom Number Geeratio.............................. 96 7.6 Sigificace Levels.................................... 96 7.7 Equal Group Probabilities................................ 96 8 Multiormal Distributio 99 8. Itroductio........................................ 99 8. Coditioal Probability Desity............................. 99 8.3 Probability Cotet................................... 99 8.4 Radom Number Geeratio.............................. 9 Negative Biomial Distributio 9. Itroductio........................................ 9. Momets......................................... 9.3 Probability Geeratig Fuctio............................ 3 9.4 Relatios to Other Distributios............................ 3 9.4. Poisso Distributio............................... 3 9.4. Gamma Distributio............................... 4 9.4.3 Logarithmic Distributio............................ 5 9.4.4 Brachig Process................................ 5 9.4.5 Poisso ad Gamma Distributios....................... 6 9.5 Radom Number Geeratio.............................. 7 v

3 No-cetral Beta-distributio 8 3. Itroductio........................................ 8 3. Derivatio of distributio................................ 8 3.3 Momets......................................... 9 3.4 Cumulative distributio................................. 9 3.5 Radom Number Geeratio.............................. 9 3 No-cetral Chi-square Distributio 3. Itroductio........................................ 3. Characteristic Fuctio................................. 3.3 Momets......................................... 3.4 Cumulative Distributio................................. 3.5 Approximatios...................................... 3.6 Radom Number Geeratio.............................. 3 No-cetral F -Distributio 3 3. Itroductio........................................ 3 3. Momets......................................... 4 3.3 Cumulative Distributio................................. 4 3.4 Approximatios...................................... 4 3.5 Radom Number Geeratio.............................. 5 33 No-cetral t-distributio 6 33. Itroductio........................................ 6 33. Derivatio of distributio................................ 6 33.3 Momets......................................... 7 33.4 Cumulative Distributio................................. 7 33.5 Approximatio...................................... 8 33.6 Radom Number Geeratio.............................. 8 34 Normal Distributio 9 34. Itroductio........................................ 9 34. Momets......................................... 9 34.3 Cumulative Fuctio................................... 34.4 Characteristic Fuctio................................. 34.5 Additio Theorem.................................... 34.6 Idepedece of x ad s................................ 34.7 Probability Cotet................................... 34.8 Radom Number Geeratio.............................. 4 34.8. Cetral Limit Theory Approach........................ 4 34.8. Exact Trasformatio.............................. 4 34.8.3 Polar Method................................... 4 34.8.4 Trapezoidal Method............................... 5 34.8.5 Ceter-tail method................................ 6 34.8.6 Compositio-rejectio Methods......................... 6 34.8.7 Method by Marsaglia.............................. 7 34.8.8 Histogram Techique............................... 8 34.8.9 Ratio of Uiform Deviates............................ 9 34.8. Compariso of radom umber geerators................... 3 vi

34.9 Tests o Parameters of a Normal Distributio..................... 3 35 Pareto Distributio 33 35. Itroductio........................................ 33 35. Cumulative Distributio................................. 33 35.3 Momets......................................... 33 35.4 Radom Numbers.................................... 33 36 Poisso Distributio 34 36. Itroductio........................................ 34 36. Momets......................................... 34 36.3 Probability Geeratig Fuctio............................ 35 36.4 Cumulative Distributio................................. 35 36.5 Additio Theorem.................................... 35 36.6 Derivatio of the Poisso Distributio......................... 36 36.7 Histogram......................................... 36 36.8 Radom Number Geeratio.............................. 37 37 Rayleigh Distributio 38 37. Itroductio........................................ 38 37. Momets......................................... 38 37.3 Cumulative Distributio................................. 39 37.4 Two-dimesioal Kietic Theory............................ 39 37.5 Radom Number Geeratio.............................. 4 38 Studet s t-distributio 4 38. Itroductio........................................ 4 38. History.......................................... 4 38.3 Momets......................................... 4 38.4 Cumulative Fuctio................................... 43 38.5 Relatios to Other Distributios............................ 43 38.6 t-ratio........................................... 44 38.7 Oe Normal Sample................................... 44 38.8 Two Normal Samples................................... 45 38.9 Paired Data........................................ 45 38. Cofidece Levels.................................... 45 38. Testig Hypotheses................................... 46 38. Calculatio of Probability Cotet........................... 46 38.. Eve umber of degrees of freedom...................... 47 38.. Odd umber of degrees of freedom....................... 48 38..3 Fial algorithm................................. 49 38.3 Radom Number Geeratio.............................. 49 39 Triagular Distributio 5 39. Itroductio........................................ 5 39. Momets......................................... 5 39.3 Radom Number Geeratio.............................. 5 vii

4 Uiform Distributio 5 4. Itroductio........................................ 5 4. Momets......................................... 5 4.3 Radom Number Geeratio.............................. 5 4 Weibull Distributio 5 4. Itroductio........................................ 5 4. Cumulative Distributio................................. 53 4.3 Momets......................................... 53 4.4 Radom Number Geeratio.............................. 53 4 Appedix A: The Gamma ad Beta Fuctios 54 4. Itroductio........................................ 54 4. The Gamma Fuctio.................................. 54 4.. Numerical Calculatio.............................. 55 4.. Formulæ...................................... 56 4.3 Digamma Fuctio.................................... 56 4.4 Polygamma Fuctio................................... 58 4.5 The Icomplete Gamma Fuctio............................ 59 4.5. Numerical Calculatio.............................. 59 4.5. Formulæ...................................... 6 4.5.3 Special Cases................................... 6 4.6 The Beta Fuctio.................................... 6 4.7 The Icomplete Beta Fuctio............................. 6 4.7. Numerical Calculatio.............................. 6 4.7. Approximatio.................................. 6 4.8 Relatios to Probability Desity Fuctios...................... 63 4.8. The Beta Distributio.............................. 63 4.8. The Biomial Distributio............................ 63 4.8.3 The Chi-squared Distributio.......................... 63 4.8.4 The F -distributio................................ 64 4.8.5 The Gamma Distributio............................ 64 4.8.6 The Negative Biomial Distributio...................... 64 4.8.7 The Normal Distributio............................ 64 4.8.8 The Poisso Distributio............................ 65 4.8.9 Studet s t-distributio............................. 65 4.8. Summary..................................... 66 43 Appedix B: Hypergeometric Fuctios 67 43. Itroductio........................................ 67 43. Hypergeometric Fuctio................................ 67 43.3 Cofluet Hypergeometric Fuctio.......................... 68 Mathematical Costats.................................. 8 Errata et Addeda....................................... 8 viii

Refereces............................................... 85 Idex.................................................... 88 List of Tables Percetage poits of the chi-square distributio.................... 7 Extreme cofidece levels for the chi-square distributio............... 7 3 Extreme cofidece levels for the chi-square distributio (as χ /d.f. values).... 73 4 Exact ad approximate values for the Beroulli umbers............... 74 5 Percetage poits of the F -distributio......................... 75 6 Probability cotet from z to z of Gauss distributio i %............. 76 7 Stadard ormal distributio z-values for a specific probability cotet....... 77 8 Percetage poits of the t-distributio......................... 78 9 Expressios for the Beta fuctio B(m, ) for iteger ad half-iteger argumets. 79 ix

x

Itroductio I experimetal work e.g. i physics oe ofte ecouters problems where a stadard statistical probability desity fuctio is applicable. It is ofte of great help to be able to hadle these i differet ways such as calculatig probability cotets or geeratig radom umbers. For these purposes there are excellet text-books i statistics e.g. the classical work of Maurice G. Kedall ad Ala Stuart [,] or more moder text-books as [3] ad others. Some books are particularly aimed at experimetal physics or eve specifically at particle physics [4,5,6,7,8]. Cocerig umerical methods a valuable refereces worth metioig is [9] which has bee surpassed by a ew editio []. Also had-books, especially [], has bee of great help throughout. However, whe it comes to actual applicatios it ofte turs out to be hard to fid detailed explaatios i the literature ready for implemetatio. This work has bee collected over may years i parallel with actual experimetal work. I this way some material may be historical ad sometimes be aïve ad have somewhat clumsy solutios ot always made i the mathematically most striget may. We apologize for this but still hope that it will be of iterest ad help for people who is strugglig to fid methods to solve their statistical problems i makig real applicatios ad ot oly learig statistics as a course. Eve if oe has the skill ad may be able to fid solutios it seems worthwhile to have easy ad fast access to formulæ ready for applicatio. Similar books ad reports exist e.g. [,3] but we hope the preset work may compete i describig more distributios, beig more complete, ad icludig more explaatios o relatios give. The material could most probably have bee divided i a more logical way but we have chose to preset the distributios i alphabetic order. I this way it is more of a had-book tha a proper text-book. After the first release the report has bee modestly chaged. Mior chages to correct misprits is made wheever foud. I a few cases subsectios ad tables have bee added. These alteratios are described o page 8. I October 998 the first somewhat bigger revisio was made where i particular a lot of material o the o-cetral samplig distributios were added.. Radom Number Geeratio I moder computig Mote Carlo simulatios are of vital importace ad we give methods to achieve radom umbers from the distributios. A earlier report dealt etirely with these matters [4]. Not all text-books o statistics iclude iformatio o this subject which we fid extremely useful. Large simulatios are commo i particle physics as well as i other areas but ofte it is also useful to make small toy Mote Carlo programs to ivestigate ad study aalysis tools developed o ideal, but statistically soud, radom samples. A related ad importat field which we will oly metio briefly here, is how to get good basic geerators for achievig radom umbers uiformly distributed betwee zero ad oe. Those are the basis for all the methods described i order to get radom umbers

from specific distributios i this documet. For a review see e.g. [5]. From older methods ofte usig so called multiplicative cogruetial method or shiftgeerators G. Marsaglia et al [6] itroduced i 989 a ew uiversal geerator which became the ew stadard i may fields. We implemeted this i our experimets at CERN ad also made a package of routies for geeral use [7]. This method is still a very good choice but later alteratives, claimed to be eve better, have tured up. These are based o o the same type of lagged Fiboacci sequeces as is used i the uiversal geerator ad was origially proposed by the same authors [8]. A implemetatios of this method was proposed by F. James [5] ad this versio was further developed by M. Lüscher [9]. A similar package of routie as was prepared for the uiversal geerator has bee implemeted for this method [].

Probability Desity Fuctios. Itroductio Probability desity fuctios i oe, discrete or cotiuous, variable are deoted p(r) ad f(x), respectively. They are assumed to be properly ormalized such that p(r) = r ad f(x)dx = where the sum or the itegral are take over all relevat values for which the probability desity fuctio is defied. Statisticias ofte use the distributio fuctio or as physicists more ofte call it the cumulative fuctio which is defied as r P (r) = p(i) ad F (x) = x i= f(t)dt. Momets Algebraic momets of order r are defied as the expectatio value µ r = E(x r ) = k k r p(k) or x r f(x)dx Obviously µ = from the ormalizatio coditio ad µ is equal to the mea, sometimes called the expectatio value, of the distributio. Cetral momets of order r are defied as µ r = E((k E(k)) r ) or E((x E(x)) r ) of which the most commoly used is µ which is the variace of the distributio. Istead of usig the third ad fourth cetral momets oe ofte defies the coefficiets of skewess γ ad kurtosis γ by γ = µ 3 ad γ µ 3 = µ 4 µ 3 where the shift by 3 uits i γ assures that both measures are zero for a ormal distributio. Distributios with positive kurtosis are called leptokurtic, those with kurtosis aroud zero mesokurtic ad those with egative kurtosis platykurtic. Leptokurtic distributios are ormally more peaked tha the ormal distributio while platykurtic distributios are more flat topped. From greek kyrtosis = curvature from kyrt(ós) = curved, arched, roud, swellig, bulgig. Sometimes, especially i older literature, γ is called the coefficiet of excess. 3

.. Errors of Momets For a thorough presetatio of how to estimate errors o momets we refer to the classical books by M. G. Kedall ad A. Stuart [] (pp 8 45). Below oly a brief descriptio is give. For a sample with observatios x, x,..., x we defie the momet-statistics for the algebraic ad cetral momets m r ad m r as m r = x r ad m r = r= (x m ) r r= The otatio m r ad m r are thus used for the statistics (sample values) while we deote the true, populatio, values by µ r ad µ r. The mea value of the r:th ad the samplig covariace betwee the q:th ad r:th momet-statistic are give by. E(m r) = µ r Cov(m q, m r) = ( µ q+r µ qµ r) These formula are exact. Formulæ for momets about the mea are ot as simple sice the mea itself is subject to samplig fluctuatios. E(m r ) = µ r Cov(m q, m r ) = (µ q+r µ q µ r + rqµ µ r µ q rµ r µ q+ qµ r+ µ q ) to order / ad /, respectively. The covariace betwee a algebraic ad a cetral momet is give by to order /. Note especially that Cov(m r, m q) = (µ q+r µ q µ r rµ q+ µ r ) V (m r) = ( ) µ r µ r V (m r ) = ( ) µr µ r + r µ µ r rµ r µ r+ Cov(m, m r ) = (µ r+ rµ µ r ).3 Characteristic Fuctio For a distributio i a cotiuous variable x the Fourier trasform of the probability desity fuctio φ(t) = E(e ıxt ) = 4 e ıxt f(x)dx

is called the characteristic fuctio. It has the properties that φ() = ad φ(t) for all t. If the cumulative, distributio, fuctio F (x) is cotiuous everywhere ad df (x) = f(x)dx the we reverse the trasform such that f(x) = π φ(t)e ıxt dt The characteristic fuctio is related to the momets of the distributio by φ x (t) = E(e ıtx ) = = (ıt) E(x )! = = (ıt) µ! e.g. algebraic momets may be foud by µ r = ( r d φ(t) ı dt) r To fid cetral momets (about the mea µ) use t= φ x µ (t) = E ( e ıt(x µ)) = e ıtµ φ x (t) ad thus µ r = ( r d e ı dt) ıtµ φ(t) r t= A very useful property of the characteristic fuctio is that for idepedet variables x ad y φ x+y (t) = φ x (t) φ y (t) As a example regard the sum a i z i where the z i s are distributed accordig to ormal distributios with meas µ i ad variaces σ i. The the liear combiatio will also be distributed accordig to the ormal distributio with mea a i µ i ad variace a i σ i. To show that the characteristic fuctio i two variables factorizes is the best way to show idepedece betwee two variables. Remember that a vaishig correlatio coefficiet does ot imply idepedece while the reversed is true..4 Probability Geeratig Fuctio I the case of a distributio i a discrete variable r the characteristic fuctio is give by φ(t) = E(e ıtr ) = p(r)e ıtr I this case it is ofte coveiet to write z = e ıt ad defie the probability geeratig fuctio as G(z) = E(z r ) = p(r)z r 5

Derivatives of G(z) evaluated at z = are related to factorial momets of the distributio G () = d dz G(z) z= = E(r) G() = (ormalizatio) G () = d dz G(z) = E(r(r )) z= G 3 () = d3 dz G(z) 3 = E(r(r )(r )) z= G k () = dk dz G(z) k = E(r(r )(r ) (r k + )) z= Lower order algebraic momets are the give by µ = G () µ = G () + G () µ 3 = G 3 () + 3G () + G () µ 4 = G 4 () + 6G 3 () + 7G () + G () while expressio for cetral momets become more complicated. A useful property of the probability geeratig fuctio is for a brachig process i steps where G(z) = G (G (... G (G (z))...)) with G k (z) the probability geeratig fuctio for the distributio i the k:th step. As a example see sectio 9.4.4 o page 5..5 Cumulats Although ot much used i physics the cumulats, κ r, are of statistical iterest. Oe reaso for this is that they have some useful properties such as beig ivariat for a shift i scale (except the first cumulat which is equal to the mea ad is shifted alog with the scale). Multiplyig the x-scale by a costat a has the same effect as for algebraic momets amely to multiply κ r by a r. As the algebraic momet µ is the coefficiet of (ıt) /! i the expasio of φ(t) the cumulat κ is the coefficiet of (ıt) /! i the expasio of the logarithm of φ(t) (sometimes called the cumulat geeratig fuctio) i.e. (ıt) l φ(t) = κ =! ad thus κ r = ( r d l φ(t) ı dt) r t= Relatios betwee cumulats ad cetral momets for some lower orders are as follows 6

κ = µ κ = µ µ = κ κ 3 = µ 3 µ 3 = κ 3 κ 4 = µ 4 3µ µ 4 = κ 4 + 3κ κ 5 = µ 5 µ 3 µ µ 5 = κ 5 + κ 3 κ κ 6 = µ 6 5µ 4 µ µ 3 + 3µ 3 µ 6 = κ 6 + 5κ 4 κ + κ 3 + 5κ 3 κ 7 = µ 7 µ 5 µ 35µ 4 µ 3 + µ 3 µ µ 7 = κ 7 + κ 5 κ + 35κ 4 κ 3 + 5κ 3 κ κ 8 = µ 8 8µ 6 µ 56µ 5 µ 3 35µ 4+ µ 8 = κ 8 + 8κ 6 κ + 56κ 5 κ 3 + 35κ 4+ +4µ 4 µ + 56µ 3µ 63µ 4 +κ 4 κ + 8κ 3κ + 5κ 4.6 Radom Number Geeratio Whe geeratig radom umbers from differet distributio it is assumed that a good geerator for uiform pseudoradom umbers betwee zero ad oe exist (ormally the ed-poits are excluded)..6. Cumulative Techique The most direct techique to obtai radom umbers from a cotiuous probability desity fuctio f(x) with a limited rage from x mi to x max is to solve for x i the equatio ξ = F (x) F (x mi) F (x max ) F (x mi ) where ξ is uiformly distributed betwee zero ad oe ad F (x) is the cumulative distributio (or as statisticias say the distributio fuctio). For a properly ormalized probability desity fuctio thus x = F (ξ) The techique is sometimes also of use i the discrete case if the cumulative sum may be expressed i aalytical form as e.g. for the geometric distributio. Also for geeral cases, discrete or cotiuous, e.g. from a arbitrary histogram the cumulative method is coveiet ad ofte faster tha more elaborate methods. I this case the task is to costruct a cumulative vector ad assig a radom umber accordig to the value of a uiform radom umber (iterpolatig withi bis i the cotiuous case)..6. Accept-Reject techique A useful techique is the acceptace-rejectio, or hit-miss, method where we choose f max to be greater tha or equal to f(x) i the etire iterval betwee x mi ad x max ad proceed as follows i Geerate a pair of uiform pseudoradom umbers ξ ad ξ. ii Determie x = x mi + ξ (x max x mi ). iii Determie y = f max ξ. iv If y f(x) > reject ad go to i else accept x as a pseudoradom umber from f(x). 7

The efficiecy of this method depeds o the average value of f(x)/f max over the iterval. If this value is close to oe the method is efficiet. O the other had, if this average is close to zero, the method is extremely iefficiet. If α is the fractio of the area f max (x max x mi ) covered by the fuctio the average umber of rejects i step iv is α ad uiform pseudoradom umbers are required o average. α The efficiecy of this method ca be icreased if we are able to choose a fuctio h(x), from which radom umbers are more easily obtaied, such that f(x) αh(x) = g(x) over the etire iterval uder cosideratio (where α is a costat). A radom sample from f(x) is obtaied by i Geerate i x a radom umber from h(x). ii Geerate a uiform radom umber ξ. iii If ξ f(x)/g(x) go back to i else accept x as a pseudoradom umber from f(x). Yet aother situatio is whe a fuctio g(x), from which fast geeratio may be obtaied, ca be iscribed i such a way that a big proportio (f) of the area uder the fuctio is covered (as a example see the trapezoidal method for the ormal distributio). The proceed as follows: i Geerate a uiform radom umber ξ. ii If ξ < f the geerate a radom umber from g(x). iii Else use the acceptace/rejectio techique for h(x) = f(x) g(x) (i subitervals if more efficiet)..6.3 Compositio Techiques If f(x) may be writte i the form f(x) = g z (x)dh(z) where we kow how to sample radom umbers from the p.d.f. g(x) ad the distributio fuctio H(z). A radom umber from f(x) is the obtaied by i Geerate two uiform radom umbers ξ ad ξ. ii Determie z = H (ξ ). iii Determie x = G z (ξ ) where G z is the distributio fuctio correspodig to the p.d.f. g z (x). For more detailed iformatio o the Compositio techique see [] or []. 8

A combiatio of the compositio ad the rejectio method has bee proposed by J. C. Butcher [3]. If f(x) ca be writte f(x) = α i f i (x)g i (x) i= where α i are positive costats, f i (x) p.d.f. s for which we kow how to sample a radom umber ad g i (x) are fuctios takig values betwee zero ad oe. The method is the as follows: i Geerate uiform radom umbers ξ ad ξ. ii Determie a iteger k from the discrete distributio p i = α i /(α + α +... + α ) usig ξ. iii Geerate a radom umber x from f k (x). iv Determie g k (x) ad if ξ > g k (x) the go to i. v Accept x as a radom umber from f(x)..7 Multivariate Distributios Joit probability desity fuctios i several variables are deoted by f(x, x,..., x ) ad p(r, r,..., r ) for cotiuous ad discrete variables, respectively. It is assumed that they are properly ormalized i.e. itegrated (or summed) over all variables the result is uity..7. Multivariate Momets The geeralizatio of algebraic ad cetral momets to multivariate distributios is straightforward. As a example we take a bivariate distributio f(x, y) i two cotiuous variables x ad y ad defie algebraic ad cetral bivariate momets of order k, l as µ kl E(x k y l ) = x k y l f(x, y)dxdy µ kl E((x µ x ) k (y µ y ) l ) = (x µ x ) k (y µ y ) l f(x, y)dxdy where µ x ad µ y are the mea values of x ad y. The covariace is a cetral bivariate momet of order, i.e. Cov(x, y) = µ. Similarly oe easily defies multivariate momets for distributio i discrete variables..7. Errors of Bivariate Momets Algebraic (m rs) ad cetral (m rs ) bivariate momets are defied by: m rs = x r i yi s ad m rs = i= (x i m ) r (y i m ) s i= Whe there is a risk of ambiguity we write m r,s istead of m rs. 9

The otatios m rs ad m rs are used for the statistics (sample values) while we write µ rs ad µ rs for the populatio values. The errors of bivariate momets are give by Cov(m rs, m uv) = (µ r+u,s+v µ rsµ uv) especially Cov(m rs, m uv ) = (µ r+u,s+v µ rs µ uv + ruµ µ r,s µ u,v + svµ µ r,s µ u,v +rvµ µ r,s µ u,v + suµ µ r,s µ u,v uµ r+,s µ u,v vµ r,s+ µ u,v rµ r,s µ u+,v sµ r,s µ u,v+ ) V (m rs) = (µ r,s µ rs) V (m rs ) = (µ r,s µ rs + r µ µ r,s + s µ µ r,s +rsµ µ r,s µ r,s rµ r+,s µ r,s sµ r,s+ µ r,s ) For the covariace (m ) we get by error propagatio V (m ) = (µ µ ) Cov(m, m ) = µ Cov(m, m ) = (µ 3 µ µ ) For the correlatio coefficiet (deoted by ρ = µ / µ µ for the populatio value ad by r for the sample value) we get V (r) = ρ { µ + [ µ4 + µ 4 + µ ] [ µ3 + µ ]} 3 µ 4 µ µ µ µ µ µ µ Beware, however, that the samplig distributio of r teds to ormality very slowly..7.3 Joit Characteristic Fuctio The joit characteristic fuctio is defied by φ(t, t,..., t ) = E(e ıt x +ıt x +...t x ) = =... e ıt x +ıt x +...+ıt x f(x, x,..., x )dx dx... dx From this fuctio multivariate momets may be obtaied e.g. for a bivariate distributio algebraic bivariate momets are give by µ rs = E(x r x s ) = r+s φ(t, t ) (ıt ) r (ıt ) s t =t =

.7.4 Radom Number Geeratio Radom samplig from a may dimesioal distributio with a joit probability desity fuctio f(x, x,..., x ) ca be made by the followig method: Defie the margial distributios g m (x, x,..., x m ) = f(x,..., x )dx m+ dx m+...dx = g m+ (x,..., x m+ )dx m+ Cosider the coditioal desity fuctio h m give by h m (x m x, x,...x m ) g m (x, x,..., x m )/g m (x, x,..., x m ) We see that g = f ad that h m (x m x, x,..., x m )dx m = from the defiitios. Thus h m is the coditioal distributio i x m give fixed values for x, x,..., x m. We ca ow factorize f as f(x, x,..., x ) = h (x )h (x x )... h (x x, x,..., x ) We sample values for x, x,..., x from the joit probability desity fuctio f by: Geerate a value for x from h (x ). Use x ad sample x from h (x x ). Proceed step by step ad use previously sampled values for x, x,..., x m to obtai a value for x m+ from h m+ (x m+ x, x,..., x m ). Cotiue util all x i :s have bee sampled. If all x i :s are idepedet the coditioal desities will equal the margial desities ad the variables ca be sampled i ay order.

3 Beroulli Distributio 3. Itroductio The Beroulli distributio, amed after the swiss mathematicia Jacques Beroulli (654 75), describes a probabilistic experimet where a trial has two possible outcomes, a success or a failure. The parameter p is the probability for a success i a sigle trial, the probability for a failure thus beig p (ofte deoted by q). Both p ad q is limited to the iterval from zero to oe. The distributio has the simple form p(r; p) = { p = q if r = (failure) p if r = (success) ad zero elsewhere. The work of J. Beroulli, which costitutes a foudatio of probability theory, was published posthumously i Ars Cojectadi (73) [4]. The probability geeratig fuctio is G(z) = q +pz ad the distributio fuctio give by P () = q ad P () =. A radom umbers are easily obtaied by usig a uiform radom umber variate ξ ad puttig r = (success) if ξ p ad r = else (failure). 3. Relatio to Other Distributios From the Beroulli distributio we may deduce several probability desity fuctios described i this documet all of which are based o series of idepedet Beroulli trials: Biomial distributio: expresses the probability for r successes i a experimet with trials ( r ). Geometric distributio: expresses the probability of havig to wait exactly r trials before the first successful evet (r ). Negative Biomial distributio: expresses the probability of havig to wait exactly r trials util k successes have occurred (r k). This form is sometimes referred to as the Pascal distributio. Sometimes this distributio is expressed as the umber of failures occurrig while waitig for k successes ( ).

4 Beta distributio 4. Itroductio The Beta distributio is give by f(x; p, q) = B(p, q) xp ( x) q where the parameters p ad q are positive real quatities ad the variable x satisfies x. The quatity B(p, q) is the Beta fuctio defied i terms of the more commo Gamma fuctio as B(p, q) = Γ(p)Γ(q) Γ(p + q) For p = q = the Beta distributio simply becomes a uiform distributio betwee zero ad oe. For p = ad q = or vise versa we get triagular shaped distributios, f(x) = x ad f(x) = x. For p = q = we obtai a distributio of parabolic shape, f(x) = 6x( x). More geerally, if p ad q both are greater tha oe the distributio has a uique mode at x = (p )/(p + q ) ad is zero at the ed-poits. If p ad/or q is less tha oe f() ad/or f() ad the distributio is said to be J-shaped. I figure below we show the Beta distributio for two cases: p = q = ad p = 6, q = 3. Figure : Examples of Beta distributios 4. Derivatio of the Beta Distributio If y m ad y are two idepedet variables distributed accordig to the chi-squared distributio with m ad degrees of freedom, respectively, the the ratio y m /(y m + y ) follows a Beta distributio with parameters p = m ad q =. 3

To show this we make a chage of variables to x = y m /(y m + y ) ad y = y m + y which implies that y m = xy ad y = y( x). We obtai f(x, y) = = = y m x y x y y y m y y y ( x ym x Γ ( ) m+ Γ ( m ) Γ ( f(y m, y ) = ) m e ym Γ ( ) m ( y )x m ( x) ) e y ) Γ ( ( ) m y + e y Γ ( ) m+ = which we recogize as a product of a Beta distributio i the variable x ad a chi-squared distributio with m + degrees of freedom i the variable y (as expected for the sum of two idepedet chi-square variables). 4.3 Characteristic Fuctio The characteristic fuctio of the Beta distributio may be expressed i terms of the cofluet hypergeometric fuctio (see sectio 43.3) as 4.4 Momets φ(t) = M(p, p + q; ıt) The expectatio value, variace, third ad fourth cetral momet are give by E(x) = V (x) = µ 3 = µ 4 = p p + q pq (p + q) (p + q + ) pq(q p) (p + q) 3 (p + q + )(p + q + ) 3pq((p + q) + pq(p + q 6)) (p + q) 4 (p + q + )(p + q + )(p + q + 3) More geerally algebraic momets are give i terms of the Beta fuctio by µ k = B(p + k, q) B(p, q) 4.5 Probability Cotet I order to fid the probability cotet for a Beta distributio we form the cumulative distributio x F (x) = t p ( t) q dt = B x(p, q) B(p, q) B(p, q) = I x(p, q) 4

where both B x ad I x seems to be called the icomplete Beta fuctio i the literature. The icomplete Beta fuctio I x is coected to the biomial distributio for iteger values of a by ( ) a I x (a, b) = I x (b, a) = ( x) a+b ( a + b x i x or expressed i the opposite directio s=a i= ( ) p s ( p) s = I p (a, a + ) s Also to the egative biomial distributio there is a coectio by the relatio ( ) + s p q s = I q (a, ) s=a s The icomplete Beta fuctio is also coected to the probability cotet of Studet s t-distributio ad the F -distributio. See further sectio 4.7 for more iformatio o I x. 4.6 Radom Number Geeratio I order to obtai radom umbers from a Beta distributio we first sigle out a few special cases. For p = ad/or q = we may easily solve the equatio F (x) = ξ where F (x) is the cumulative fuctio ad ξ a uiform radom umber betwee zero ad oe. I these cases p = x = ξ /q q = x = ξ /p For p ad q half-itegers we may use the relatio to the chi-square distributio by formig the ratio y m y m + y with y m ad y two idepedet radom umbers from chi-square distributios with m = p ad = q degrees of freedom, respectively. Yet aother way of obtaiig radom umbers from a Beta distributio valid whe p ad q are both itegers is to take the l:th out of k ( l k) idepedet uiform radom umbers betwee zero ad oe (sorted i ascedig order). Doig this we obtai a Beta distributio with parameters p = l ad q = k + l. Coversely, if we wat to geerate radom umbers from a Beta distributio with iteger parameters p ad q we could use this techique with l = p ad k = p+q. This last techique implies that for low iteger values of p ad q simple code may be used, e.g. for p = ad q = we may simply take max(ξ, ξ ) i.e. the maximum of two uiform radom umbers. ) i 5

5 Biomial Distributio 5. Itroductio The Biomial distributio is give by p(r; N, p) = ( ) N p r ( p) N r r where the variable r with r N ad the parameter N (N > ) are itegers ad the parameter p ( p ) is a real quatity. The distributio describes the probability of exactly r successes i N trials if the probability of a success i a sigle trial is p (we sometimes also use q = p, the probability for a failure, for coveiece). It was first preseted by Jacques Beroulli i a work which was posthumously published [4]. 5. Momets The expectatio value, variace, third ad fourth momet are give by E(r) = Np V (r) = Np( p) = Npq µ 3 = Np( p)( p) = Npq(q p) µ 4 = Np( p) [ + 3p( p)(n )] = Npq [ + 3pq(N )] Cetral momets of higher orders may be obtaied by the recursive formula µ r+ = pq { Nrµ r + µ r p startig with µ = ad µ =. The coefficiets of skewess ad kurtosis are give by γ = q p Npq ad γ = 6pq Npq 5.3 Probability Geeratig Fuctio The probability geeratig fuctio is give by ) N G(z) = E(z r ) = z r( N p r ( p) N r = (pz + q) N r= r ad the characteristic fuctio thus by φ(t) = G(e ıt ) = ( q + pe ıt) N } 6

5.4 Cumulative Fuctio For fixed N ad p oe may easily costruct the cumulative fuctio P (r) by a recursive formula, see sectio o radom umbers below. However, a iterestig ad useful relatio exist betwee P (r) ad the icomplete Beta fuctio I x amely k P (k) = p(r; N, p) = I p (N k, k + ) r= For further iformatio o I x see sectio 4.7. 5.5 Radom Number Geeratio I order to achieve radom umbers from a biomial distributio we may either Geerate N uiform radom umbers ad accumulate the umber of such that are less or equal to p, or Use the cumulative techique, i.e. costruct the cumulative, distributio, fuctio ad by use of this ad oe uiform radom umber obtai the required radom umber, or for larger values of N, say N >, use a approximatio to the ormal distributio with mea Np ad variace Npq. Except for very small values of N ad very high values of p the cumulative techique is the fastest for umerical calculatios. This is especially true if we proceed by costructig the cumulative vector oce for all (as opposed to makig this at each call) usig the recursive formula p(i) = p(i ) p N + i q i for i =,,..., N startig with p() = q N. However, usig the relatio give i the previous sectio with a well optimized code for the icomplete Beta fuctio (see [] or sectio 4.7) turs out to be a umerically more stable way of creatig the cumulative distributio tha a simple loop addig up the idividual probabilities. 5.6 Estimatio of Parameters Experimetally the quatity r, the relative umber of successes i N trials, ofte is of more N iterest tha r itself. This variable has expectatio E( r ) = p ad variace V ( r ) = pq. N N N The estimated value for p i a experimet givig r successes i N trials is ˆp = r. N If p is ukow a ubiased estimate of the variace of a biomial distributio is give by V (r) = N ( ) ( r N N r ) = N N ˆp( ˆp) N N N This is possible oly if we require radom umbers from oe ad the same biomial distributio with fixed values of N ad p. 7

To fid lower ad upper cofidece levels for p we proceed as follows. For lower limits fid a p low such that N r=k ( ) N p r r low( p low ) N r = α or expressed i terms of the icomplete Beta fuctio I p (N k +, k) = α for upper limits fid a p up such that k r= ( ) N p r r up( p up ) N r = α which is equivalet to I p (N k, k + ) = α i.e. I p (k +, N k) = α. As a example we take a experimet with N = where a certai umber of successes k N have bee observed. The cofidece levels correspodig to 9%, 95%, 99% as well as the levels correspodig to oe, two ad three stadard deviatios for a ormal distributio (84.3%, 97.7% ad 99.87% probability cotet) are give below. Lower cofidece levels Upper cofidece levels k 3σ 99% σ 95% 9% σ ˆp σ 9% 95% σ 99% 3σ..7..6.3.37.48........9.34.39.45.5.6....4.5.7..4.45.5.56.6.7 3..5.6.9..4.3.5.55.6.66.7.79 4.5.9..5.9..4.6.65.7.74.78.85 5..5.8..7.3.5.7.73.78.8.85.9 6.5..6.3.35.4.6.78.8.85.88.9.95 7..3.34.39.45.49.7.86.88.9.94.95.98 8.9.39.44.49.55.59.8.93.95.96.98.98.99 9.39.5.55.6.66.7.9.98.99.99....5.63.69.74.79.83. 5.7 Probability Cotet It is sometimes of iterest to judge the sigificace level of a certai outcome give the hypothesis that p =. If N trials are made ad we fid k successes (let s say k < N/ else use N k istead of k) we wat to estimate the probability to have k or fewer successes plus the probability for N k or more successes. Sice the assumptio is that p = we wat the two-tailed probability cotet. To calculate this either sum the idividual probabilities or use the relatio to the icomplete beta fuctio. The former may seem more straightforward but the latter may be computatioally easier give a routie for the icomplete beta fuctio. If k = N/ we watch up ot to add the cetral term twice (i this case the requested probability is % ayway). I the table below we show such cofidece levels i % for values of N ragig from to. E.g. the probability to observe 3 successes (or failures) or less ad failures (or successes) or more for = 5 is 3.5%. 8

k N 3 4 5 6 7 8 9. 5.. 3 5.. 4.5 6.5. 5 6.5 37.5. 6 3.3.88 68.75. 7.56.5 45.3. 8.78 7.3 8.9 7.66. 9.39 3.9 7.97 5.78...5.94 34.38 75.39...7 6.54.66 54.88..5.63 3.86 4.6 38.77 77.44. 3..34.5 9.3 6.68 58.. 4..8.9 5.74 7.96 4.4 79.5. 5...74 3.5.85 3.8 6.7. 6..5.4.3 7.68. 45.45 8.36. 7..3.3.7 4.9 4.35 33.3 6.9. 8...3.75 3.9 9.63 3.79 48.7 8.45. 9...7.44.9 6.36 6.7 35.93 64.76....4.6.8 4.4.53 6.3 5.34 8.38. 9

6 Biormal Distributio 6. Itroductio As a geeralizatio of the ormal or Gauss distributio to two dimesios we defie the biormal distributio as ( ( ) ( ) ) x µ x f(x, x ) = πσ σ ρ e ( ρ + µ ρ x µ x µ ) σ σ σ σ where µ ad µ are the expectatio values of x ad x, σ ad σ their stadard deviatios ad ρ the correlatio coefficiet betwee them. Puttig ρ = we see that the distributio becomes the product of two oe-dimesioal Gauss distributios. 4 3 x - - -3-4 -4-3 - - 3 4 x Figure : Biormal distributio I figure we show cotours for a stadardized Biormal distributio i.e puttig µ = µ = ad σ = σ = (these parameters are ayway shift- ad scale-parameters oly). I the example show ρ =.5. Usig stadardized variables the cotours rage from a perfect circle for ρ = to gradually thier ellipses i the ±45 directio as ρ ±. The cotours show correspod to the oe, two, ad three stadard deviatio levels. See sectio o probability cotet below for details.

6. Coditioal Probability Desity The coditioal desity of the biormal distributio is give by f(x y) = f(x, y)/f(y) = = exp πσx ρ σx( ρ ) ( = N µ x + ρ σ ) x (y µ y ), σ σ x( ρ ) y [ x ( µ x + ρσ )] x (y µ y ) σ y = which is see to be a ormal distributio which for ρ = is, as expected, give by N(µ x, σ x) but geerally has a mea shifted from µ x ad a variace which is smaller tha σ x. 6.3 Characteristic Fuctio The characteristic fuctio of the biormal distributio is give by φ(t, t ) = E(e ıt x +ıt x ) = = exp { ıt µ + ıt µ + e ıt x +ıt x f(x, x )dx dx = [ (ıt ) σ + (ıt ) σ + (ıt )(ıt )ρσ σ ]} which shows that if the correlatio coefficiet ρ is zero the the characteristic fuctio factorizes i.e. the variables are idepedet. This is a uique property of the ormal distributio sice i geeral ρ = does ot imply idepedece. 6.4 Momets To fid bivariate momets of the biormal distributio the simplest, but still quite tedious, way is to use the characteristic fuctio give above (see sectio.7.3). Algebraic bivariate momets for the biormal distributio becomes somewhat complicated but ormally they are of less iterest tha the cetral oes. Algebraic momets of the type µ k ad µ k are, of course, equal to momets of the margial oe-dimesioal ormal distributio e.g. µ = µ, µ = µ + σ, ad µ 3 = µ (σ + µ ) (for µ k simply exchage the subscripts o µ ad σ). Some other lower order algebraic bivariate momets are give by µ = µ µ + ρσ σ µ = ρσ σ µ + σ µ + µ µ µ = σ σ + σ µ + σ µ + µ µ + ρ σ σ + 4ρσ σ µ µ Beware of the somewhat cofusig otatio where µ with two subscripts deotes bivariate momets while µ with oe subscript deotes expectatio values. Lower order cetral bivariate momets µ kl, arraged i matrix form, are give by

l = l = l = l = 3 l = 4 k = σ 3σ 4 k = ρσ σ 3ρσ σ 3 k = σ σ σ (ρ + ) 3σ σ 4 (4ρ + ) k = 3 3ρσ 3 σ 3ρσ 3 σ 3 (ρ + 3) k = 4 3σ 4 3σ 4 σ (4ρ + ) 3σ 4 σ 4 (8ρ 4 + 4ρ + 3) 6.5 Box-Muller Trasformatio Recall that if we have a distributio i oe set of variables {x, x,..., x } ad wat to chage variables to aother set {y, y,..., y } the distributio i the ew variables are give by x x x y y... y x x x f(y, y,..., y ) = y y... y f(x, x,..., x )...... x x x y y... y where the symbol J deotes the absolute value of the determiat of the Jacobia J. Let x ad x be two idepedet stochastic variables from a uiform distributio betwee zero ad oe ad defie y = l x si πx y = l x cos πx Note that with the defiitio above < y < ad < y <. I order to obtai the joit probability desity fuctio i y ad y we eed to calculate the Jacobia matrix ) (x, x ) (y, y ) = ( x y x y x y x y I order to obtai these partial derivatives we express x ad x i y ad y by rewritig the origial equatios. which implies y + y = l x y y = ta πx x = e (y +y ) x = π arcta ( ) y The the Jacobia matrix becomes ( (x, x ) (y, y ) = y e (y +y ) y e (y +y ) y πy cos arcta ( y y ) y πy cos arcta ( y y ) )

The distributio f(y, y ) is give by (x, x ) f(y, y ) = (y, y ) f(x, x ) where f(x, x ) is the uiform distributio i x ad x. Now f(x, x ) = i the iterval x ad x ad zero outside this regio. ad the absolute value of the determiat of the Jacobia is ) ) but ( y y ad thus (x, x ) (y, y ) = + ) π e cos arcta f(y, y ) = (y +y ) ( y y ( ) y y π e + cos arcta ( y y = (ta πx + ) cos πx = (y +y ) = e y π e y π i.e. the product of two stadard ormal distributios. Thus the result is that y ad y are distributed as two idepedet stadard ormal variables. This is a well kow method, ofte called the Box-Muller trasformatio, used i order to achieve pseudoradom umbers from the stadard ormal distributio give a uiform pseudoradom umber geerator (see below). The method was itroduced by G. E. P. Box ad M. E. Muller [5]. 6.6 Probability Cotet I figure cotours correspodig to oe, two, ad three stadard deviatios were show. The projectio o each axis for e.g. the oe stadard deviatio cotour covers the rage x i ad cotais a probability cotet of 68.3% which is well kow from the oe-dimesioal case. More geerally, for a cotour correspodig to z stadard deviatios the cotour has the equatio (x + x ) + (x x ) = z + ρ ρ i.e. the major ad mior semi-axes are z + ρ ad z ρ, respectively. The fuctio value at the cotour is give by { } f(x, x ) = π ρ exp z Expressed i polar coordiates (r, φ) the cotour is described by r = z ( ρ ) ρ si φ cos φ While the projected probability cotets follow the usual figures for oe-dimesioal ormal distributios the joit probability cotet withi each ellipse is smaller. For the 3

oe, two, ad three stadard deviatio cotours the probability cotet, regardless of the correlatio coefficiet ρ, iside the ellipse is approximately 39.3%, 86.5%, ad 98.9%. If we would like to fid the ellipse with a joit probability cotet of 68.3% we must chose z.5 (for a cotet of 95.5% use z.5 ad for 99.7% use z 3.4). Se further discussio o probability cotet for a multiormal distributio i sectio 8.3. 6.7 Radom Number Geeratio The joit distributio of y ad y i sectio 6.5 above is a biormal distributio havig ρ =. For arbitrary correlatio coefficiets ρ the biormal distributio is give by f(x, x ) = ( ( ) ( ) ) x µ x πσ σ ρ e ( ρ + µ ρ x µ x µ ) σ σ σ σ where µ ad µ are the expectatio values of x ad x, σ ad σ their stadard deviatios ad ρ the correlatio coefficiet betwee them. Variables distributed accordig to the biormal distributio may be obtaied by trasformig the two idepedet umbers y ad y foud i the sectio 6.5 either as ( ) z = µ + σ y ρ + y ρ z = µ + σ y or as z = µ + σ ( ) y + ρ + y ρ z = µ + σ ( ) y + ρ y ρ which ca be proved by expressig y ad y as fuctios of z ad z ad evaluate I the first case (y, y ) f(z, z ) = (z, z ) f(y, y ) = y = y = z µ σ y y z z y y z z ( z µ ρ z ) µ ρ σ σ f(y, y ) ad i the secod case y = y = ( z µ + ρ ρ σ + z µ σ ( z µ σ z µ σ ) ) 4

I both cases the absolute value of the determiat of the Jacobia is /σ σ ρ ad we get f(z, z ) = σ σ ρ e y π e y = π πσ σ ρ e (y +y ) Isertig the relatios expressig y ad y i z ad z i the expoet we fially obtai the biormal distributio i both cases. Thus we have foud methods which give two idepedet uiform pseudoradom umbers betwee zero ad oe supplies us with a pair of umbers from a biormal distributio with arbitrary meas, stadard deviatios ad correlatio coefficiet. 5

7 Cauchy Distributio 7. Itroductio The Cauchy distributio is give by f(x) = π + x ad is defied for < x <. It is a symmetric uimodal distributio as is show i figure 3. Figure 3: Graph of the Cauchy distributio The distributio is amed after the famous frech mathematicia Augusti Louis Cauchy (789-857) who was a professor at École Polytechique i Paris from 86. He was oe of the most productive mathematicias which have ever existed. 7. Momets This probability desity fuctio is peculiar iasmuch as it has udefied expectatio value ad all higher momets diverge. For the expectatio value the itegral is ot completely coverget, i.e. E(x) = π lim a,b π b a x + x dx x + x dx 6

does ot exist. However, the pricipal value lim a π a a x + x dx does exist ad is equal to zero. Ayway the covetio is to regard the expectatio value of the Cauchy distributio as udefied. Other measures of locatio ad dispersio which are useful i the case of the Cauchy distributio is the media ad the mode which are at x = ad the half-width at halfmaximum which is (half-maxima at x = ±). 7.3 Normalizatio I spite of the somewhat awkward property of ot havig ay momets the distributio at least fulfil the ormalizatio requiremet for a proper probability desity fuctio i.e. N = f(x)dx = π + x dx = π π/ π/ + ta φ dφ cos φ = where we have made the substitutio ta φ = x i order to simplify the itegratio. 7.4 Characteristic Fuctio The characteristic fuctio for the Cauchy distributio is give by φ(t) = = π = π e ıtx f(x)dx = π cos tx + x dx + cos tx dx = e t + x cos tx + ı si tx + x dx = cos tx + x dx + ı si tx + x dx + ı si tx + x dx = where we have used that the two sie itegrals are equal but with opposite sig whereas the two cosie itegrals are equal. The fial itegral we have take from stadard itegral tables. Note that the characteristic fuctio has o derivatives at t = oce agai tellig us that the distributio has o momets. 7.5 Locatio ad Scale Parameters I the form give above the Cauchy distributio has o parameters. It is useful, however, to itroduce locatio (x ) ad scale (Γ > ) parameters writig f(x; x, Γ) = π 7 Γ Γ + (x x )

where x is the mode of the distributio ad Γ the half-width at half-maximum (HWHM). Icludig these two parameters the characteristic fuctio is modified to 7.6 Breit-Wiger Distributio φ(t) = e itx Γ t I this last form we recogize the Breit-Wiger formula, amed after the two physicists Gregory Breit ad Eugee Wiger, which arises i physics e.g. i the descriptio of the cross sectio depedece o eergy (mass) for two-body resoace scatterig. Resoaces like e.g. the ++ i π + p scatterig or the ρ i ππ scatterig ca be quite well described i terms of the Cauchy distributio. This is the reaso why the Cauchy distributio i physics ofte is referred to as the Breit-Wiger distributio. However, i more elaborate physics calculatios the width may be eergy-depedet i which case thigs become more complicated. 7.7 Compariso to Other Distributios The Cauchy distributio is ofte compared to the ormal (or Gaussia) distributio with mea µ ad stadard deviatio σ > f(x; µ, σ) = σ π e ( x µ σ ) ad the double-expoetial distributio with mea µ ad slope parameter λ > f(x; µ, λ) = λ e λ x µ These are also examples of symmetric uimodal distributios. The Cauchy distributio has loger tails tha the double-expoetial distributio which i tur has loger tails tha the ormal distributio. I figure 4 we compare the Cauchy distributio with the stadard ormal (µ = ad σ = ) ad the double-expoetial distributios (λ = ) for x >. The ormal ad double-expoetial distributios have well defied momets. Sice they are symmetric all cetral momets of odd order vaish while cetral momets of eve order are give by µ = ()!σ /! (for ) for the ormal ad by µ =!/λ (for eve ) for the double-expoetial distributio. E.g. the variaces are σ ad /λ ad the fourth cetral momets 3σ 4 ad 4/λ 4, respectively. The Cauchy distributio is related to Studet s t-distributio with degrees of freedom (with a positive iteger) f(t; ) = Γ ( ) + ( ( ) πγ + t ) + = ( + t ) + B (, ) where Γ(x) is the Euler gamma-fuctio ot o be mixed up with the width parameter for the Cauchy distributio used elsewhere i this sectio. B is the beta-fuctio defied i terms of the Γ-fuctio as B(p, q) = Γ(p)Γ(q). As ca be see the Cauchy distributio arises Γ(p+q) 8

Figure 4: Compariso betwee the Cauchy distributio, the stadard ormal distributio, ad the double-expoetial distributio as the special case where =. If we chage variable to x = t/ ad put m = + the Studet s t-distributio becomes f(x; m) = k ( + x ) with k = Γ(m) m Γ ( ) ( ) = Γ m B ( m, ) where k is simply a ormalizatio costat. Here it is easier to see the more geeral form of this distributio which for m = gives the Cauchy distributio. The requiremet correspods to m beig a half-iteger but we could eve allow for m beig a real umber. As for the Cauchy distributio the Studet s t-distributio have problems with diverget momets ad momets of order does ot exist. Below this limit odd cetral momets are zero (the distributio is symmetric) ad eve cetral momets are give by µ r = Γ ( ) ( r + r Γ r) Γ ( ( ) ) Γ = B ( r +, r r) B (, ) for r a positive iteger (r < ). More specifically the expectatio value is E(t) =, the variace V (t) = ad the fourth cetral momet is give by µ 3 4 = whe they ( )( 4) exist. As the Studet s t-distributio approaches a stadard ormal distributio. 7.8 Trucatio I order to avoid the log tails of the distributio oe sometimes itroduces a trucatio. This, of course, also cures the problem with the udefied mea ad diverget higher 9

momets. For a symmetric trucatio X x X we obtai the reormalized probability desity fuctio f(x) = arcta X + x which has expectatio value E(x) =, variace V (x) = X, third cetral momet ( arcta X µ 3 = ad fourth cetral momet µ 4 = X X arcta X 3 ) +. The fractio of the origial Cauchy distributio withi the symmetric iterval is f = arcta X. We will, however, π ot make ay trucatio of the Cauchy distributio i the cosideratios made i this ote. 7.9 Sum ad Average of Cauchy Variables I most cases oe would expect the sum ad average of may variables draw from the same populatio to approach a ormal distributio. This follows from the famous Cetral Limit Theorem. However, due to the diverget variace of the Cauchy distributio the requiremets for this theorem to hold is ot fulfilled ad thus this is ot the case here. We defie S = x i ad S = S i= with x i idepedet variables from a Cauchy distributio. The characteristic fuctio of a sum of idepedet radom variables is equal to the product of the idividual characteristic fuctios ad hece Φ(t) = φ(t) = e t for S. Turig this ito a probability desity fuctio we get (puttig x = S for coveiece) f(x) = Φ(t)e ıxt dt = e (ıxt+ t ) dt = e t ıxt dt + e ıxt t dt = π π π = [ ] e t( ıx) [ ] e t(ıx+) + = ( π ıx ıx π ıx + ) = + ıx π + x This we recogize as a Cauchy distributio with scale parameter Γ = ad thus for each additioal Cauchy variable the HWHM icreases by oe uit. Moreover, the probability desity fuctio of S is give by f(s ) = ds ds f(s ) = π + S i.e. the somewhat amazig result is that the average of ay umber of idepedet radom variables from a Cauchy distributio is also distributed accordig to the Cauchy distributio. 3

7. Estimatio of the Media For the Cauchy distributio the sample mea is ot a cosistet estimator of the media of the distributio. I fact, as we saw i the previous sectio, the sample mea is itself distributed accordig to the Cauchy distributio ad therefore has diverget variace. However, the sample media for a sample of idepedet observatios from a Cauchy distributio is a cosistet estimator of the true media. I the table below we give the expectatios ad variaces of the sample mea ad sample media estimators for the ormal, double-expoetial ad Cauchy distributios (see above for defiitios of distributios). Sortig all the observatios the media is take as the value for the cetral observatio for odd ad as the average of the two cetral values for eve. The variace of the sample mea is simply the variace of the distributio divided by the sample size. For large the variace of the sample media m is give by V (m) = /4f where f is the fuctio value at the media. Distributio E(x) V (x) E(m) V (m) σ Normal µ πσ µ Double-expoetial µ λ µ λ Cauchy udef. x π Γ 4 For a ormal distributio the sample mea is superior to the media as a estimator of the mea (i.e. it has the smaller variace). However, the double-expoetial distributio is a example of a distributio where the sample media is the best estimator of the mea of the distributio. I the case of the Cauchy distributio oly the media works of the above alteratives but eve better is a proper Maximum Likelihood estimator. I the case of the ormal ad double-expoetial the mea ad media, respectively, are idetical to the maximum likelihood estimators but for the Cauchy distributio such a estimator may ot be expressed i a simple way. The large approximatio for the variace of the sample media gives coservative estimates for lower values of i the case of the ormal distributio. Beware, however, that for the Cauchy ad the double-expoetial distributios it is ot coservative but gives too small values. Calculatig the stadard deviatio this is withi % of the true value already at = 5 for the ormal distributio whereas for the Cauchy distributio this is true at about = ad for the double-expoetial distributio oly at about = 6. 7. Estimatio of the HWHM To fid a estimator for the half-width at half-maximum is ot trivial. It implies biig the data, fidig the maximum ad the locatig the positios where the curve is at halfmaximum. Ofte it is preferable to fit the probability desity fuctio to the observatios i such a case. 3

However, it turs out that aother measure of dispersio the so called semi-iterquartile rage ca be used as a estimator. The semi-iterquartile rage is defied as half the differece betwee the upper ad the lower quartiles. The quartiles are the values which divide the probability desity fuctio ito four parts with equal probability cotet, i.e. 5% each. The secod quartile is thus idetical to the media. The defiitio is thus S = (Q 3 Q ) = Γ where Q is the lower ad Q 3 the upper quartile which for the Cauchy distributio is equal to x Γ ad x + Γ, respectively. As is see S = HW HM = Γ ad thus this estimator may be used i order to estimate Γ. We gave above the large approximatio for the variace of the media. The media ad the quartiles are examples of the more geeral cocept quatiles. Geerally the large approximatio for the variace of a quatile Q is give by V (Q) = pq/f where f is the ordiate at the quatile ad p ad q = p are the probability cotets above ad below the quatile, respectively. The covariace betwee two quatiles Q ad Q is, with similar otatios, give by Cov(Q, Q ) = p q /f f where Q should be the leftmost quatile. For large the variace of the semi-iterquartile rage for a sample of size is thus foud by error propagatio isertig the formulæ above V (S) = 4 (V (Q ) + V (Q 3 ) Cov(Q, Q 3 )) = 64 ( 3 f + 3 f 3 ) = f f 3 6f = π Γ 4 where f ad f 3 are the fuctio values at the lower ad upper quartile which are both equal to /πγ. This turs out to be exactly the same as the variace we foud for the media i the previous sectio. After sortig the sample the quartiles are determied by extrapolatio betwee the two observatios closest to the quartile. I the case where + is a multiple of 4 i.e. the series =, 6,... the lower quartile is exactly at the + :th observatio ad the upper quartile 4 at the 3+ :th observatio. I the table below we give the expectatios ad variaces of 4 the estimator of S as well as the variace estimator s for the ormal, double-expoetial ad Cauchy distributios. The variace estimator s ad its variace are give by s = (x i x) ad V (s ) = µ 4 µ i= + µ ( ) with µ ad µ 4 the secod ad fourth cetral momets. The expectatio value of s is equal to the variace ad thus it is a ubiased estimator. Distributio HWHM E(s ) V (s ) E(S) V (S) Normal σ l σ σ4.6745σ 6f(Q ) Double-expoetial l λ l λ λ α 4 λ λ Cauchy Γ Γ π Γ 4 3

I this table α = +.4 if we iclude the secod term i the expressio of V (s ) above ad α = otherwise. It ca be see that the double-expoetial distributio also has HW HM = S but for the ormal distributio HW HM.774σ as compared to S.6745σ. For the three distributios tested the semi-iterquartile rage estimator is biased. I the case of the ormal distributio the values are approachig the true value from below while for the Cauchy ad double-expoetial distributios from above. The large approximatio for V (S) is coservative for the ormal ad double-expoetial distributio but ot coservative for the Cauchy distributio. I the latter case the stadard deviatio of S is withi % of the true value for > 5 but for small values of it is substatially larger tha give by the formula. The estimated value for Γ is less tha % too big for > 5. 7. Radom Number Geeratio I order to geerate pseudoradom umbers from a Cauchy distributio we may solve the equatio F (x) = ξ where F (x) is the cumulative distributio fuctio ad ξ is a uiform pseudoradom umber betwee ad. This meas solvig for x i the equatio F (x) = Γ π x Γ + (t x ) dt = ξ If we make the substitutio ta φ = (t x )/Γ usig that dφ/ cos φ = dt/γ we obtai ( ) x π arcta x + Γ = ξ which fially gives x = x + Γ ta ( π ( )) ξ as a pseudoradom umber from a Cauchy distributio. equivalet to use x = x + Γ ta(πξ) Oe may easily see that it is which is a somewhat simpler expressio. A alterative method (see also below) to achieve radom umbers from a Cauchy distributio would be to use x = x + Γ z z where z ad z are two idepedet radom umbers from a stadard ormal distributio. However, if the stadard ormal radom umbers are achieved through the Box-Muller trasformatio the z /z = ta πξ ad we are back to the previous method. I geeratig pseudoradom umbers oe may, if profitable, avoid the taget by a Geerate i u ad v two radom umbers from a uiform distributio betwee - ad. b If u + v > (outside circle with radius oe i uv-plae) go back to a. c Obtai x = x + Γ u v as a radom umber from a Cauchy distributio. 33

7.3 Physical Picture A physical picture givig rise to the Cauchy distributio is as follows: Regard a plae i which there is a poit source which emits particles isotropically i the plae (either i the full π regio or i oe hemisphere π radias wide). The source is at the x-coordiate x ad the particles are detected i a detector extedig alog a lie Γ legth uits from the source. This sceario is depicted i figure 5 SOURCE φ = π φ = π φ = π φ 4 φ = π 4 φ = Γ DETECTOR AXIS x x x-axis Figure 5: Physical sceario leadig to a Cauchy distributio The distributio i the variable x alog the detector will the follow the Cauchy distributio. As ca be see by pure geometrical cosideratios this is i accordace with the result above where pseudoradom umbers from a Cauchy distributio could be obtaied by x = x + Γ ta φ, i.e. ta φ = x x, with φ uiformly distributed betwee π ad π. Γ To prove this let us start with the distributio i φ f(φ) = π for π φ π To chage variables from φ to x requires the derivative dφ/dx which is give by dφ dx = cos φ Γ = ( ) x x Γ cos arcta Γ Note that the iterval from π to π i φ maps oto the iterval < x <. We get dφ f(x) = dx f(φ) = ( ) x x πγ cos arcta = Γ πγ cos φ = = πγ Γ Γ + (x x ) = π Γ Γ + (x x ) i.e. the Cauchy distributio. It is just as easy to make the proof i the reversed directio, i.e. give a Cauchy distributio i x oe may show that the φ-distributio is uiform betwee π ad π. 34

7.4 Ratio Betwee Two Stadard Normal Variables As metioed above the Cauchy distributio also arises if we take the ratio betwee two stadard ormal variables z ad z, viz. x = x + Γ z z. I order to deduce the distributio i x we first itroduce a dummy variable y which we simply take as z itself. We the make a chage of variables from z ad z to x ad y. The trasformatio is give by or if we express z ad z i x ad y x = x + Γ z z y = z z = y(x x )/Γ z = y The distributio i x ad y is give by (z, z ) f(x, y) = (x, y) f(z, z ) where the absolute value of the determiat of the Jacobia is equal to y/γ ad f(z, z ) is the product of two idepedet stadard ormal distributios. We get ( ) y (x x ) Γ +y f(x, y) = y Γ π e (z +z ) = y πγ e I order to obtai the margial distributio i x we itegrate over y where we have put f(x) = f(x, y)dy = πγ α = ( (x ) ) x + for coveiece. If we make the substitutio z = y we get Γ ye αy dy f(x) = αz dz e πγ = πγα Note that the first factor of comes from the fact that the regio < y < maps twice oto the regio < z <. Fially i.e. a Cauchy distributio. f(x) = πγα = πγ ( x x Γ ) = + π 35 Γ (x x ) + Γ

8 Chi-square Distributio 8. Itroductio The chi-square distributio is give by f(x; ) = ( ) x e x Γ ( ) where the variable x ad the parameter, the umber of degrees of freedom, is a positive iteger. I figure 6 the distributio is show for -values of,, 5 ad. For the distributio has a maximum at. Figure 6: Graph of chi-square distributio for some values of 8. Momets Algebraic momets of order k are give by µ k = E(x k ) = Γ ( ) x k ( x ) e x dx = k Γ ( ) y +k e y dy = k Γ ( + k) Γ ( ) = = k ( + ) ( + k )( + k ) = ( + )( + 4) ( + k ) e.g. the first algebraic momet which is the expectatio value is equal to. A recursive formula to calculate algebraic momets is thus give by µ k = µ k ( + k ) where we may start with µ = to fid the expectatio value µ =, µ = ( + ) etc. 36

From this we may calculate the cetral momets which for the lowest orders become µ =, µ 3 = 8, µ 4 = ( + 4), µ 5 = 3(5 + ) ad µ 6 = 4(3 + 5 + 96). The coefficiets of skewess ad kurtosis thus becomes γ = / ad γ = /. The fact that the expectatio value of a chi-square distributio equals the umber of degrees of freedom has led to a bad habit to give the ratio betwee a foud chi-square value ad the umber of degrees of freedom. This is, however, ot a very good variable ad it may be misleadig. We strogly recommed that oe always should give both the chi-square value ad degrees of freedom e.g. as χ /.d.f.=9.7/5. To judge the quality of the fit we wat a better measure. Sice the exact samplig distributio is kow oe should stick to the chi-square probability as calculated from a itegral of the tail i.e. give a specific chi-square value for a certai umber of degrees of freedom we itegrate from this value to ifiity (see below). As a illustratio we show i figure 7 the chi-square probability for costat ratios of χ /.d.f. Figure 7: Chi-square probability for costat ratios of χ /.d.f. Note e.g. that for few degrees of freedom we may have a acceptable chi-square value eve for larger ratios. 8.3 Characteristic Fuctio The characteristic fuctio for a chi-square distributio with degrees of freedom is give by φ(t) = E(e ıtx ) = Γ ( ) ( x ) e ( ıt)x dx = 37 Γ ( ) ( y ıt ) e y dy ıt =

= Γ ( ) ( ıt) y e y dy = ( ıt) 8.4 Cumulative Fuctio The cumulative, or distributio, fuctio for a chi-square distributio with degrees of freedom is give by F (x) = Γ ( ) = γ ( x ), x Γ ( ) = P ( x ) e x dx = (, x ) Γ ( ) x y e y dy = where P (, ) x is the icomplete Gamma fuctio (see sectio 4.5). I this calculatio we have made the simple substitutio y = x/ i simplifyig the itegral. 8.5 Origi of the Chi-square Distributio If z, z,..., z are idepedet stadard ormal radom variables the zi is distributed as a chi-square variable with degrees of freedom. I order to prove this first regard the characteristic fuctio for the square of a stadard ormal variable E(e ıtz ) = π e z ( ıt) dz = π e y dy ıt = i= ıt where we made the substitutio y = z ıt. For a sum of such idepedet variables the characteristic fuctio is the give by φ(t) = ( it) which we recogize as the characteristic fuctio for a chi-square distributio with degrees of freedom. This property implies that if x ad y are idepedetly distributed accordig to the chisquare distributio with ad m degrees of freedom, respectively, the x + y is distributed as a chi-square variable with m + degrees of freedom. Ideed the requiremet that all z s come from a stadard ormal distributio is more tha what is eeded. The result is the same if all observatios x i come from differet ormal populatios with meas µ i ad variace σi if we i each case calculate a stadardized variable by subtractig the mea ad dividig with the stadard deviatio i.e. takig z i = (x i µ i )/σ i. 38

8.6 Approximatios For large umber of degrees of freedom the chi-square distributio may be approximated by a ormal distributio. There are at least three differet approximatios. Firstly we may aïvely costruct a stadardized variable z = x E(x) V (x) = x which would ted to ormality as icreases. Secodly a approximatio, due to R. A. Fisher, is that the quatity z = x approaches a stadard ormal distributio faster tha the stadardized variable. Thirdly a trasformatio, due to E. B. Wilso ad M. M. Hilferty, is that the cubic root of x/ is closely distributed as a stadard ormal distributio usig z 3 = ( x ) 3 ( The secod approximatio is probably the most well kow but the latter is approachig ormality eve faster. I fact there are eve correctio factors which may be applied to z 3 to give a eve more accurate approximatio (see e.g. [6]) 9 9 z 4 = z 3 + h = z 3 + 6 h 6 with h 6 give for values of z from 3.5 to 3.5 i steps of.5 (i this order the values of h 6 are.8,.67,.33,.,.,.6,.6,.,.3,.6,.5,.,.7,.43, ad.8). To compare the quality of all these approximatios we calculate the maximum deviatio betwee the cumulative fuctio for the true chi-square distributio ad each of these approximatios for =3 ad =. The results are show i the table below. Normally oe accepts z for > while z 3, ad certaily z 4, are eve better already for > 3. Approximatio = 3 = z.34.9 z.85.47 z 3.39. z 4.44.35 8.7 Radom Number Geeratio As we saw above the sum of idepedet stadard ormal radom variables gave a chi-square distributio with degrees of freedom. This may be used as a techique to produce pseudoradom umbers from a chi-square distributio. This required a geerator for stadard ormal radom umbers ad may be quite slow. However, if we make use of ) 39

the Box-Muller trasformatio i order to obtai the stadard ormal radom umbers we may simplify the calculatios. First we recall the Box-Muller trasformatio which give two pseudoradom umbers uiformly distributed betwee zero ad oe through the trasformatio z = z = l ξ cos πξ l ξ si πξ gives, i z ad z, two idepedet pseudoradom umbers from a stadard ormal distributio. Addig such squared radom umbers implies that y k = l(ξ ξ ξ k ) y k+ = l(ξ ξ ξ k ) l ξ k+ cos πξ k+ for k a positive iteger will be distributed as chi-square variable with eve or odd umber of degrees of freedom. I this maer a lot of uecessary operatios are avoided. Sice the chi-square distributio is a special case of the Gamma distributio we may also use a geerator for this distributio. 8.8 Cofidece Itervals for the Variace If x, x,..., x are idepedet ormal radom variables from a N(µ, σ ) distributio the ( )s is distributed accordig to the chi-square distributio with degrees of freedom. σ A α cofidece iterval for the variace is the give by ( )s χ α/, σ ( )s χ α/, where χ α, is the chi-square value for a distributio with degrees of freedom for which the probability to be greater or equal to this value is give by α. See also below for calculatios of the probability cotet of the chi-square distributio. 8.9 Hypothesis Testig Let x, x,..., x be idepedet ormal radom variables distributed accordig to a N(µ, σ ) distributio. To test the ull hypothesis H : σ = σ versus H : σ σ at the α level of sigificace, we would reject the ull hypothesis if ( )s /σ is less tha χ α/, or greater tha χ α/,. 8. Probability Cotet I testig hypotheses usig the chi-square distributio we defie x α = χ α, from F (x α ) = x α f(x; )dx = α 4

i.e. α is the probability that a variable distributed accordig to the chi-square distributio with degrees of freedom exceeds x α. This formula ca be used i order to determie cofidece levels for certai values of α. This is what is doe i producig the tables which is commo i all statistics text-books. However, more ofte the equatio is used i order to calculate the cofidece level α give a experimetally determied chi-square value x α. I calculatig the probability cotet of a chi-square distributio we differ betwee the case with eve ad odd umber of degrees of freedom. This is described i the two followig subsectios. Note that oe may argue that it is as ulikely to obtai a very small chi-square value as a very big oe. It is customary, however, to use oly the upper tail i calculatio of sigificace levels. A too small chi-square value is regarded as ot a big problem. However, i such a case oe should be somewhat critical sice it idicates that oe either is cheatig, are usig selected (biased) data or has (udeliberately) overestimated measuremet errors (e.g. icluded systematic errors). To proceed i calculatig the cumulative fuctio we write α = F (x α ) = Γ ( ) x α ( ) x e x dx = ) Γ ( x α/ z e z dz = P (, x ) α where we have made the substitutio z = x/. From this we see that we may use the icomplete Gamma fuctio P (see sectio 4.5) i evaluatig probability cotets but for historical reasos we have solved the problem by cosiderig the cases with eve ad odd degrees of freedom separately as is show i the ext two subsectios. Although we prefer exact routies to calculate the probability i each specific case a classical table may sometimes be useful. I table o page 7 we show percetage poits, i.e. poits where the cumulative probability is α, for differet degrees of freedom. It is sometimes of iterest e.g. whe rejectig a hypothesis usig a chi-square test to scrutiize extremely small cofidece levels. I table o page 7 we show this for cofidece levels dow to as chi-square values. I table 3 o page 73 we show the same thig i terms of chi-square over degrees of freedom ratios (reluctatly sice we do ot like such ratios). As discussed i sectio 8. we see, perhaps eve more clearly, that for few degrees of freedom the ratios may be very high while for large umber of degrees of freedom this is ot the case for the same cofidece level. 4

8. Eve Number of Degrees of Freedom With eve the power of z i the last itegral i the formula for F (x α ) above is a iteger. From stadard itegral tables we fid x m e ax dx = e ax m r= m!x m r ( ) r (m r)!a r+ where, i our case, a =. Puttig m = ad usig this itegral we obtai α = Γ ( ) = e xα x α/ z e z dz = m r= [ m! e z x m r α m r (m r)! = xα e m ( ) r m!z m r r= (m r)!( ) r+ r= x r α r r! a result which ideed is idetical to the formula for P (, x) for iteger give o page 6. 8. Odd Number of Degrees of Freedom I the case of odd umber of degrees of freedom we make the substitutio z = x yieldig x α ( ) x α = F (x α ) = Γ ( ) e x dx = Γ ( ) = Γ ( ) xα ( z ) xα e z dz = m Γ ( ) m + ] xα ( z ) e z zdz = xα z m e z dz where we have put m = which for odd is a iteger. By partial itegratio i m steps z m e z dz = z m ze z dz = z m e z + (m ) z m e z dz z m e z dz = z m 3 e z + (m 3) z m 4 e z dz. z 4 e z dz = z 3 e z + 3 z e z dz z e z dz = ze z + e z dz we obtai z m e z Applyig this to our case gives α = m Γ ( ) m + = (m )!! (m )!! m e z dz xα r= [ m e z dz 4 (m )!! (r + )!! zr+ e z r= (m )!! (r + )!! zr+ e z ] x α =

= π xα = G( x α ) [ m e z dz r= xα xα π e (r + )!! zr+ e z m r= x r α (r + )!! ] x α = where G(z) is the itegral of the stadard ormal distributio from to z. Here we have used Γ ( ) m + = (m )!! π i order to simplify the coefficiet. This result may be m compared to the formula give o page 6 for the icomplete Gamma fuctio whe the first argumet is a half-iteger. 8.3 Fial Algorithm The fial algorithm to evaluate the probability cotet from to x for a chi-square distributio with degrees of freedom is For eve: Put m =. Set u =, s = ad i =. For i =,,..., m set s = s + u i, i = i + ad u i = u i x i. α = s e x. For odd: Put m =. Set u =, s = ad i =. For i =,,..., m set s = s + u i, i = i + ad u i = u i α = G( x) + x π e x s. 8.4 Chi Distributio x. i+ Sometimes, but less ofte, the chi distributio i.e. the distributio of y = x is used. By a simple chage of variables this distributio is give by ( dx y f(y) = dy f(y ) = y ) e y Γ ( ) = y ( ) y e Γ ( I figure 8 the chi distributio is show for -values of,, 5, ad. The mode of the distributio is at. The cumulative fuctio for the chi distributio becomes ) F (y) = ( ) Γ ( ) y ( ) x e x dx = P, y 43

Figure 8: Graph of chi distributio for some values of ad algebraic momets are give by µ k = ( ) Γ ( ) y k y e y dy = k Γ ( + k Γ ( ) ) 44

9 Compoud Poisso Distributio 9. Itroductio The compoud Poisso distributio describes the brachig process for Poisso variables ad is give by (µ) r e µ λ e λ p(r; µ, λ) = r!! = where the iteger variable r ad the parameters µ ad λ are positive real quatities. 9. Brachig Process The distributio describes the brachig of Poisso variables i all with mea µ where is also distributed accordig to the Poisso distributio with mea λ i.e. r = i with p( i ) = µ i e µ i= i! ad p() = λ e λ! ad thus p(r) = p(r )p() = Due to the so called additio theorem (see page ) for Poisso variables with mea µ the sum of such variables are distributed as a Poisso variable with mea µ ad thus the distributio give above results. 9.3 Momets The expectatio value ad variace of the Compoud Poisso distributio are give by E(r) = λµ ad V (r) = λµ( + µ) while higher momets gets slightly more complicated: µ 3 = µλ { µ + (µ + ) } µ 4 = µλ { µ 3 + 6µ + 7µ + + 3µλ( + µ) } µ 5 = µλ { µ 4 + µ 3 + 5µ + 5µ + + µλ(µ + )(µ + ( + µ) ) } µ 6 = µλ { µ 5 + 5µ 4 + 65µ 3 + 9µ + 3µ + + 5µλ ( 5µ 4 + 33µ 3 + 6µ + 36µ + 5 ) + 5µ λ (µ + ) 3} 9.4 Probability Geeratig Fuctio The probability geeratig fuctio of the compoud Poisso distributio is give by G(z) = exp { λ + λe µ+µz} 45

This is easily foud by usig the rules for brachig processes where the probability geeratig fuctio (p.g.f.) is give by G(z) = G P (G P (z)) where G P (z) is the p.g.f. for the Poisso distributio. 9.5 Radom Number Geeratio Usig the basic defiitio we may proceed by first geerate a radom umber from a Poisso distributio with mea λ ad the aother oe with mea µ. For fixed µ ad λ it is, however, ormally much faster to prepare a cumulative vector for values ragig from zero up to the poit where computer precisio gives uity ad the use this vector for radom umber geeratio. Usig a biary search techique oe may allow for quite log vectors givig good precisio without much loss i efficiecy. 46

Double-Expoetial Distributio. Itroductio The Double-expoetial distributio is give by f(x; µ, λ) = λ e λ x µ where the variable x is a real umber as is the locatio parameter µ while the parameter λ is a real positive umber. The distributio is sometimes called the Laplace distributio after the frech astroomer, mathematicia ad physicist marquis Pierre Simo de Laplace (749 87). It is a symmetric distributio whose tails fall off less sharply tha the Gaussia distributio but faster tha the Cauchy distributio. It has a cusp, discotiuous first derivative, at x = µ. The distributio has a iterestig feature iasmuch as the best estimator for the mea µ is the media ad ot the sample mea. See further the discussio i sectio 7 o the Cauchy distributio where the Double-expoetial distributio is discussed i some detail.. Momets For the Double-expoetial distributio cetral momets are more easy to determie tha algebraic momets (the mea is µ = µ). They are give by µ = (x µ) f(x)dx = λ = ( y λ) e y dy + µ (x µ) e λ(µ x) + µ (x µ) e λ(x µ) = ( ) y e y dy λ =!! + ( ) λ λ i.e. odd momets vaish as they should due to the symmetry of the distributio ad eve momets are give by the simple relatio µ =!/λ. From this oe easily fids that the coefficiet of skewess is zero ad the coefficiet of kurtosis 3. If required algebraic momets may be calculated from the cetral momets especially the lowest order algebraic momets become but more geerally µ = µ, µ = λ + µ, µ 3 = 6µ λ + µ3, ad µ 4 = 4 λ 4 + µ λ + µ 4 µ =.3 Characteristic Fuctio r= ( ) µ r µ r r The characteristic fuctio which geerates cetral momets is give by φ x µ (t) = λ λ + t 47

from which we may fid the characteristic fuctio which geerates algebraic momets φ x (t) = E(e ıtx ) = e ıtµ E(e ıt(x µ) ) = e ıtµ φ x µ (t) = e ıtµ λ λ + t Sometimes a alterative which geerates the sequece µ, µ, µ 3,... is give as.4 Cumulative Fuctio λ φ(t) = ıtµ + λ + t The cumulative fuctio, or distributio fuctio, for the Double-expoetial distributio is give by { F (x) = e λ(µ x) if x µ e λ(x µ) if x > µ From this we see ot oly the obvious that the media is at x = µ but also that the lower ad upper quartile is located at µ l /λ..5 Radom Number Geeratio Give a uiform radom umber betwee zero ad oe i ξ a radom umber from a Double-expoetial distributio is give by solvig the equatio F (x) = ξ for x givig For ξ x = µ + l(ξ)/λ for ξ > x = µ l( ξ)/λ 48

Doubly No-Cetral F -Distributio. Itroductio If x ad x are idepedetly distributed accordig to two o-cetral chi-square distributios with ad degrees of freedom ad o-cetral parameters λ ad λ, respectively, the the variable F = x / x / is said to have a doubly o-cetral F -distributio with, degrees of freedom (positive itegers) ad o-cetrality parameters λ, λ (both ). This distributio may be writte as f(x;,, λ, λ ) = e λ r= s= ( λ ) r r! ( λ ) s s! ( ) x +r ( + x ) +r+s B ( + r, + s ) where we have put = + ad λ = λ + λ. For λ = we obtai the (sigly) o-cetral F -distributio (see sectio 3) ad if also λ = we are back to the ordiary variace ratio, or F -, distributio (see sectio 6). With four parameters a variety of shapes are possible. As a example figure 9 shows the doubly o-cetral F -distributio for the case with =, = 5 ad λ = varyig λ from zero (a ordiary o-cetral F -distributio) to five. Figure 9: Examples of doubly o-cetral F -distributios. Momets Algebraic momets of this distributios become ( ) ( ) r E(x k k λ ) = e λ r! = ( ) k e λ r= r= ( λ ) r r! Γ ( + r + k ) Γ ( + r ) s= ( λ ) s s! ( + r + k ) ( + r ) 49 Γ ( + s k ) Γ ( + s ) =

s= ( λ ) s s! ( + s ) ( + s k ) The r-sum ivolved, with a polyomial i the umerator, is quite easily solvable givig similar expressios as for the (sigly) o-cetral F -distributio. The s-sums, however, with polyomials i the deomiator give rise to cofluet hypergeometric fuctios M(a, b; x) (see appedix B). Lower order algebraic momets are give by E(x) = e λ m m + λ M (, ; λ ) λ + (λ + m)(m + ) E(x ) = e λ ( m E(x 3 ) = e λ ( m E(x 4 ) = e λ ( m.3 Cumulative Distributio ) M ( 4 ( )( 4), ; ) λ ) 3 λ 3 + 3(m + 4)λ + (3λ + m)(m + )(m + 4) M ( 6 ( )( 4)( 6), ; ) λ ) 4 λ 4 + (m + 6) {4λ 3 + (m + 4) [6λ + (4λ + m)(m + )]} ( )( 4)( 6)( 8) M ( 8, ; ) λ The cumulative, or distributio, fuctio may be deduced by simple itegratio F (x) = e λ r= s= = e λ = e λ r= s= r= s= ( λ ) r r! ( λ ) r r! ( ) r λ r! ( λ ) s s! ( λ ) s s! ( λ B ( + r, + s ) B q ( + r, + s ) B ( + r, + s ) = ) s ( I q s! + r, + s ) x ( ) u +r ( ) + u +r+s du = with q = x + x.4 Radom Number Geeratio Radom umbers from a doubly o-cetral F -distributio is easily obtaied usig the defiitio i terms of the ratio betwee two idepedet radom umbers from o-cetral chi-square distributios. This ought to be sufficiet for most applicatios but if eeded more efficiet techiques may easily be developed e.g. usig more geeral techiques. 5

Doubly No-Cetral t-distributio. Itroductio If x ad y are idepedet ad x is ormally distributed with mea δ ad uit variace while y is distributed accordig a o-cetral chi-square distributio with degrees of freedom ad o-cetrality parameter λ the the variable t = x/ y/ is said to have a doubly o-cetral t-distributio with degrees of freedom (positive iteger) ad o-cetrality parameters δ ad λ (with λ ). This distributio may be expressed as f(t;, δ, λ) = e δ e λ π ( ) r λ r= r! Γ ( + r) s= (tδ) s s! ( ) s ( + t ) ( +s+ +r) Γ ( +s+ + r ) For λ = we obtai the (sigly) o-cetral t-distributio (see sectio 33) ad if also δ = we are back to the ordiary t-distributio (see sectio 38). Examples of doubly o-cetral t-distributios are show i figure 9 for the case with = ad δ = 5 varyig λ from zero (a ordiary o-cetral t-distributio) to te. Figure : Examples of doubly o-cetral t-distributios. Momets Algebraic momets may be deduced from the expressio E(t k ) = e δ e λ π = e δ e λ π ( ) r λ r! r= ( ) r λ r= r! Γ ( + r) Γ ( + r) s= s= δ s s! s δ s s! s k 5 s Γ ( +s+ + r ) ( Γ s+k+ ) Γ ( k + r ) t s+k ( + t ) +s+ +r dt =

where the sum should be take for eve values of s + k i.e. for eve (odd) orders sum oly over eve (odd) s-values. Differig betwee momets of eve ad odd order the followig expressios for lower order algebraic momets of the doubly o-cetral t-distributio may be expressed i terms of the cofluet hypergeometric fuctio M(a, b; x) (see appedix B for details) as E(t) = δ λ e Γ ( ) Γ ( ) M (, ; λ E(t ) = e λ (δ + ) M (, ; ) λ E(t 3 ) = δ(δ 3 λ Γ ( ) 3 + 3) 8 e Γ ( E(t 4 ) = ( )( 4).3 Cumulative Distributio The cumulative, distributio, fuctio is give by F (t) = e δ e λ π = e δ e λ π ( ) r λ r! r= ( ) r λ r= r! Γ ( + r) s= s= ) ) M ( 3, ; λ ( δ 4 + 6δ + 3 ) e λ M ( 4, ; λ δ s s! ( ) s Γ ( +s+ + r ) ) t ) ( + u δ s s! s Γ ( ) { ( s+ s + s I s+ q, + r)} u s ) +s+ +r du = where q = (t /)/( + t /) ad s, s are sigs differig betwee cases with positive or egative t as well as odd or eve s i the summatio. More specific, the sig s is if s is odd ad + if it is eve while s is + uless t < ad s is eve i which case it is..4 Radom Number Geeratio Radom umbers from a doubly o-cetral t-distributio is easily obtaied with the defiitio give above usig radom umbers from a ormal distributio ad a o-cetral chi-square distributio. This ought to be sufficiet for most applicatios but if eeded more efficiet techiques may easily be developed e.g. usig more geeral techiques. 5

3 Error Fuctio 3. Itroductio A fuctio, related to the probability cotet of the ormal distributio, which ofte is referred to is the error fuctio ad its complemet erf z = z π e t dt erfc z = e t dt = erf z π z These fuctios may be defied for complex argumets, for may relatios cocerig the error fuctio see [7], but here we are maily iterested i the fuctio for real positive values of z. However, sometimes oe may still wat to defie the fuctio values for egative real values of z usig symmetry relatios erf( z) = erf(z) erfc( z) = erf( z) = + erf(z) 3. Probability Desity Fuctio As is see the error fuctio erf is a distributio (or cumulative) fuctio ad the correspodig probability desity fuctio is give by f(z) = π e z If we make the trasformatio z = (x µ)/σ we obtai a folded ormal distributio f(x; µ, σ) = σ π e ( x µ where the fuctio is defied for x > µ correspodig to z >, µ may be ay real umber while σ >. This implies that erf(z/ ) is equal to the symmetric itegral of a stadard ormal distributio betwee z ad z. The error fuctio may also be expressed i terms of the icomplete Gamma fuctio defied for x. erf x = x π σ ) e t dt = P (, x) 53

4 Expoetial Distributio 4. Itroductio The expoetial distributio is give by f(x; α) = α e x α where the variable x as well as the parameter α is positive real quatities. The expoetial distributio occur i may differet coectios such as the radioactive or particle decays or the time betwee evets i a Poisso process where evets happe at a costat rate. 4. Cumulative Fuctio The cumulative (distributio) fuctio is F (x) = x f(x)dx = e x α ad it is thus straightforward to calculate the probability cotet i ay give situatio. E.g. we fid that the media ad the lower ad upper quartiles are at M = α l.693α, Q = α l 3 4.88α, ad Q 3 = α l 4.386α 4.3 Momets The expectatio value, variace, ad lowest order cetral momets are give by E(x) = α, V (x) = α, µ 3 = α 3, µ 4 = 9α 4, µ 5 = 44α 5, µ 6 = 65α 6, µ 7 = 854α 7, ad µ 8 = 4833α 8 More geerally algebraic momets are give by Cetral momets thereby becomes µ = α! m= ( ) m m! µ = α! α! e = µ e whe the approximatio is, i fact, quite good already for = 5 where the absolute error is.46α 5 ad the relative error.3%. 4.4 Characteristic Fuctio The characteristic fuctio of the expoetial distributio is give by φ(t) = E(e ıtx ) = α 54 e (ıt α )x dx = ıtα

4.5 Radom Number Geeratio The most commo way to achieve radom umbers from a expoetial distributio is to use the iverse to the cumulative distributio such that x = F (ξ) = α l( ξ) = α l ξ where ξ is a uiform radom umber betwee zero ad oe (aware ot to iclude exactly zero i the rage) ad so is, of course, also ξ = ξ. There are, however, alteratives some of which may be of some iterest ad useful if the pealty of usig the logarithm would be big o ay system [8]. 4.5. Method by vo Neuma The first of these is due to J. vo Neuma [9] ad is as follows (with differet ξ s deotig idepedet uiform radom umbers betwee zero ad oe) i Set a =. ii Geerate ξ ad put ξ = ξ. iii Geerate ξ ad if ξ < ξ the go to vi. iv Geerate ξ ad if ξ < ξ the go to iii. v Put a = a + ad go to ii. vi Put x = α(a + ξ ) as a radom umber from a expoetial distributio. 4.5. Method by Marsaglia The secod techique is attributed to G. Marsaglia [3]. Prepare p = e ad q = ( e! +! + + )! for =,,... util the largest represetable fractio below oe is exceeded i both vectors. i Put i = ad geerate ξ. ii If ξ > p i+ put i = i + ad perform this step agai. iii Put k =, geerate ξ ad ξ, ad set ξ mi = ξ. iv If ξ q k the go to vi else set k = k +. v Geerate a ew ξ ad if ξ < ξ mi set ξ mi = ξ ad go to iv. vi Put x = α(i + ξ mi ) as a expoetially distributed radom umber. 55

4.5.3 Method by Ahres The third method is due to J. H. Ahres [8] Prepare q = l (l ) (l ) + + +!!! for =,,... util the largest represetable fractio less tha oe is exceeded. i Put a = ad geerate ξ. ii If ξ < set a = a + l = a + q, ξ = ξ ad perform this step agai. iii Set ξ = ξ ad if ξ l = q the exit with x = α(a + ξ) else put i = ad geerate ξ mi. iv Geerate ξ ad put ξ mi = ξ if ξ < ξ mi the if ξ > q i put i = i + ad perform this step agai else exit with x = α(a + q ξ mi ). Of these three methods the method by Ahres is the fastest. This is much due to the fact that the average umber of uiform radom umbers cosumed by the three methods is.69 for Ahres, 3.58 for Marsaglia, ad 4.3 for vo Neuma. The method by Ahres is ofte as fast as the direct logarithm method o may computers. 56

5 Extreme Value Distributio 5. Itroductio The extreme value distributio is give by f(x; µ, σ) = σ exp { x µ σ } e x µ σ where the upper sig is for the maximum ad the lower sig for the miimum (ofte oly the maximum is cosidered). The variable x ad the parameter µ (the mode) are real umbers while σ is a positive real umber. The distributio is sometimes referred to as the Fisher-Tippett distributio (type I), the log-weibull distributio, or the Gumbel distributio after E. J. Gumbel (89 966). The extreme value distributio gives the limitig distributio for the largest or smallest elemets of a set of idepedet observatios from a distributio of expoetial type (ormal, gamma, expoetial, etc.). A ormalized form, useful to simplify calculatios, is obtaied by makig the substitutio to the variable z = ± x µ which has the distributio σ g(z) = e z e z I figure we show the distributio i this ormalized form. The shape correspods to the case for the maximum value while the distributio for the miimum value would be mirrored i z =. Figure : The ormalized Extreme Value Distributio 57

5. Cumulative Distributio The cumulative distributio for the extreme value distributio is give by F (x) = x f(u)du = ± x µ σ g(z)dz = G(± x µ σ ) where G(z) is the cumulative fuctio of g(z) which is give by G(z) = z e u e u du = e z e y dy = e e z where we have made the substitutio y = e u i simplifyig the itegral. From this, ad usig x = µ ± σz, we fid the positio of the media ad the lower ad upper quartile as 5.3 Characteristic Fuctio M = µ σ l l µ ±.367σ, Q = µ σ l l 4 µ.37σ, ad Q 3 = µ σ l l 4 3 µ ±.46σ The characteristic fuctio of the extreme value distributio is give by φ(t) = E ( e ıtx) = = e ıt(µ σ l z) ze z ( e ıtx σ exp { x µ σ σdz ) z } e x µ σ dx = = e ıtµ z ıtσ e z dz = e ıtµ Γ( ıtσ) where we have made the substitutio z = exp ( (x µ)/σ) i.e. x = µ σ l z ad thus dx = σdz/z to achieve a itegral which could be expressed i terms of the Gamma fuctio (see sectio 4.). As a check we may calculate the first algebraic momet, the mea, by µ = dφ(t) = [ıµγ() + Γ()ψ()( ıσ)] = µ ± σγ ı dt t= ı Here ψ() = γ is the digamma fuctio, see sectio 4.3, ad γ is Euler s costat. Similarly higher momets could be obtaied from the characteristic fuctio or, perhaps eve easier, we may fid cumulats from the cumulat geeratig fuctio l φ(t). I the sectio below, however, momets are determied by a more direct techique. 5.4 Momets Algebraic momets for f(x) are give by E(x ) = x f(x)dx = 58 (µ ± σz) g(z)dz

which are related to momets of g(z) E(z ) = z e z e z dz = ( l y) e y dy The first six such itegrals, for values from to 6, are give by ( l x)e x dx = γ ( l x) e x dx = γ + π 6 ( l x) 3 e x dx = γ 3 + γπ + ζ 3 ( l x) 4 e x dx = γ 4 + γ π + 3π4 + 8γζ 3 ( l x) 5 e x dx = γ 5 + 5γ3 π ( l x) 6 e x dx = γ 6 + 5γ4 π 3 + 3γπ4 4 + 9γ π 4 4 +γπ ζ 3 + 4ζ 3 + 44γζ 5 correspodig to the six first algebraic momets of g(z). Mascheroi) costat γ = lim ( k= + γ ζ 3 + π ζ 3 3 + 6π6 68 + 4γ3 ζ 3 + ) k l =.577 56649 53 866 65... ad ζ is a short had otatio for Riema s zeta-fuctio ζ() give by ζ(z) = k= (see also [3]). For z a eve iteger we may use k = x z dx for z > z Γ(z) e x + 4ζ 5 Here γ is Euler s (or Euler- ζ() = π B ()! for =,,... where B are the Beroulli umbers give by B =, B 6 4 =, B 3 6 =, B 4 8 = 3 etc (see table 4 o page 74 for a extesive table of the Beroulli umbers). This implies ζ = π, ζ 6 4 = π4, ζ 9 6 = π6 etc. 945 For odd iteger argumets o similar relatio as for eve itegers exists but evaluatig the sum of reciprocal powers the two umbers eeded i the calculatios above are give 59

by ζ 3 =.5 693 59594 854... ad ζ 5 =.369 7755 43369 9633... The umber ζ 3 is sometimes referred to as Apéry s costat after the perso who i 979 showed that it is a irratioal umber (but sofar it is ot kow if it is also trascedetal) [3]. Usig the algebraic momets of g(z) as give above we may fid the low order cetral momets of g(z) as µ = π 6 = ζ µ 3 = ζ 3 µ 4 = 3π 4 / µ 5 = π ζ 3 3 + 4ζ 5 µ 6 = 6π6 68 + 4ζ 3 ad thus the coefficiets of skewess γ ad kurtosis γ are give by γ = µ 3 /µ 3 = 6ζ 3 /π 3.3955 γ = µ 4 /µ 3 =.4 Algebraic momets of f(x) may be foud from this with some effort. Cetral momets are simpler beig coected to those for g(z) through the relatio µ (x) = (±) σ µ (z). I particular the expectatio value ad the variace of f(x) are give by E(x) = µ ± σe(z) = µ ± σγ V (x) = σ V (z) = σ π The coefficiets of skewess (except for a sig ±) ad kurtosis are the same as for g(z). 5.5 Radom Number Geeratio Usig the expressio for the cumulative distributio we may use a radom umber ξ, uiformly distributed betwee zero ad oe, to obtai a radom umber from the extreme value distributio by G(z) = e e z = ξ z = l( l ξ) which gives a radom umber from the ormalized fuctio g(z). A radom umber from f(x) is the easily obtaied by x = µ ± σz. 6 6

6 F-distributio 6. Itroductio The F -distributio is give by f(f ; m, ) = m m Γ ( ) m+ Γ ( ) ( ) m Γ F m (mf + ) m+ = m m B ( m, ) F m (mf + ) m+ where the parameters m ad are positive itegers, degrees of freedom ad the variable F is a positive real umber. The fuctios Γ ad B are the usual Gamma ad Beta fuctios. The distributio is ofte called the Fisher F -distributio, after the famous british statisticia Sir Roald Aylmer Fisher (89-96), sometimes the Sedecor F -distributio ad sometimes the Fisher-Sedecor F -distributio. I figure we show the F -distributio for low values of m ad. Figure : The F -distributio (a) for m = ad =,,..., ad (b) for m =,,..., ad = For m the distributio has its maximum at F = ad is mootoically decreasig. Otherwise the distributio has the mode at F mode = m m + This distributio is also kow as the variace-ratio distributio sice it, as will be show below, describes the distributio of the ratio of the estimated variaces from two idepedet samples from ormal distributios with equal variace. 6

6. Relatios to Other Distributios For m = we obtai a t -distributio, the distributio of the square of a variable distributed accordig to Studet s t-distributio. As the quatity mf approaches a chi-square distributio with m degrees of freedom. For large values of m ad the F -distributio teds to a ormal distributio. There are several approximatios foud i the literature all of which are better tha a simplemided stadardized variable. Oe is mf z = m + mf ad a eve better choice is z = F 3 ( 9 ) ( ) 9m + F 3 9m 9 For large values of m ad also the distributio i the variable z = l F, the distributio ( of which is kow as the Fisher z-distributio, is approximately ormal with mea ) ( m ad variace + ) m. This approximatio is, however, ot as good as z above. 6.3 /F If F is distributed accordig to the F -distributio with m ad degrees of freedom the F has the F -distributio with ad m degrees of freedom. This is easily verified by a chage of variables. Puttig G = we have F df f(g) = dg f(f ) = G m m B( m, ) ( m G ( G)m m+ + ) = m m B( m, ) G (m + G) m+ which is see to be idetical to a F -distributio with ad m degrees of freedom for G = F. 6.4 Characteristic Fuctio The characteristic fuctio for the F -distributio may be expressed i terms of the cofluet hypergeometric fuctio M (see sectio 43.3) as 6.5 Momets Algebraic momets are give by µ r = m m F m +r B( m, ) (mf + ) m+ φ(t) = E(e ıf t ) = M ( m, ; m ıt) df = 6 ( ) m m B( m, ) ( mf F m +r m+ + ) df =

= = ( m ) m B( m, ) ( u m ) m +r (u + ) m+ ( r Γ m) ( m + r) Γ ( r) Γ ( ) ( ) m Γ ( ) r m du = B( m + r, r) m B( m, ) = ad are defied for r <. This may be writte ( ) r m µ r = ( m + ) ( m + r ) m ( r)( r + ) ( ) a form which may be more coveiet for computatios especially whe m or are large. A recursive formula to obtai the algebraic momets would thus be ( µ r = µ r m) startig with µ =. The first algebraic momet, the mea, becomes ad the variace is give by E(F ) = m + r r for > V (F ) = (m + ) m( ) ( 4) for > 4 6.6 F-ratio Regard F = u/m where u ad v are two idepedet variables distributed accordig to the v/ chi-square distributio with m ad degrees of freedom, respectively. The idepedece implies that the joit probability fuctio i u ad v is give by the product of the two chi-square distributios f(u, v; m, ) = If we chage variables to x = u/m v/ ( u ) m e u Γ ( ) m ( v ) e v Γ ( ) ad y = v the distributio i x ad y becomes (u, v) f(x, y; m, ) = f(u, v; m, ) (x, y) The determiat of the Jacobia of the trasformatio is ym ( ) m xym e xym ) f(x, y; m, ) = ym m Γ ( m 63 ad thus we have y e y Γ ( )

Fially, sice we are iterested i the margial distributio i x we itegrate over y f(x; m, ) = = ( ) m m x m f(x, y; m, )dy = m+ Γ ( ) ( ) m Γ ( ) m m x m m+ Γ ( ) ( ) m Γ m+ Γ ( ) m+ ( m+ = xm + ) y m+ e y ( xm +) dy = ( m ) m B ( m, ) ( xm x m m+ + ) which with x = F is the F -distributio with m ad degrees of freedom. Here we used the itegral t z e αt dt = Γ(z) α z i simplifyig the expressio. 6.7 Variace Ratio A practical example where the F -distributio is applicable is whe estimates of the variace for two idepedet samples from ormal distributios s = m i= (x i x) m ad s = i= (y i y) have bee made. I this case s ad s are so called ormal theory estimates of σ ad σ i.e. (m )s /σ ad ( )s /σ are distributed accordig to the chi-square distributio with m ad degrees of freedom, respectively. I this case the quatity F = s σ is distributed accordig to the F -distributio with m ad degrees of freedom. If the true variaces of the two populatios are ideed the same the the variace ratio s /s have the F -distributio. We may thus use this ratio to test the ull hypothesis H : σ = σ versus the alterative H : σ σ usig the F -distributio. We would reject the ull hypotheses at the α cofidece level if the F -ratio is less tha F α/,m, or greater tha F α/,m, where F α,m, is defied by F α,m, σ s f(f ; m, )df = α i.e. α is the probability cotet of the distributio above the value F α,m,. Note that the followig relatio betwee F -values correspodig to the same upper ad lower cofidece levels is valid F α,m, = F α,,m 64

6.8 Aalysis of Variace As a simple example, which is ofte called aalysis of variace, we regard observatios of a depedet variable x with overall mea x divided ito k classes o a idepedet variable. The mea i each class is deoted x j for j =,,..., k. I each of the k classes there are j observatios together addig up to, the total umber of observatios. Below we deote by x ji the i:th observatio i class j. Rewrite the total sum of squares of the deviatios from the mea SS x = = = = k j k j (x ji x) = ((x ji x j ) + (x j x)) = j= i= j= i= k j [ (xji x j ) + (x j x) + (x ji x j )(x j x) ] = j= i= k j k j k j j= i= k j (x ji x j ) + (x ji x j ) + j= i= j= (x j x) + (x j x) j= i= j= i= k j (x j x) = SS withi + SS betwee (x ji x j ) = i.e. the total sum of squares is the sum of the sum of squares withi classes ad the sum of squares betwee classes. Expressed i terms of variaces k k V (x) = j V j (x) + j (x j x) j= j= If the variable x is idepedet o the classificatio the the variace withi groups ad the variace betwee groups are both estimates of the same true variace. The quatity F = SS betwee/(k ) SS withi /( k) is the distributed accordig to the F -distributio with k ad k degrees of freedom. This may the be used i order to test the hypothesis of o depedece. A too high F -value would be ulikely ad thus we ca choose a cofidece level at which we would reject the hypothesis of o depedece of x o the classificatio. Sometimes oe also defies η = SS betwee /SS x, the proportio of variace explaied, as a measure of the stregth of the effects of classes o the variable x. 6.9 Calculatio of Probability Cotet I order to set cofidece levels for the F -distributio we eed to evaluate the cumulative fuctio i.e. the itegral α = F α f(f ; m, )df 65

where we have used the otatio F α istead of F α,m, for coveiece. α = m m B( m, ) = ( m ) m B( m, ) F α mfα F m (mf + ) m+ ( u m ) m (u + ) m+ df = ( m ) m B( m, ) m du = B( m, ) F α ( mf mfα F m m+ + ) u m ( + u) m+ df = where we made the substitutio u = mf. The last itegral we recogize as the icomplete Beta fuctio B x defied for x as du B x (p, q) = x t p ( t) q dt = x x u p du ( + u) p+q where we made the substitutio u = t i.e. t = t u. We thus obtai +u α = B x( m, ) B( m, ) = I x( m, ) with x = mfα i.e. x = mfα. The variable x thus has a Beta distributio. Note that x +mf α also I x (a, b) is called the icomplete Beta fuctio (for historical reasos discussed below but see also sectio 4.7). 6.9. The Icomplete Beta fuctio I order to evaluate the icomplete Beta fuctio we may use the serial expasio B x (p, q) = x p [ p + q p + ( q)( q) x + x +... +!(p + ) ] ( q)( q) ( q) x +...!(p + ) For iteger values of q correspodig to eve values of the sum may be stopped at = q sice all remaiig terms will be idetical to zero i this case. We may express the sum with successive terms expressed recursively i the previous term B x (p, q) = x p t r with t r = t r r= x(r q)(p + r ) r(p + r) startig with t = p The sum ormally coverges quite fast but beware that e.g. for p = q = (m = = ) the covergece is very slow. Also some cases with q very big but p small seem pathological sice i these cases big terms with alterate sigs cacel each other causig roudoff problems. It seems preferable to keep q < p to assure faster covergece. This may be doe by usig the relatio B x (p, q) = B (q, p) B x (q, p) 66

which if iserted i the formula for α gives α = B (, m ) B x(, m ) B( m, ) α = B x(, m ) B( m, ) = I x (, m ) sice B (p, q) = B(p, q) = B(q, p). A umerically better way to evaluate the icomplete Beta fuctio I x (a, b) is by the cotiued fractio formula [] Here I x (a, b) = xa ( x) b ab(a, b) (a + m)(a + b + m)x d m+ = (a + m)(a + m + ) [ ] d d + + + ad d m = m(b m)x (a + m )(a + m) ad the formula coverges rapidly for x < (a + )/(a + b + ). For other x-values the same formula may be used after applyig the symmetry relatio 6.9. Fial Formulæ I x (a, b) = I x (b, a) Usig the serial expressio for B x give i the previous subsectio the probability cotet of the F-distributio may be calculated. The umerical situatio is, however, ot ideal. For iteger a- or b-values 3 the followig relatio to the biomial distributio valid for iteger values of a is useful ( ) a a + b I x (a, b) = I x (b, a) = x i ( x) a+b i i i= Our fial formulæ are take from [6], usig x = previous defiitio of x), +mf (ote that this is oe mius our Eve m: Eve : α = x [... + + α = ( x) m... + ( x) + ( + ) 4 ( x) +... ( + )... (m + 4) ( x) m 4... (m ) [ + m x + m(m + ) 4 m(m + )... (m + 4) x 4... ( ) ] x +... ] 3 If oly b is a iteger use the relatio I x (a, b) = I x (b, a). 67

Odd m ad : α = A + β with A = [ ( θ + si θ cos θ + π 3 cos3 θ +... )] 4... ( 3)... + 3... ( ) cos θ for > ad β = Γ ( ) + [ π Γ ( ) si θ cos θ + + si θ +... 3 ] ( + )( + 3)... (m + 4)... + si m 3 θ for m > where θ = arcta 3 5... ( ) ) F Γ m ad ( + Γ ( ) = If = the A = θ/π ad if m = the β =. ( )!! ( )!! For large values of m ad we use a approximatio usig the stadard ormal distributio where z = F ( ) ( ) 3 9 9m + F 3 9m 9 is approximately distributed accordig to the stadard ormal distributio. Cofidece levels are obtaied by α = π z e x dx I table 5 o page 75 we show some percetage poits for the F -distributio. Here is the degrees of freedom of the greater mea square ad m the degrees of freedom for the lesser mea square. The values express the values of F which would be exceeded by pure chace i %, 5% ad % of the cases, respectively. 6. Radom Number Geeratio Followig the defiitio the quatity F = y m/m y / where y ad y m are two variables distributed accordig to the chi-square distributio with ad mx degrees of freedom respectively follows the F-distributio. We may thus use this relatio isertig radom umbers from chi-square distributios (see sectio 8.7). π 68

7 Gamma Distributio 7. Itroductio The Gamma distributio is give by f(x; a, b) = a(ax) b e ax /Γ(b) where the parameters a ad b are positive real quatities as is the variable x. Note that the parameter a is simply a scale factor. For b the distributio is J-shaped ad for b > it is uimodal with its maximum at x = b. a I the special case where b is a positive iteger this distributio is ofte referred to as the Erlagia distributio. For b = we obtai the expoetial distributio ad with a = ad b = with a iteger we obtai the chi-squared distributio with degrees of freedom. I figure 3 we show the Gamma distributio for b-values of ad 5. Figure 3: Examples of Gamma distributios 7. Derivatio of the Gamma Distributio For iteger values of b, i.e. for Erlagia distributios, we may derive the Gamma distributio from the Poisso assumptios. For a Poisso process where evets happe at a rate of λ the umber of evets i a time iterval t is give by Poisso distributio P (r) = (λt)r e λt r! 69

The probability that the k:th evet occur at time t is the give by k r= P (r) = k r= (λt) r e λt i.e. the probability that there are at least k evets i the time t is give by k (λt) r e λt F (t) = P (r) = = r=k r= r! λt r! z k e z (k )! dz = t λ k z k e λz dz (k )! where the sum has bee replaced by a itegral (o proof give here) ad the substitutio z = λz made at the ed. This is the cumulative Gamma distributio with a = λ ad b = k, i.e. the time distributio for the k:th evet follows a Gamma distributio. I particular we may ote that the time distributio for the occurrece of the first evet follows a expoetial distributio. The Erlagia distributio thus describes the time distributio for expoetially distributed evets occurrig i a series. For expoetial processes i parallel the appropriate distributio is the hyperexpoetial distributio. 7.3 Momets The distributio has expectatio value, variace, third ad fourth cetral momets give by E(x) = b a, V (x) = b a, µ 3 = b a, ad µ 3b( + b) 3 4 = a 4 The coefficiets of skewess ad kurtosis is give by γ = b ad γ = 6 b More geerally algebraic momets are give by µ = = ab Γ(b) x f(x)dx = ( y a ab x +b e ax dx = Γ(b) ) +b e y dy a = Γ( + b) a Γ(b) b(b + ) (b + ) = a where we have made the substitutio y = ax i simplifyig the itegral. 7.4 Characteristic Fuctio The characteristic fuctio is φ(t) = E(e ıtx ) = ab Γ(b) = ab Γ(b) (a ıt) b 7 x b e x(a ıt) dx = y b e y dy = = ( ıt ) b a

where we made the trasformatio y = x(a ıt) i evaluatig the itegral. 7.5 Probability Cotet I order to calculate the probability cotet for a Gamma distributio we eed the cumulative (or distributio) fuctio F (x) = x = ab Γ(b) f(x)dx = ax ( v a ab x u b e au du = Γ(b) ) b e v dv a = ax v b e v dv = Γ(b) γ(b, ax) Γ(b) where γ(b, ax) deotes the icomplete gamma fuctio 4. 7.6 Radom Number Geeratio 7.6. Erlagia distributio I the case of a Erlagia distributio (b a positive iteger) we obtai a radom umber by addig b idepedet radom umbers from a expoetial distributio i.e. x = l(ξ ξ... ξ b )/a where all the ξ i are uiform radom umbers i the iterval from zero to oe. Note that care must be take if b is large i which case the product of uiform radom umbers may become zero due to machie precisio. I such cases simply divide the product i pieces ad add the logarithms afterwards. 7.6. Geeral case I a more geeral case we use the so called Johk s algorithm i Deote the iteger part of b with i ad the fractioal part with f ad put r =. Let ξ deote uiform radom umbers i the iterval from zero to oe. ii If i > the put r = l(ξ ξ... ξ i ). iii If f = the go to vii. iv Calculate w = ξ /f i+ ad w = ξ /( f) i+. v If w + w > the go back to iv. vi Put r = r l(ξ i+3 ) w w +w. vii Quit with r = r/a. 4 Whe itegrated from zero to x the icomplete gamma fuctio is ofte deoted by γ(a, x) while for the complemet, itegrated from x to ifiity, it is deoted Γ(a, x). Sometimes the ratio P (a, x) = γ(a, x)/γ(a) is called the icomplete Gamma fuctio. 7

7.6.3 Asymptotic Approximatio For b big, say b > 5, we may use the Wilso-Hilferty approximatio: i Calculate q = + + 9b distributio. z 3 b where z is a radom umber from a stadard ormal ii Calculate r = b q 3. iii If r < the go back to i. iv Quit with r = r/a. 7

8 Geeralized Gamma Distributio 8. Itroductio The Gamma distributio is ofte used to describe variables bouded o oe side. A eve more flexible versio of this distributio is obtaied by addig a third parameter givig the so called geeralized Gamma distributio f(x; a, b, c) = ac(ax) bc e (ax)c /Γ(b) where a (a scale parameter) ad b are the same real positive parameters as is used for the Gamma distributio but a third parameter c has bee added (c = for the ordiary Gamma distributio). This ew parameter may i priciple take ay real value but ormally we cosider the case where c > or eve c. Put c i the ormalizatio for f(x) if c <. Accordig to Hegyi [33] this desity fuctio first appeared i 95 whe L. Amoroso used it i aalyzig the distributio of ecoomic icome. Later it has bee used to describe the sizes of grais produced i commiutio ad drop size distributios i sprays etc. I figure 4 we show the geeralized Gamma distributio for differet values of c for the case a = ad b =. Figure 4: Examples of geeralized Gamma distributios 8. Cumulative Fuctio The cumulative fuctio is give by F (x) = where P is the icomplete Gamma fuctio. { γ (b, (ax) c ) /Γ(b) = P (b, (ax) c ) if c > Γ (b, (ax) c ) /Γ(b) = P (b, (ax) c ) if c < 73

8.3 Momets Algebraic momets are give by µ = a Γ ( b + c Γ(b) ) For egative values of c the momets are fiite for raks satisfyig /c > b (or eve just avoidig the sigularities +,,...). a c 8.4 Relatio to Other Distributios The geeralized Gamma distributio is a geeral form which for certai parameter combiatios gives may other distributios as special cases. I the table below we idicate some such relatios. For otatios see the correspodig sectio. Distributio a b c Sectio Geeralized gamma a b c 8 Gamma a b 7 Chi-squared 8 Expoetial α 4 Weibull σ η 4 Rayleigh α Maxwell Stadard ormal (folded) α 3 37 5 34 I referece [33], where this distributio is used i the descriptio of multiplicity distributios i high eergy particle collisios, more examples o special cases as well as more details regardig the distributio are give. 74

9 Geometric Distributio 9. Itroductio The geometric distributio is give by p(r; p) = p( p) r where the iteger variable r ad the parameter < p < (o eed to iclude limits sice this give trivial special cases). It expresses the probability of havig to wait exactly r trials before the first successful evet if the probability of a success i a sigle trial is p (probability of failure q = p). It is a special case of the egative biomial distributio (with k = ). 9. Momets The expectatio value, variace, third ad fourth momet are give by E(r) = p V (r) = p p µ 3 = The coefficiets of skewess ad kurtosis is thus γ = ( p)( p) µ p 3 4 = ( p)(p 9p + 9) p 4 p ad γ = p 6p + 6 p p 9.3 Probability Geeratig Fuctio The probability geeratig fuctio is G(z) = E(z r ) = z r p( p) r = r= 9.4 Radom Number Geeratio The cumulative distributio may be writte pz qz k P (k) = p(r) = q k with q = p r= which ca be used i order to obtai a radom umber from a geometric distributio by geeratig uiform radom umbers betwee zero ad oe util such a umber (the k:th) is above q k. A more straightforward techique is to geerate uiform radom umbers ξ i util we fid a success where ξ k p. These two methods are both very iefficiet for low values of p. However, the first techique may be solved explicitly k r= P (r) = ξ k = l ξ l q 75

which implies takig the largest iteger less tha k+ as a radom umber from a geometric distributio. This method is quite idepedet of the value of p ad we foud [4] that a reasoable breakpoit below which to use this techique is p =.7 ad use the first method metioed above this limit. With such a method we do ot gai by creatig a cumulative vector for the radom umber geeratio as we do for may other discrete distributios. 76

Hyperexpoetial Distributio. Itroductio The hyperexpoetial distributio describes expoetial processes i parallel ad is give by f(x; p, λ, λ ) = pλ e λ x + qλ e λ x where the variable x ad the parameters λ ad λ are positive real quatities ad p is the proportio for the first process ad q = p the proportio of the secod. The distributio describes the time betwee evets i a process where the evets are geerated from two idepedet expoetial distributios. For expoetial processes i series we obtai the Erlagia distributio (a special case of the Gamma distributio). The hyperexpoetial distributio is easily geeralized to the case with k expoetial processes i parallel k f(x) = p i λ i e λ ix i= where λ i is the slope ad p i the proportio for each process (with the costrait that pi = ). The cumulative (distributio) fuctio is F (x) = p ( e λ x ) + q ( e λ x ) ad it is thus straightforward to calculate the probability cotet i ay give situatio.. Momets Algebraic momets are give by ( p µ =! λ + q ) λ Cetral momets becomes somewhat complicated but the secod cetral momet, the variace of the distributio, is give by µ = V (x) = p λ + q ( + pq ) λ λ λ.3 Characteristic Fuctio The characteristic fuctio of the hyperexpoetial distributio is give by φ(t) = p ıt λ + q ıt λ 77

.4 Radom Number Geeratio Geeratig two uiform radom umbers betwee zero ad oe, ξ ad ξ, we obtai a radom umber from a hyperexpoetial distributio by If ξ p the put x = l ξ λ. If ξ > p the put x = l ξ λ. i.e. usig ξ we choose which of the two processes to use ad with ξ we geerate a expoetial radom umber for this process. The same techique is easily geeralized to the case with k processes. 78

Hypergeometric Distributio. Itroductio The Hypergeometric distributio is give by p(r;, N, M) = ( )( ) M N M r r ( N ) where the discrete variable r has limits from max(, N + M) to mi(, M) (iclusive). The parameters ( N), N (N ) ad M (M ) are all itegers. This distributio describes the experimet where elemets are picked at radom without replacemet. More precisely, suppose that we have N elemets out of which M has a certai attribute (ad N M has ot). If we pick elemets at radom without replacemet p(r) is the probability that exactly r of the selected elemets come from the group with the attribute. If N this distributio approaches a biomial distributio with p = M. N If istead of two groups there are k groups with differet attributes the geeralized hypergeometric distributio k p(r;, N, M) = ( ) Mi r i= i ( N ) where, as before, N is the total umber of elemets, the umber of elemets picked ad M a vector with the umber of elemets of each attribute (whose sum should equal N). Here = r i ad the limits for each r k is give by max(, N +M k ) r k mi(, M k ).. Probability Geeratig Fuctio The Hypergeometric distributio is closely related to the hypergeometric fuctio, see appedix B o page 67, ad the probability geeratig fuctio is give by ) G(z) = ( N M ( N ) F (, M; N M +; z).3 Momets With the otatio p = M ad q = p, i.e. the proportios of elemets with ad without N the attribute, the expectatio value, variace, third ad fourth cetral momets are give by E(r) = p V (r) = pq N N µ 3 = pq(q p) (N )(N ) (N )(N ) µ 4 = pq(n ) N(N + ) 6(N ) + 3pq(N ( ) N + 6(N )) (N )(N )(N 3) 79

For the geeralized hypergeometric distributio usig p i = M i /N ad q i = p i we fid momets of r i usig the formulæ above regardig the group i as havig a attribute ad all other groups as ot havig the attribute. the covariaces are give by Cov(r i, r j ) = p i p j N N.4 Radom Number Geeratio To geerate radom umbers from a hypergeometric distributio oe may costruct a routie which follow the recipe above by pickig elemets at radom. The same techique may be applied for the geeralized hypergeometric distributio. Such techiques may be sufficiet for may purposes but become quite slow. For the hypergeometric distributio a better choice is to costruct the cumulative fuctio by addig up the idividual probabilities usig the recursive formula p(r) = (M r + )( r + ) p(r ) r(n M + r) for the appropriate r-rage (see above) startig with p(r mi ). With the cumulative vector ad oe sigle uiform radom umber oe may easily make a fast algorithm i order to obtai the required radom umber. 8

Logarithmic Distributio. Itroductio The logarithmic distributio is give by ( p)r p(r; p) = r l p where the variable r is a iteger ad the parameter < p < is a real quatity. It is a limitig form of the egative biomial distributio whe the zero class has bee omitted ad the parameter k (see sectio 9.4.3).. Momets The expectatio value ad variace are give by E(r) = αq p ad αq( + αq) V (r) = p where we have itroduced q = p ad α = / l p for coveiece. The third ad fourth cetral momets are give by µ 3 = αq ( + q + 3αq + α q ) p 3 µ 4 = αq ( + 4q + q + 4αq( + q) + 6α q + 3α 3 q 3) p 4 More geerally factorial momets are easily foud usig the probability geeratig fuctio E(r(r ) (r k + )) = dk dz G(z) k = ( )!α qk z= p k From these momets ordiary algebraic ad cetral momets may be foud by straightforward but somewhat tedious algebra..3 Probability Geeratig Fuctio The probability geeratig fuctio is give by G(z) = E(z r ) = r= zr ( p) r r l p = l p (zq) r r= r = l( zq) l( q) where q = p ad sice l( x) = ( ) x + x + x3 3 + x4 4 +... for x < 8

.4 Radom Number Geeratio The most straightforward way to obtai radom umbers from a logarithmic distributio is to use the cumulative techique. If p is fixed the most efficiet way is to prepare a cumulative vector startig with p() = αq ad subsequet elemets by the recursive formula p(i) = p(i )q/i. The cumulative vector may, however, become very log for small values of p. Ideally it should exted util the cumulative vector elemet is exactly oe due to computer precisio. It p is ot fixed the same procedure has to be made at each geeratio. 8

3 Logistic Distributio 3. Itroductio The Logistic distributio is give by f(x; a, k) = e z with z = x a k( + e z ) k where the variable x is a real quatity, the parameter a a real locatio parameter (the mode, media, ad mea) ad k a positive real scale parameter (related to the stadard deviatio). I figure 5 the logistic distributio with parameters a= ad k = (i.e. z =x) is show. Figure 5: Graph of logistic distributio for a = ad k = 3. Cumulative Distributio The distributio fuctio is give by F (x) = + e = z + e = z + e x a k The iverse fuctio is foud by solvig F (x)=α givig ( ) α x = F (α) = a k l α from which we may fid e.g. the media as M=a. Similarly the lower ad upper quartiles are give by Q, =a k l 3. 83

3.3 Characteristic Fuctio The characteristic fuctio is give by φ(t) = E(e ıtx ) = e ıtx e x a k k ( + e x a k ) dx = e ıta = e ıta y ıtk y ( + y) dy y = eıta B(+ıtk, ıtk) = ıta Γ(+ıtk)Γ( ıtk) = e Γ() e ıtzk e z k( + e z ) kdz = = e ıta ıtkγ(ıtk)γ( ıtk) = e ıta ıtkπ si πıtk where we have used the trasformatios z =(x a)/k ad y =e z i simplifyig the itegral, at the ed idetifyig the beta fuctio, ad usig relatio of this i terms of Gamma fuctios ad their properties (see appedix A i sectio 4). 3.4 Momets The characteristic fuctio is slightly awkward to use i determiig the algebraic momets by takig partial derivatives i t. However, usig l φ(t) = ıta + l Γ(+ıtk) + l Γ( ıtk) we may determie the cumulats of the distributios. I the process we take derivatives of l φ(t) which ivolves polygamma fuctios (see sectio 4.4) but all of them with argumet whe isertig t = a case which may be explicitly writte i terms of Riema s zetafuctios with eve real argumet (see page 59). It is quite easily foud that all cumulats of odd order except κ = a vaish ad that for eve orders κ = k ψ ( ) () = ( )!k ζ() = ( )!k π B ()! for =,,... ad where B are the Beroulli umbers (see table 4 o page 74). Usig this formula lower order momets ad the coefficiets of skewess ad kurtosis is foud to be µ = E(x) = κ = a µ = V (x) = κ = k π /3 µ 3 = µ 4 = κ 4 + 3κ = k4 π 4 µ 5 = 5 + k4 π 4 3 = 7k4 π 4 5 µ 6 = κ 6 + 5κ 4 κ + κ 3 + 5κ 3 = = 6k6 π 6 + k6 π 6 + 5k6 π 6 = 3k6 π 6 63 3 7 γ = γ =. (exact) 84

3.5 Radom umbers Usig the iverse cumulative fuctio oe easily obtais a radom umber from a logistic distributio by ( ) ξ x = a + k l ξ with ξ a uiform radom umber betwee zero ad oe (limits ot icluded). 85

4 Log-ormal Distributio 4. Itroductio The log-ormal distributio or is give by f(x; µ, σ) = xσ ( π e l x µ σ ) where the variable x > ad the parameters µ ad σ > all are real umbers. It is sometimes deoted Λ(µ, σ ) i the same spirit as we ofte deote a ormally distributed variable by N(µ, σ ). If u is distributed as N(µ, σ ) ad u = l x the x is distributed accordig to the log-ormal distributio. Note also that if x has the distributio Λ(µ, σ ) the y = e a x b is distributed as Λ(a + bµ, b σ ). I figure 6 we show the log-ormal distributio for the basic form, with µ = ad σ =. Figure 6: Log-ormal distributio The log-ormal distributio is sometimes used as a first approximatio to the Ladau distributio describig the eergy loss by ioizatio of a heavy charged particle (cf also the Moyal distributio i sectio 6). 4. Momets The expectatio value ad the variace of the distributio are give by σ µ+ E(x) = e ad V (x) = e ( µ+σ e σ ) 86

ad the coefficiets of skewess ad kurtosis becomes γ = e σ ( e σ + ) ad γ = ( e σ ) ( e 3σ + 3e σ + 6e σ + 6 ) More geerally algebraic momets of the log-ormal distributio are give by µ k = E(x k ) = = σ π σ k σ π ekµ+ x k e l x µ ( σ ) dx = σ π ( e y µ kσ σ ) dy = e kµ+ k σ e yk e ( y µ σ ) dy = where we have used the trasformatio y = l x i simplifyig the itegral. 4.3 Cumulative Distributio The cumulative distributio, or distributio fuctio, for the log-ormal distributio is give by F (x) = x σ ( π t e ) = ± P (, z l t µ σ ) dt = σ π l x e ( y µ σ ) dy = where we have put z = (l x µ)/σ ad the positive sig is valid for z ad the egative sig for z <. 4.4 Radom Number Geeratio The most straightforward way of achievig radom umbers from a log-ormal distributio is to geerate a radom umber u from a ormal distributio with mea µ ad stadard deviatio σ ad costruct r = e u. 87

5 Maxwell Distributio 5. Itroductio The Maxwell distributio is give by f(x; α) = α 3 π x e x α where the variable x with x ad the parameter α with α > are real quatities. It is amed after the famous scottish physicist James Clerk Maxwell (83 879). The parameter α is simply a scale factor ad the variable y = x/α has the simplified distributio g(y) = π y e y Figure 7: The Maxwell distributio The distributio, show i figure 7, has a mode at x = α ad is positively skewed. 5. Momets Algebraic momets are give by E(x ) = x f(x)dx = α 3 π x + e x /α i.e. we have a coectio to the absolute momets of the Gauss distributio. Usig these (see sectio o the ormal distributio) the result is { E(x ) = π k k!α k for = k ( + )!!α for eve 88

Specifically we ote that the expectatio value, variace, ad the third ad fourth cetral momets are give by E(x) = α π, V (x) = α The coefficiets of skewess ad kurtosis is thus ( 3 8 ) ( ) ( 6, µ 3 = α 3 π π 5 π, ad µ 4 = α 4 5 8 ) π γ = ( 6 π 5) π ( ) 3 3 8 π.48569 ad γ = 5 8 π ( ) 3.88 3 8 π 5.3 Cumulative Distributio The cumulative distributio, or the distributio fuctio, is give by F (x) = x f(y)dy = a 3 π x y e y α dy = π x α ze z dz = γ ( 3, ) x α Γ ( 3 ) = P ( 3, x ) α where we have made the substitutio z = y i order to simplify the itegratio. Here α P (a, x) is the icomplete Gamma fuctio. Usig the above relatio we may estimate the media M ad the lower ad upper quartile, Q ad Q 3, as Q = α P ( 3, ).5 α M = α P ( 3, ).5387 α Q 3 = α P ( 3, ).69 α where P (a, p) deotes the iverse of the icomplete Gamma fuctio i.e. the value x for which P (a, x) = p. 5.4 Kietic Theory The followig is take from kietic theory, see e.g. [34]. Let v = (v x, v y, v z ) be the velocity vector of a particle where each compoet is distributed idepedetly accordig to ormal distributios with zero mea ad the same variace σ. First costruct w = v σ = v x σ + v y σ + v z σ Sice v x /σ, v y /σ, ad v z /σ are distributed as stadard ormal variables the sum of their squares has the chi-squared distributio with 3 degrees of freedom i.e. g(w) = w π e w/ which leads to ( ) dw v v f(v) = g(w) dv = g σ σ = σ 3 which we recogize as a Maxwell distributio with α = σ. 89 π v e v σ

I kietic theory σ = kt/m, where k is Boltzma s costat, T the temperature, ad m the mass of the particles, ad we thus have m 3 f(v) = πk 3 T 3 v e mv kt The distributio i kietic eergy E = mv / becomes 4E g(e) = πk 3 T 3 e which is a Gamma distributio with parameters a = /kt ad b = 3. 5.5 Radom Number Geeratio To obtai radom umbers from the Maxwell distributio we first make the trasformatio y = x /α a variable which follow the Gamma distributio g(y) = ye y /Γ ( ) 3. A radom umber from this distributio may be obtaied usig the so called Johk s algorithm which i this particular case becomes (deotig idepedet pseudoradom umbers from a uiform distributio from zero to oe by ξ i ) i Put r = l ξ i.e. a radom umber from a expoetial distributio. ii Calculate w = ξ ad w = ξ 3 (with ew uiform radom umbers ξ ad ξ 3 each iteratio, of course). iii If w = w + w > the go back to ii above. iv Put r = r w w l ξ 4 v Fially costruct a r as a radom umber from the Maxwell distributio with parameter r. Followig the examples give above we may also use three idepedet radom umbers from a stadard ormal distributio, z, z, ad z 3, ad costruct r = α E kt z + z + z 3 However, this techique is ot as efficiet as the oe outlied above. As a third alterative we could also use the cumulative distributio puttig F (x) = ξ P ( 3, x α ) = ξ x = α P ( 3, ξ) where P (a, p), as above, deotes the value x where P (a, x) = p. This techique is, however, much slower tha the alteratives give above. The first techique described above is ot very fast but still the best alterative preseted here. Also it is less depedet o umerical algorithms (such as those to fid the iverse of the icomplete Gamma fuctio) which may affect the precisio of the method. 9

6 Moyal Distributio 6. Itroductio The Moyal distributio is give by f(z) = exp { ( )} π z + e z for real values of z. A scale shift ad a scale factor is itroduced by makig the stadardized variable z = (x µ)/σ ad hece the distributio i the variable x is give by g(x) = ( ) x µ σ f σ Without loss of geerality we treat the Moyal distributio i its simpler form, f(z), i this documet. Properties for g(x) are easily obtaied from these results which is sometimes idicated. The Moyal distributio is a uiversal form for (a) the eergy loss by ioizatio for a fast charged particle ad (b) the umber of io pairs produced i this process. It was proposed by J. E. Moyal [35] as a good approximatio to the Ladau distributio. It was also show that it remais valid takig ito accout quatum resoace effects ad details of atomic structure of the absorber. Figure 8: The Moyal distributio The distributio, show i figure 8, has a mode at z = ad is positively skewed. This implies that the mode of the x distributio, g(x), is equal to the parameter µ. 9

6. Normalizatio Makig the trasformatio x = e z we fid that f(z)dz = = exp { ( l x + dx π x)} x = π e x x dx = e y dy = e y dy = Γ ( ) = π y π y π where we have made the simple substitutio y = x/ i order to clearly recogize the Gamma fuctio at the ed. The distributio is thus properly ormalized. 6.3 Characteristic Fuctio The characteristic fuctio for the Moyal distributio becomes φ(t) = E(e ıtz ) = π = ( ıt) π e ıtz e (z+e z ) dz = (x) ( ıt) e x dx π x = x (+ıt) e x dx = ıt π Γ ( ıt) where we made the substitutio x = e z / i simplifyig the itegral. The last relatio to the Gamma fuctio with complex argumet is valid whe the real part of the argumet is positive which ideed is true i the case at had. 6.4 Momets As i some other cases the most coveiet way to fid the momets of the distributio is via its cumulats (see sectio.5). We fid that κ = l ψ( ) = l + γ κ = ( ) ψ ( ) ( ) = ( )!( )ζ for with γ.57756649 Euler s costat, ψ () polygamma fuctios (see sectio 4.4) ad ζ Riema s zeta-fuctio (see page 59). Usig the cumulats we fid the lower order momets ad the coefficiets of skewess ad kurtosis to be µ = E(z) = κ = l + γ.736 µ = V (z) = κ = ψ () ( ) = π 4.9348 µ 3 = κ 3 = ψ () ( ) = 4ζ 3 µ 4 = κ 4 + 3κ = ψ (3) ( ) + 3ψ() ( ) = 7π4 4 γ = 8 ζ 3 π 3.5354 γ = 4 9

For the distributio g(x) we have E(x) = σe(z) + µ, V (x) = σ V (z) or more geerally cetral momets are obtaied by µ (x) = σ µ (z) for while γ ad γ are idetical. 6.5 Cumulative Distributio Usig the same trasformatios as was used above i evaluatig the ormalizatio of the distributio we write the cumulative (or distributio) fuctio as F (Z) = Z = π f(z)dz = π e Z / Z e y dy = ( Γ y π, e Z exp { ( )} z + e z = π e Z ) = Γ (, ) e Z Γ ( ) = P e x x dx = ( ), e Z where P is the icomplete Gamma fuctio. Usig the iverse of the cumulative fuctio we fid the media M.7876 ad the lower ad upper quartiles Q -.83 ad Q 3.8739. 6.6 Radom Number Geeratio To obtai radom umbers from the Moyal distributio we may either make use of the iverse to the icomplete Gamma fuctio such that give a pseudoradom umber ξ we get a radom umber by solvig the equatio P ( ), e z = ξ for z. If P (a, p) deotes the value x where P (a, x) = p the z = l { P (, ξ)} is a radom umber from a Moyal distributio. This is, however, a very slow method ad oe may istead use a straightforward rejectaccept (or hit-miss) method. To do this we prefer to trasform the distributio to get it ito a fiite iterval. For this purpose we make the trasformatio ta y = x givig h(y) = f(ta y) cos y = { π cos y exp ( )} ta y + e ta y This distributio, show i figure 9, has a maximum of about.9 ad is limited to the iterval π y π. A simple algorithm to get radom umbers from a Moyal distributio, either f(z) or g(x), usig the reject-accept techique is as follows: a Get ito ξ ad ξ two uiform radom umbers uiformly distributed betwee zero ad oe usig a good basic pseudoradom umber geerator. 93

Figure 9: Trasformed Moyal distributio b Calculate uiformly distributed variables alog the horizotal ad vertical directio by y = πξ π ad h = ξ h max where h max =.9 is chose slightly larger tha the maximum value of the fuctio. c Calculate z = ta y ad the fuctio value h(y). d If h h(y) the accept z as a radom umber from the Moyal distributio f(z) else go back to poit a above. e If required the scale ad shift the result by x = zσ + µ i order to obtai a radom umber from g(x). This method is easily improved e.g. by makig a more tight evelope to the distributio tha a uiform distributio. The efficiecy of the reject-accept techique outlied here is oly /.9π.35 (the ratio betwee the area of the curve ad the uiform distributio). The method seems, however, fast eough for most applicatios. 94

7 Multiomial Distributio 7. Itroductio The Multiomial distributio is give by p(r; N, k, p) = N! r!r! r k! pr p r p r k k = N! where the variable r is a vector with k iteger elemets for which r i N ad r i = N. The parameters N > ad k > are itegers ad p is a vector with elemets p i with the costrait that p i =. The distributio is a geeralizatio of the Biomial distributio (k = ) to may dimesios where, istead of two groups, the N elemets are divided ito k groups each with a probability p i with i ragig from to k. A commo example is a histogram with N etries i k bis. 7. Histogram The histogram example is valid whe the total umber of evets N is regarded as a fixed umber. The variace i each bi the becomes, see also below, V (r i ) = Np i ( p ) r i if p i which ormally is the case for a histogram with may bis. If, however, we may regard the total umber of evets N as a radom variable distributed accordig to the Poisso distributio we fid: Give a multiomial distributio, here deoted M(r; N, p), for the distributio of evets ito bis for fixed N ad a Poisso distributio, deoted P (N; ν), for the distributio of N we write the joit distributio P(r, N) = M(r; N, p)p (N; ν) = = ( ) ( r! (νp ) r e νp r! (νp ) r e νp ( k i= N! r!r!... r k! pr p r... p r k k p r i i r i! ) ( ν N e ν ) = N! ) ) (... r k! (νp k) r k e νp k where we have used that k k p i = ad r i = N i= i= i.e. we get a product of idepedet Poisso distributios with meas νp i for each idividual bi. As see, i both cases, we fid justificatio for the ormal rule of thumb to assig the square root of the bi cotets as the error i a certai bi. Note, however, that i priciple we should isert the true value of r i for this error. Sice this ormally is ukow we use the observed umber of evets i accordace with the law of large umbers. This meas that cautio must be take i bis with few etries. 7.3 Momets For each specific r i we may obtai momets usig the Biomial distributio with q i = p i E(r i ) = Np i ad V (r i ) = Np i ( p i ) = Np i q i 95

The covariace betwee two groups are give by Cov(r i, r j ) = Np i p j for i j 7.4 Probability Geeratig Fuctio The probability geeratig fuctio for the multiomial distributio is give by ( k ) N G(z) = p i z i i= 7.5 Radom Number Geeratio The straightforward but time cosumig way to geerate radom umbers from a multiomial distributio is to follow the defiitio ad geerate N uiform radom umbers which are assiged to specific bis accordig to the cumulative value of the p-vector. 7.6 Sigificace Levels To determie a sigificace level for a certai outcome from a multiomial distributio oe may add all outcomes which are as likely or less likely tha the probability of the observed outcome. This may be a o-trivial calculatio for large values of N sice the umber of possible outcomes grows very fast. A alterative, although quite clumsy, is to geerate a umber of multiomial radom umbers ad evaluate how ofte these outcomes are as likely or less likely tha the observed oe. If we as a example observe the outcome r = (4,,,,, ) for a case with 5 observatios i 6 groups (N = 5 ad k = 6) ad the probability for all groups are the same p i = /k = /6 we obtai a probability of p.. This icludes all orderigs of the same outcome sice these are all equally probable but also all less likely outcomes of the type p = (5,,,,, ). If a probability calculated i this maer is too small oe may coclude that the ull hypothesis that all probabilities are equal is wrog. Thus if our cofidece level is preset to 95% this coclusio would be draw i the above example. Of course, the coclusio would be wrog i % of all cases. 7.7 Equal Group Probabilities A commo case or ull hypothesis for a multiomial distributio is that the probability of the k groups is the same i.e. p = /k. I this case the multiomial distributio is simplified ad sice orderig become isigificat much fewer uique outcomes are possible. Take as a example a game where five dices are throw. The probabilities for differet outcomes may quite readily be evaluated from basic probability theory properly accoutig for the 6 5 = 7776 possible outcomes. But oe may also use the multiomial distributio with k = 6 ad N = 5 to fid probabilities for differet outcomes. If we properly take care of combiatorial coefficiets for each outcome we obtai (with zeros for empty groups suppressed) 96

ame outcome # combiatios probability oe doublet,,, 36.4696 two doublets,, 8.348 triplets 3,,.543 othig,,,, 7.959 full house 3, 3.3858 quadruplets 4, 5.99 quituplets 5 6.77 total 7776. The experieced dice player may ote that the othig group icludes 4 combiatios givig straights ( to 5 or to 6). From this table we may verify the statemet from the previous subsectio that the probability to get a outcome with quadruplets or less likely outcomes is give by.6. Geerally we have for N < k that the two extremes of either all observatios i separate groups p sep or all observatios i oe group p all p sep = p all = k! k N (k N)! = k k k k k N k N + k which we could have cocluded directly from a quite simple probability calculatio. The first case is the formula which shows the quite well kow fact that if 3 people or more are gathered the probability that at least two have the same birthday, i.e. p sep, is greater tha 5% (usig N = 3 ad k = 365 ad ot botherig about leap-years or possible deviatios from the hypothesis of equal probabilities for each day). This somewhat oituitive result becomes eve more proouced for higher values of k ad the level above which p sep <.5 is approximately give by N. k For higher sigificace levels we may ote that i the case with k = 365 the probability p sep becomes greater tha 9% at N = 4, greater tha 99% at N = 57 ad greater tha 99.9% at N = 7 i.e. already for N << k a bet would be almost certai. I Fig. we show, i liear scale to the left ad logarithmic scale to the right, the lower limit o N for which the probability to have p sep above 5%, 9%, 99% ad 99.9.% for k-values ragig up to. By use of the gamma fuctio the problem has bee geeralized to real umbers. Note that the curves start at certai values where k = N sice for N > k it is impossible to have all evets i separate groups 5. 5 This limit is at N = k = for the 5%-curve, 3.9659 for 9%, 6.476 for 99% ad 8.9377 for 99.9% 97

Figure : Limits for N at several cofidece levels as a fuctio of k (liear scale to the left ad logarithmic scale to the right). 98

8 Multiormal Distributio 8. Itroductio As a geeralizatio of the ormal or Gauss distributio to may dimesios we defie the multiormal distributio. A multiormal distributio i x = {x, x,..., x } with parameters µ (mea vector) ad V (variace matrix) is give by f(x µ, V ) = e (x µ)v (x µ) T (π) V The variace matrix V has to be a positive semi-defiite matrix i order for f to be a proper probability desity fuctio (ecessary i order that the ormalizatio itegral f(x)dx should coverge). If x is ormal ad V o-sigular the (x µ)v (x µ) T is called the covariace form of x ad has a χ -distributio with degrees of freedom. Note that the distributio has costat probability desity for costat values of the covariace form. The characteristic fuctio is give by where t is a vector of legth. φ(t) = e ıtµ tt V t 8. Coditioal Probability Desity The coditioal desity for a fixed value of ay x i is give by a multiormal desity with dimesios where the ew variace matrix is obtaied by deletig the i:th row ad colum of V ad ivertig the resultig matrix. This may be compared to the case where we istead just wat to eglect oe of the variables x i. I this case the remaiig variables has a multiormal distributio with dimesios with a variace matrix obtaied by deletig the i:th row ad colum of V. 8.3 Probability Cotet As discussed i sectio 6.6 o the biormal distributio the joit probability cotet of a multidimesioal ormal distributio is differet, ad smaller, tha the correspodig well kow figures for the oe-dimesioal ormal distributio. I the case of the biormal distributio the ellipse (see figure o page ) correspodig to oe stadard deviatio has a joit probability cotet of 39.3%. The same is eve more true for the probability cotet withi the hyperellipsoid i the case of a multiormal distributio. I the table below we show, for differet dimesios, the probability cotet for the oe (deoted z = ), two ad three stadard deviatio cotours. We also give z-values z, z, ad z 3 adjusted to give a probability cotet withi the hyperellipsoid correspodig to the oe-dimesioal oe, two, ad three stadard deviatio cotets ( 68.3%, 95.5%, ad 99.7%). Fially z-value correspodig to joit probability 99

cotets of 9%, 95% ad 99% i z 9, z 95, ad z 99, respectively, are give. Note that these probability cotets are idepedet of the variace matrix which oly has the effect to chage the shape of the hyperellipsoid from a perfect hypersphere with radius z whe all variables are ucorrelated to e.g. cigar shapes whe correlatios are large. Note that this has implicatios o errors estimated from a chi-square or a maximum likelihood fit. If a multiparameter cofidece limit is requested ad the chi-square miimum is at χ mi or the logarithmic likelihood maximum at l L max, oe should look for the error cotour at χ mi + z or l L max z / usig a z-value from the right-had side of the table below. The probability cotet for a -dimesioal multiormal distributio as give below may be expressed i terms of the icomplete Gamma fuctio by p = P (, ) z as may be deduced by itegratig a stadard multiormal distributio out to a radius z. Special formulæ for the icomplete Gamma fuctio P (a, x) for iteger ad half-iteger a are give i sectio 4.5.3. Probability cotet i % Adjusted z-values z = z = z = 3 z z z 3 z 9 z 95 z 99 68.7 95.45 99.73.. 3..645.96.576 39.35 86.47 98.89.55.486 3.439.46.448 3.35 3 9.87 73.85 97.7.878.833 3.763.5.795 3.368 4 9. 59.4 93.89.7 3.7 4.3.789 3.8 3.644 5 3.743 45.6 89.9.46 3.364 4.67 3.39 3.37 3.884 6.439 3.33 8.64.653 3.585 4.479 3.63 3.548 4. 7.57. 74.73.859 3.786 4.674 3.467 3.75 4.98 8.75 4.9 65.77 3.5 3.974 4.855 3.655 3.938 4.48 9.56 8.859 56.7 3.9 4.49 5.6 3.83 4.3 4.655.7 5.65 46.79 3.396 4.34 5.87 3.998 4.79 4.88.54 3.8 37.8 3.556 4.47 5.34 4.56 4.436 4.97.4.656 9.7 3.77 4.6 5.486 4.37 4.585 5. 3.38.88.7 3.853 4.764 5.66 4.45 4.79 5.6 4..453 6.89 3.99 4.9 5.76 4.59 4.867 5.398 5.3.6.5 4.6 5.34 5.89 4.73 5. 5.53 6..97 8.659 4.56 5.63 6.8 4.85 5.8 5.657 7.57 5.974 4.38 5.87 6.4 4.977 5.5 5.78 8.37 4.6 4.53 5.48 6.59 5.98 5.373 5.9 9.6.65 4.6 5.55 6.374 5.6 5.49 6.6.465.79 4.737 5.639 6.487 5.33 5.65 6.9 5.5.44 5.7 6.7 7. 5.864 6.36 6.657 3.74 5.755 6.65 7.486 6.345 6.66 7.34 8.4 Radom Number Geeratio I order to obtai radom umbers from a multiormal distributio we proceed as follows:

If x = {x, x,..., x } is distributed multiormally with mea (zero vector) ad variace matrix I (uity matrix) the each x i (i =,,..., ) ca be foud idepedetly from a stadard ormal distributio. If x is multiormally distributed with mea µ ad variace matrix V the ay liear combiatio y = Sx is also multiormally distributed with mea Sµ ad variace matrix SV S T, If we wat to geerate vectors, y, from a multiormal distributio with mea µ ad variace matrix V we may make a so called Cholesky decompositio of V, i.e. we fid a triagular matrix S such that V = SS T. We the calculate y = Sx + µ with the compoets of x geerated idepedetly from a stadard ormal distributio. Thus we have foud a quite ice way of geeratig multiormally distributed radom umbers which is importat i may simulatios where correlatios betwee variables may ot be igored. If may radom umbers are to be geerated for multiormal variables from the same distributio it is beeficial to make the Cholesky decompositio oce ad store the matrix S for further usage.

9 Negative Biomial Distributio 9. Itroductio The Negative Biomial distributio is give by ( ) r p(r; k, p) = p k ( p) r k k where the variable r k ad the parameter k > are itegers ad the parameter p ( p ) is a real umber. The distributio expresses the probability of havig to wait exactly r trials util k successes have occurred if the probability of a success i a sigle trial is p (probability of failure q = p). The above form of the Negative Biomial distributio is ofte referred to as the Pascal distributio after the frech mathematicia, physicist ad philosopher Blaise Pascal (63 66). The distributio is sometimes expressed i terms of the umber of failures occurrig while waitig for k successes, = r k, i which case we write ( ) + k p(; k, p) = p k ( p) where the ew variable. Chagig variables, for this last form, to ad k istead of p ad k we sometimes use ( ) + k k k ( + k p(;, k) = ( + k) = +k ) ( + k ) ( k ) k + k The distributio may also be geeralized to real values of k, although this may seem obscure from the above probability view-poit ( fractioal success ), writig the biomial coefficiet as ( + k )( + k ) (k + )k/!. 9. Momets I the first form give above the expectatio value, variace, third ad fourth cetral momets of the distributio are E(r) = k p, kq V (r) = p, µ 3 = The coefficiets of skewess ad kurtosis are kq( p), ad µ p 3 4 = kq(p 6p + 6 + 3kq) p 4 γ = p kq ad γ = p 6p + 6 kq I the secod formulatio above, p(), the oly differece is that the expectatio value becomes k( p) E() = E(r) k = = kq p p

while higher momets remai uchaged as they should sice we have oly shifted the scale by a fixed amout. I the last form give, usig the parameters ad k, the expectatio value ad the variace are E() = ad V () = + k 9.3 Probability Geeratig Fuctio The probability geeratig fuctio is give by i the first case (p(r)) ad G(z) = G(z) = ( pz ) k zq ( ) k ( p = zq + ( z) k i the secod case (p()) for the two differet parameterizatios. 9.4 Relatios to Other Distributios There are several iterestig coectios betwee the Negative Biomial distributio ad other stadard statistical distributios. I the followig subsectios we briefly address some of these coectios. 9.4. Poisso Distributio Regard the egative biomial distributio i the form p(;, k) = ( ) ( + k + /k ) k ) k ( /k ) + /k where, k > ad >. As k the three terms become ( ) + k ( + k )( + k )... k =! ( ) k = k + /k k + k(k + ) k!, ( ) ( ) k(k + )(k + ) 3 +... e ad k 6 k ( ) /k k + /k where, for the last term we have icorporated the factor k from the first term. Thus we have show that lim p(;, k) = e k! 3

i.e. a Poisso distributio. This proof could perhaps better be made usig the probability geeratig fuctio of the egative biomial distributio G(z) = ( p zq ) k = ( (z )/k Makig a Taylor expasio of this for (z )/k we get G(z) = + (z ) + k + (z ) (k + )(k + ) (z ) 3 3 + +... e (z ) k k 6 as k. This result we recogize as the probability geeratig fuctio of the Poisso distributio. 9.4. Gamma Distributio Regard the egative biomial distributio i the form ( ) + k p(; k, p) = p k q where, k > ad p ad where we have itroduced q = p. If we chage parameters from k ad p to k ad = kq/p this may be writte ( ) ( ) k ( ) + k /k p(;, k) = + /k + /k Chagig variable from to z = / we get (d/dz = ) p(z;, k) = p(;, k) d ( ) ( ) k ( ) z z + k dz = /k = z + /k + /k ( ) (z + k )(z + k )... (z + ) k ( ) z = k k Γ(k) k + k/ + ( ) k k (z)k k ( ) z = Γ(k) k + k/ + = zk k k ( ) ( ) k z zk k k e kz Γ(k) k + k/ + Γ(k) where we have used that for k ( ) k ad k + ( k/ + ) z = z k z(z + ) + ) k ( ) k z(z + )(z + ) 6 ( ) 3 k +... zk + z k z3 k 3 +... = e kz 6 as. Thus we have show that as ad k we obtai a gamma distributio i the variable z = /. 4

9.4.3 Logarithmic Distributio Regard the egative biomial distributio i the form ( ) + k p(; k, p) = p k q where, k > ad p ad where we have itroduced q = p. The probabilities for =,,, 3... are give by { p(), p(), p(), p(3),...} = p k {, kq, if we omit the zero class (=) ad reormalize we get kp k {, q, p k k + q,! k(k + ) q,! } (k + )(k + ) q 3,... 3! } k(k + )(k + ) q 3,... 3! ad if we let k we fially obtai {, q, l p q, q 3 } 3,... where we have used that lim k k p k = l p which is easily realized expadig p k = e k l p ito a power series. This we recogize as the logarithmic distributio p(; p) = ( p) l p thus we have show that omittig the zero class ad lettig k the egative biomial distributio becomes the logarithmic distributio. 9.4.4 Brachig Process I a process where a brachig occurs from a Poisso to a logarithmic distributio the most elegat way to determie the resultig distributio is by use of the probability geeratig fuctio. The probability geeratig fuctios for a Poisso distributio with parameter (mea) µ ad for a logarithmic distributio with parameter p (q = p) are give by G P (z) = e µ(z ) ad G L (z) = l( zq)/ l( q) = α l( zq) where µ >, q ad α = / l p. For a brachig process i steps G(z) = G (G (... G (G (z))...)) 5

where G k (z) is the probability geeratig fuctio i the k:th step. I the above case this gives G(z) = G P (G L (z)) = exp {µ(α l( zq) )} = = exp {αµ l( zq) µ} = ( zq) αµ e µ = = ( zq) k ( q) k = p k /( zq) k where we have put k = αµ. This we recogize as the probability geeratig fuctio of a egative biomial distributio with parameters k ad p. We have thus show that a Poisso distributio with mea µ brachig ito a logarithmic distributio with parameter p gives rise to a egative biomial distributio with parameters k = αµ = µ/ l p ad p (or = kq/p). Coversely a egative biomial distributio with parameters k ad p or could arise from the combiatio of a Poisso distributio with parameter µ = k l p = k l( + ) k ad a logarithmic distributio with parameter p ad mea /µ. A particle physics example would be a charged multiplicity distributio arisig from the productio of idepedet clusters subsequetly decayig ito charged particles accordig to a logarithmic distributio. The UA5 experimet [36] foud o the SppS collider at CERN that at a cetre of mass eergy of 54 GeV a egative biomial distributio with = 8.3 ad k = 3.69 fitted the data well. With the above sceario this would correspod to 8 clusters beig idepedetly produced (Poisso distributio with µ = 7.97) each oe decayig, accordig to a logarithmic distributio, ito 3.55 charged particles o average. 9.4.5 Poisso ad Gamma Distributios If a Poisso distributio with mea µ > p(; µ) = e µ µ! for is weighted by a gamma distributio with parameters a > ad b > f(x; a, b) = a(ax)b e ax Γ(b) for x > we obtai P() = = = p(; µ)f(µ; a, b)dµ = a b µ +b e µ(a+) dµ =!Γ(b) ( ) ( + b a a + e µ µ a(aµ) b e aµ dµ =! Γ(b) ) b ( ) a + a b!(b )! ( + b )!(a + ) (+b) = which is a egative biomial distributio with parameters p = a, i.e. q = p =, a+ a+ ad k = b. If we aim at a egative biomial distributio with parameters ad k we should 6

thus weight a Poisso distributio with a gamma distributio with parameters a = k/ ad b = k. This is the same as superimposig Poisso distributios with meas comig from a gamma distributio with mea. I the calculatio above we have made use of itegral tables for the itegral x e αx dx =!α (+) 9.5 Radom Number Geeratio I order to obtai radom umbers from a Negative Biomial distributio we may use the recursive formula qr p(r + ) = p(r) r + k q(k + ) or p( + ) = p() + for r = k, k +,... ad =,,... i the two cases startig with the first term (p(k) or p()) beig equal to p k. This techique may be speeded up cosiderably, if p ad k are costats, by preparig a cumulative vector oce for all. Oe may also use some of the relatios described above such as the brachig of a Poisso to a Logarithmic distributio 6 or a Poisso distributio weighted by a Gamma distributio 7. This, however, will always be less efficiet tha the straightforward cumulative techique. 6 Geeratig radom umbers from a Poisso distributio with mea µ = k l p brachig to a Logarithmic distributio with parameter p will give a Negative Biomial distributio with parameters k ad p. 7 Takig a Poisso distributio with a mea distributed accordig to a Gamma distributio with parameters a = k/ ad b = k. 7

3 No-cetral Beta-distributio 3. Itroductio The o-cetral Beta-distributio is give by ( ) r λ f(x; p, q) = e λ r= r! x p+r ( x) q B (p + r, q) where p ad q are positive real quatities ad the o-cetrality parameter λ. I figure we show examples of a o-cetral Beta distributio with p = 3 ad q = 3 varyig the o-cetral parameter λ from zero (a ordiary Beta distributio) to te i steps of two. Figure : Graph of o-cetral Beta-distributio for p = 3, q = 3 ad some values of λ 3. Derivatio of distributio If y m ad y are two idepedet variables distributed accordig to the chi-squared distributio with m ad degrees of freedom, respectively, the the ratio y m /(y m + y ) follows a Beta distributio with parameters p = m ad q =. If istead y m follows a o-cetral chi-square distributio we may proceed i a similar way as was doe for the derivatio of the Beta-distributio (see sectio 4.). We make a chage of variables to x = y m /(y m + y ) ad y = y m + y which implies that y m = xy ad y = y( x) obtaiig f(x, y) = = y m x y x y y y m y y f(y m, y ) = y ( ) r ( x λ ym e λ r= r! x 8 ) m +r e ym Γ ( m + r) ( y ) e y Γ ( ) =

( ) r ( ) m λ xy +r e xy = y e λ r= r! Γ ( m + r) ( ) r λ = e λ x m +r ( x) r= r! B ( m + r, ) ( ) y( x) e y( x) Γ ( ) ( y ) m+ +r e y Γ ( m+ + r ) = I the last braces we see a chi-square distributio i y with m + + r degrees of freedom ad itegratig f(x, y) over y i order to get the margial distributio i x gives us the o-cetral Beta-distributio as give above with p = m/ ad q = /. If istead y were distributed as a o-cetral chi-square distributio we would get a very similar expressio (ot amazig sice y m /(y m + y ) = y /(y m + y )) but it s the form obtaied whe y m is o-cetral, that is ormally referred to as the o-cetral Beta-distributio. 3.3 Momets Algebraic momets of the o-cetral Beta-distributio are give i terms of the hypergeometric fuctio F as ( ) r λ E(x k ) = x k f(x; p, q)dx = e λ x p+r+k ( x) q dx = r= r! B (p + r, q) ( ) r ( ) r λ λ = e λ B (p + r + k, q) = e λ Γ (p + r + k) Γ (p + r + q) r= r! B (p + r, q) r= r! Γ (p + r) Γ (p + r + q + k) = ( ) r λ = e λ (p + r + k ) (p + r + )(p + r) r= r! (p + q + r + k ) (p + q + r + )(p + q + r) = = e λ Γ(p + k) Γ(p + q) Γ(p) Γ(p + q + k) ( ) F p + q, p + k; p, p + q + k; λ However, to evaluate the hypergeometric fuctio ivolves a summatio so it is more efficiet to directly use the peultimate expressio above. 3.4 Cumulative distributio The cumulative distributio is foud by straightforward itegratio F (x) = ( ) r x λ e λ r= r! u p+r ( u) q du = B (p + r, q) ( ) r λ e λ I x (p + r, q) r= r! 3.5 Radom Number Geeratio Radom umbers from a o-cetral Beta-distributio with iteger or half-iteger p ad q values is easily obtaied usig the defiitio above i.e. by usig a radom umber from a o-cetral chi-square distributio ad aother from a (cetral) chi-square distributio. 9

3 No-cetral Chi-square Distributio 3. Itroductio If we istead of addig squares of idepedet stadard ormal, N(, ), variables, givig rise to the chi-square distributio with degrees of freedom, add squares of N(µ i, ) variables we obtai the o-cetral chi-square distributio ( ) r λ f(x;, λ) = e λ f(x; + r) = r= r! Γ ( )x e (x+λ) r= (λx) r (r)! Γ ( + r) Γ ( + r) where λ = µ i is the o-cetral parameter ad f(x; ) the ordiary chi-square distributio. As for the latter the variable x ad the parameter a positive iteger. The additioal parameter λ ad i the limit λ = we retai the ordiary chi-square distributio. Accordig to [] pp 7 9 the o-cetral chi-square distributio was first itroduced by R. A. Fisher i 98. I figure we show the distributio for = 5 ad o-cetral parameter λ =,,, 3, 4, 5 (zero correspodig to the ordiary chi-squared distributio). Figure : Graph of o-cetral chi-square distributio for = 5 ad some values of λ 3. Characteristic Fuctio The characteristic fuctio for the o-cetral chi-square distributio is give by φ(t) = exp ( ) ıtλ ıt ( ıt) but eve more useful i determiig momets is l φ(t) = ıtλ ıt l( ıt)

from which cumulats may be determied i a similar maer as we ormally obtai algebraic momets from φ(t) (see below). By lookig at the characteristic fuctio oe sees that the sum of two o-cetral chisquare variates has the same distributio with degrees of freedoms as well as o-cetral parameters beig the sum of the correspodig parameters for the idividual distributios. 3.3 Momets To use the characteristic fuctio to obtai algebraic momets is ot trivial but the cumulats (see sectio.5) are easily foud to be give by the formula κ r = r (r )!( + rλ) for r from which we may fid the lower order algebraic ad cetral momets (with a = + λ ad b = λ/a) as µ = κ = a = + λ µ = κ = a( + b) = ( + λ) µ 3 = κ 3 = 8a( + b) = 8( + 3λ) µ 4 = κ 4 + 3κ = 48( + 4λ) + ( + λ) µ 5 = κ 5 + κ 3 κ = 384( + 5λ) + 6( + λ)( + 3λ) µ 6 = κ 6 + 5κ 4 κ + κ 3 + 5κ 3 = = 384( + 6λ) + 44( + λ)( + 4λ) + 64( + 3λ) + ( + λ) 3 γ = ( ) 3 + b 8( + 3λ) = + b a [( + λ)] 3 γ = a + 3b ( + 4λ) = ( + b) ( + λ) 3.4 Cumulative Distributio The cumulative, or distributio, fuctio may be foud by F (x) = Γ ( )e λ r= 3.5 Approximatios r= λ r (r)! Γ ( + r) Γ ( + r) x u +r e u du = = Γ ( )e λ λ r Γ ( + r) r= (r)! Γ ( + +r γ ( + r, ) x r) = ( ) r λ = e λ P ( r! + r, ) x A approximatio to a chi-square distributio is foud by equatig the first two cumulats of a o-cetral chi-square distributio with those of ρ times a chi-square distributio.

Here ρ is a costat to be determied. The result is that with ρ = + λ + λ = + λ + λ ad = ( + λ) + λ = + λ + λ we may approximate a o-cetral chi-square distributio f(x;, λ) with a (cetral) chisquare distributio i x/ρ with degrees of freedom ( i geeral beig fractioal). Approximatios to the stadard ormal distributio are give usig ( ) x 3 x a z = + b + b or z = [ ] +b a 9 a +b 9 a 3.6 Radom Number Geeratio Radom umbers from a o-cetral chi-square distributio is easily obtaied usig the defiitio above by e.g. Put µ = λ/ Sum radom umbers from a ormal distributio with mea µ ad variace uity. Note that this is ot a uique choice. The oly requiremet is that λ = µ i. Retur the sum as a radom umber from a o-cetral chi-square distributio with degrees of freedom ad o-cetral parameter λ. This ought to be sufficiet for most applicatios but if eeded more efficiet techiques may easily be developed e.g. usig more geeral techiques.

3 No-cetral F -Distributio 3. Itroductio If x is distributed accordig to a o-cetral chi-square distributio with m degrees of freedom ad o-cetral parameter λ ad x accordig to a (cetral) chi-square distributio with degrees of freedom the, provided x ad x are idepedet, the variable F = x /m x / is said to have a o-cetral F -distributio with m, degrees of freedom (positive itegers) ad o-cetral parameter λ. As the o-cetral chi-square distributio it was first discussed by R. A. Fisher i 98. This distributio i F may be writte f(f ; m,, λ) = e λ r= r! ( ) ( r λ Γ m+ + r ) Γ ( m + r) Γ ( ) ( ) m m + r (F ) m + r ( + mf ) (m+)+r I figure 3 we show the o-cetral F -distributio for the case with m = ad = 5 varyig λ from zero (a ordiary, cetral, F -distributio) to five. Figure 3: Graph of o-cetral F -distributio for m =, = 5 ad some values of λ Whe m = the o-cetral F -distributio reduces to a o-cetral t -distributio with δ = λ. As the F approaches a o-cetral chi-square distributio with m degrees of freedom ad o-cetral parameter λ. 3

3. Momets Algebraic momets of the o-cetral F -distributio may be achieved by straightforward, but somewhat tedious, algebra as E(F k ) = x k f(x; m,, λ)dx = ( = e λ k Γ m) ( k) Γ ( ) r= r! ( ) r λ Γ ( m + r + k) Γ ( m + r) a expressio which may be used to fid lower order momets (defied for > k) E(F ) = m m + λ ( ) { E(F ) = λ + (λ + m)(m + ) } m ( )( 4) ( ) 3 E(F 3 ) = m ( )( 4)( 6) { λ 3 + 3(m + 4)λ + (3λ + m)(m + 4)(m + ) } ( ) 4 E(F 4 ) = m ( )( 4)( 6)( 8) { λ 4 + 4(m + 6)λ 3 + 6(m + 6)(m + 4)λ + +(4λ + m)(m + 6)(m + 4)(m + )} ( ) { } (λ + m) V (F ) = + λ + m m ( )( 4) 3.3 Cumulative Distributio The cumulative, or distributio, fuctio may be foud by with F (x) = x = e λ = e λ u k f(u; m,, λ)du = r= r= r! r! 3.4 Approximatios ( ) ( r λ Γ m+ + r ) Γ ( m + r) Γ ( ) ( ) ( r λ B m q + r, ) B ( m + r, ) = e λ q = mx + mx ( ) m m +r x ( ) r λ r= r! u m +r+k ) m+ ( + mu I q ( m + r, +r du = Usig the approximatio of a o-cetral chi-square distributio to a (cetral) chi-square distributio give i the previous sectio we see that m m + λ F 4 )

is approximately distributed accordig to a (cetral) F -distributio with m = m + ad degrees of freedom. Approximatios to the stadard ormal distributio is achieved with z = F E(F ) V (F ) or z = ( mf m+λ = ) 3 ( 9 [ m ( )( 4) F (m+λ) m( ) { (m+λ) + m + λ }] ) ( ) m+λ 9 (m+λ) [ 9 m+λ (m+λ) + 9 ( mf m+λ 3.5 Radom Number Geeratio ) 3 ] λ m+λ Radom umbers from a o-cetral chi-square distributio is easily obtaied usig the defiitio above i.e. by usig a radom umber from a o-cetral chi-square distributio ad aother from a (cetral) chi-square distributio. 5

33 No-cetral t-distributio 33. Itroductio If x is distributed accordig to a ormal distributio with mea δ ad variace ad y accordig to a chi-square distributio with degrees of freedom (idepedet of x) the t = x y/ has a o-cetral t-distributio with degrees of freedom (positive iteger) ad o-cetral parameter δ (real). We may also write t = z + δ w/ where z is a stadard ormal variate ad w is distributed as a chi-square variable with degrees of freedom. The distributio is give by (see commets o derivatio i sectio below) f(t ;, δ) = e δ πγ ( ) r= (t δ) r r! r ( + t ) +r+ ( ) r Γ +r+ I figure 4 we show the o-cetral t-distributio for the case with = varyig δ from zero (a ordiary t-distributio) to five. Figure 4: Graph of o-cetral t-distributio for = ad some values of δ This distributio is of importace i hypotheses testig if we are iterested i the probability of committig a Type II error implyig that we would accept a hypothesis although it was wrog, see discussio i sectio 38. o page 46. 33. Derivatio of distributio Not may text-books iclude a formula for the o-cetral t-distributio ad some turs out to give erroeous expressios. A o-cetral F -distributio with m = becomes a 6

o-cetral t -distributio which the may be trasformed to a o-cetral t-distributio. However, with this approach oe easily gets ito trouble for t <. Istead we adopt a techique very similar to what is used i sectio 38.6 to obtai the ormal (cetral) t-distributio from a t-ratio. The differece i the o-cetral case is the presece of the δ-parameter which itroduces two ew expoetial terms i the equatios to be solved. Oe is simply exp( δ /) but aother factor we treat by a serial expasio leadig to the p.d.f. above. This may ot be the best possible expressio but empirically it works quite well. 33.3 Momets With some effort the p.d.f. above may be used to calculate algebraic momets of the distributio yieldig E(t k ) = e δ ( πγ )Γ ( ) k k r= δ r r r! Γ ( ) r+k+ where the sum should be made for odd (eve) values of r if k is odd (eve). This gives for low orders Γ ( ) µ = Γ ( ) δ µ = Γ ( ) ( ) Γ ( ) + δ = ( ) + δ µ 3 = ( ) 3 Γ 3 4Γ ( ) δ ( 3 + δ ) µ 4 = Γ ( ) 4 ( 4Γ ( ) δ 4 + 6δ + 3 ) ( = δ 4 + 6δ + 3 ) ( )( 4) from which expressios for cetral momets may be foud e.g. the variace µ = V (t ) = ( ) + δ δ Γ ( ) Γ ( ) 33.4 Cumulative Distributio The cumulative, or distributio, fuctio may be foud by F (t) = = e δ πγ ( ) e δ πγ ( ) r= r= δ r r! r ( ) t r Γ +r+ u r ( + u ) +r+ du = δ r r! r Γ ( ) { +r+ s B ( r+, ) ( + s B r+ q, )} = 7

= e δ π r= δ r r! r Γ ( ) { ( r+ s + s I r+ q, )} where s are ad s are sigs differig betwee cases with positive or egative t as well as odd or eve r i the summatio. The sig s is if r is odd ad + if it is eve while s is + uless t < ad r is eve i which case it is. 33.5 Approximatio A approximatio is give by ( ) z = t 4 δ + t which is asymptotically distributed as a stadard ormal variable. 33.6 Radom Number Geeratio Radom umbers from a o-cetral t-distributio is easily obtaied usig the defiitio above i.e. by usig a radom umber from a ormal distributio ad aother from a chisquare distributio. This ought to be sufficiet for most applicatios but if eeded more efficiet techiques may easily be developed e.g. usig more geeral techiques. 8

34 Normal Distributio 34. Itroductio The ormal distributio or, as it is ofte called, the Gauss distributio is the most importat distributio i statistics. The distributio is give by f(x; µ, σ ) = σ π e ( x µ σ ) where µ is a locatio parameter, equal to the mea, ad σ the stadard deviatio. For µ = ad σ = we refer to this distributio as the stadard ormal distributio. I may coectios it is sufficiet to use this simpler form sice µ ad σ simply may be regarded as a shift ad scale parameter, respectively. I figure 5 we show the stadard ormal distributio. Figure 5: Stadard ormal distributio Below we give some useful iformatio i coectio with the ormal distributio. Note, however, that this is oly a mior collectio sice there is o limit o importat ad iterestig statistical coectios to this distributio. 34. Momets The expectatio value of the distributio is E(x) = µ ad the variace V (x) = σ. Geerally odd cetral momets vaish due to the symmetry of the distributio ad eve cetral momets are give by for r. µ r = (r)! r r! σr = (r )!!σ r 9

It is sometimes also useful to evaluate absolute momets E( x ) for the ormal distributio. To do this we make use of the itegral π e ax dx = a which if differetiated k times with respect to a yields x k e ax dx = (k )!! π k a k+ I our case a = /σ ad sice eve absolute momets are idetical to the algebraic momets it is eough to evaluate odd absolute momets for which we get E( x k+ ) = σ x k+ e x (σ ) k+ σ dx = y k e y dy π π σ The last itegral we recogize as beig equal to k! ad we fially obtai the absolute momets of the ormal distributio as { ( )!!σ for = k E( x ) = π k k!σ k+ for = k + The half-width at half-height of the ormal distributio is give by l σ.77σ which may be useful to remember whe estimatig σ usig a ruler. 34.3 Cumulative Fuctio The distributio fuctio, or cumulative fuctio, may be expressed i term of the icomplete gamma fuctio P as F (z) = + P (, ) z if z P (, ) z if z < or we may use the error fuctio erf(z/ ) i place of the icomplete gamma fuctio. 34.4 Characteristic Fuctio The characteristic fuctio for the ormal distributio is easily foud from the geeral defiitio φ(t) = E ( e ıtx) = exp { µıt σ t }

34.5 Additio Theorem The so called Additio theorem for ormally distributed variables states that ay liear combiatio of idepedet ormally distributed radom variables x i (i =,,..., ) is also distributed accordig to the ormal distributio. If each x i is draw from a ormal distributio with mea µ i ad variace σi the regard the liear combiatio S = a i x i where a i are real coefficiets. Each term a i x i has characteristic fuctio i= φ ai x i (t) = exp { (a i µ i )ıt (a i σ i )t } ad thus S has characteristic fuctio {( ) ( ) } φ S (t) = φ ai x i (t) = exp a i µ i ıt a i σi t i= i= i= which is see to be a ormal distributio with mea a i µ i ad variace a i σ i. 34.6 Idepedece of x ad s A uique property of the ormal distributio is the idepedece of the sample statistics x ad s, estimates of the mea ad variace of the distributio. Recall that the defiitio of these quatities are x = i= x i ad s = (x i x) i= where x is a estimator of the true mea µ ad s is the usual ubiased estimator for the true variace σ. For a populatio of evets from a ormal distributio x has the distributio N(µ, σ /) ad ( )s /σ is distributed accordig to a chi-square distributio with degrees of freedom. Usig the relatio ( xi µ i= σ ) = ( )s σ + ( ) x µ σ/ ad creatig the joit characteristic fuctio for the variables ( )s /σ ad ( (x µ)/σ ) oe may show that this fuctio factorizes thus implyig idepedece of these quatities ad thus also of x ad s. I summary the idepedece theorem states that give idepedet radom variables with idetical ormal distributios the two statistics x ad s are idepedet. Also coversely it holds that if the mea x ad the variace s of a radom sample are idepedet the the populatio is ormal.

34.7 Probability Cotet The probability cotet of the ormal distributio is ofte referred to i statistics. Whe the term oe stadard deviatio is metioed oe immediately thiks i terms of a probability cotet of 68.3% withi the symmetric iterval from the value give. Without loss of geerality we may treat the stadard ormal distributio oly sice the trasformatio from a more geeral case is straightforward puttig z = (x µ)/σ. I differet situatio oe may wat to fid the probability cotet, two-side or oe-sided, to exceed a certai umber of stadard deviatios, or the umber of stadard deviatios correspodig to a certai probability cotet. I calculatig this we eed to evaluate itegrals like α = e t dt π There are o explicit solutio to this itegral but it is related to the error fuctio (see sectio 3) as well as the icomplete gamma fuctio (see sectio 4). π z z e z = erf ( z ) = P ( ), z These relatios may be used to calculate the probability cotet. Especially the error fuctio is ofte available as a system fuctio o differet computers. Beware, however, that it seems to be implemeted such that erf(z) is the symmetric itegral from z to z ad thus the factor should ot be supplied. Besides from the above relatios there are also excellet approximatios to the itegral which may be used. I the tables below we give the probability cotet for exact z-values (left-had table) as well as z-values for exact probability cotets (right-had table). z z z z z z z z z z..5..5.5.6946.389.3854..8434.6869.5866.5.9339.86639.668..9775.9545.75.5.99379.98758 6. 3 3..99865.9973.35 3 3.5.99977.99953.36 4 4..99997.99994 3.67 5 4.5..99999 3.398 6 5....867 7 6... 9.866 7....8 8... 6. 6..5..5.5335.6..4.67449.75.5.5.846.8.6..855.9.8..64485.95.9.5.95996.975.95.5.3635.99.98..57583.995.99.5 3.93.999.998. 3.953.9995.999.5 3.79.9999.9998. 3.8959.99995.9999.5 4.6489.99999.99998.

It is sometimes of iterest to scrutiize extreme sigificace levels which implies itegratig the far tails of a ormal distributio. I the table below we give the umber of stadard deviatios, z, required i order to achieve a oe-tailed probability cotet of. z-values for which π z e z / dz = for =,,..., 3 z z z z z.855 6 4.7534 6.76 6 8.8 9.55.3635 7 5.9934 7.3448 7 8.49379 9.7479 3 3.93 8 5.6 3 7.3488 8 8.7579 3 9.9735 4 3.79 9 5.9978 4 7.6563 9 9.37 5 4.6489 6.3634 5 7.9435 9.634 Below are also give the oe-tailed probability cotet for a stadard ormal distributio i the regio from z to (or to z). The iformatio i the previous as well as this table is take from [6]. Probability cotet Q(z) = π z e z / dz for z =,,..., 5, 6,...,, 5,..., 5 z log Q(z) z log Q(z) z log Q(z) z log Q(z) z log Q(z).79955 4 44.87 7 6.339 4 349.437 8 39.4459.643 5 5.435 8 7.94 4 367.3664 9 76.464 3.8697 6 57.9458 9 84.4883 4 385.73 73.8754 4 4.49934 7 64.38658 3 97.39 43 43.5384 5 4888.388 5 6.5465 8 7.4 3.5694 44 4.43983 8688.58977 6 9.586 9 8.699 3 4.6344 45 44.77568 5 3574.4996 7.8985 88.56 33 38.3935 46 46.5456 3 9546.79 8 5.64 97.484 34 5.9535 47 48.74964 35 663.488 9 8.94746 6.8467 35 67.94888 48 5.38776 4 34746.5597 3.85 3 6.6353 36 83.37855 49 53.45999 45 43975.3686 7.788 4 6.85686 37 99.48 5 544.96634 5 5489.983 3.7544 5 37.5475 38 35.53979 6 783.9743 3 38.345 6 48.664 39 33.739 7 66.6576 Beware, however, that extreme sigificace levels are purely theoretical ad that oe seldom or ever should trust experimetal limits at these levels. I a experimetal situatios oe rarely fulfills the statistical laws to such detail ad ay bias or backgroud may heavily affect statemets o extremely small probabilities. Although oe ormally would use a routie to fid the probability cotet for a ormal distributio it is sometimes coveiet to have a classical table available. I table 6 o page 76 we give probability cotets for a symmetric regio from z to z for z-values ragig from. to 3.99 i steps of.. Coversely we give i table 7 o page 77 the z-values correspodig to specific probability cotets from. to.998 i steps of.. 3

34.8 Radom Number Geeratio There are may differet methods to obtai radom umbers from a ormal distributio some of which are reviewed below. It is eough to cosider the case of a stadard ormal distributio sice give such a radom umber z we may easily obtai oe from a geeral ormal distributio by makig the trasformatio x = µ + σz. Below f(x) deotes the stadard ormal distributio ad if ot explicitly stated all variables deoted by ξ are uiform radom umbers i the rage from zero to oe. 34.8. Cetral Limit Theory Approach The sum of idepedet radom umbers from a uiform distributio betwee zero ad oe, R, has expectatio value E(R ) = / ad variace V (R ) = /. By the cetral limit theorem the quatity z = R E(R ) V (R ) = R approaches the stadard ormal distributio as. A practical choice is = sice this expressio simplifies to z = R 6 which could be take as a radom umber from a stadard ormal distributio. Note, however, that this method is either accurate or fast. 34.8. Exact Trasformatio The Box-Muller trasformatio used to fid radom umbers from the biormal distributio (see sectio 6.5 o page ), usig two uiform radom umbers betwee zero ad oe i ξ ad ξ, z = l ξ si πξ z = l ξ cos πξ may be used to obtai two idepedet radom umbers from a stadard ormal distributio. 34.8.3 Polar Method The above method may be altered i order to avoid the cosie ad sie by i Geerate u ad v as two uiformly distributed radom umbers i the rage from - to by u = ξ ad v = ξ. ii Calculate w = u + v ad if w > the go back to i. iii Retur x = uz ad y = vz with z = l w/w as two idepedet radom umbers from a stadard ormal distributio. This method is ofte faster tha the previous sice it elimiates the sie ad cosie at the slight expese of π/4 % rejectio i step iii ad a few more arithmetic operatios. As is easily see u/ w ad v/ w plays the role of the cosie ad the sie i the previous method. 4

34.8.4 Trapezoidal Method The maximum trapezoid that may be iscribed uder the stadard ormal curve covers a area of 9.95% of the total area. Radom umbers from a trapezoid is easily obtaied by a liear combiatio of two uiform radom umbers. I the remaiig cases a tail-techique ad accept-reject techiques, as described i figure 6, are used. Figure 6: Trapezoidal method Below we describe, i some detail, a slightly modified versio of what is preseted i [8]. For more exact values of the costats used see this referece. i Geerate two uiform radom umbers betwee zero ad oe ξ ad ξ ii If ξ <.995 geerate a radom umber from the trapezoid by x =.44ξ +.984ξ.4 ad exit iii Else if ξ <.954 (3.45% of all cases) geerate a radom umber from the tail x >.4 a Geerate two uiform radom umbers ξ ad ξ b Put x =.4 l ξ ad if xξ >.4 the go back to a c Put x = x ad go to vii iv Else if ξ <.978 (.4% of all cases) geerate a radom umber from the regio.9 < x <.84 betwee the ormal curve ad the trapezoid a Geerate two uiform radom umbers ξ ad ξ b Put x =.9 +.55ξ ad if f(x).443 +.x <.6ξ the go to a c Go to vii v Else if ξ <.9937 (.55% of all cases) geerate a radom umber from the regio.84 < ξ <.4 betwee the ormal curve ad the trapezoid a Geerate two uiform radom umbers ξ ad ξ b Put x =.84 +.74ξ ad if f(x).443 +.x <.43ξ the go to a 5

c Go to vii vi Else, i.63% of all cases, geerate a radom umber from the regio < x <.9 betwee the ormal curve ad the trapezoid by a Geerate two uiform radom umbers ξ ad ξ b Put x =.9ξ ad if f(x).383 <.6ξ the go back to a vii Assig a mius sig to x if ξ 34.8.5 Ceter-tail method Ahres ad Dieter [8] also proposes a so called ceter-tail method. I their article they treat the tails outside z > with a special tail method which avoids the logarithm. However, it turs out that usig the same tail method as i the previous method is eve faster. The method is as follows: i Geerate a uiform radom umber ξ ad use the first bit after the decimal poit as a sig bit s i.e. for ξ put ξ = ξ ad s = ad for ξ > put ξ = ξ ad s = ii If ξ >.8477994975 (the area for < z < ) go to vi. iii Ceter method: Geerate ξ ad set ν = ξ + iv Geerate ξ ad ξ ad set ν = max(ξ, ξ ). If ν < ν calculate y = ξ ad go to viii v Geerate ξ ad ξ ad set ν = max(ξ, ξ ) If ν < ν go to iv else go to iii vi Tail method: Geerate ξ ad set y = l ξ vii Geerate ξ ad if yξ > go to vi else put y = y viii Set x = sy. 34.8.6 Compositio-rejectio Methods I referece [] two methods usig the compositio-rejectio method is proposed. The first oe, attributed to Butcher [3] ad Kah, uses oly oe term i the sum ad has α = e/π, f(x) = exp { x} ad g(x) = exp { (x ) /}. The algorithm is as follows: i Geerate ξ ad ξ ii Determie x = l ξ, i.e. a radom umber from f(x) iii Determie g(x) = exp { (x ) /} iv If ξ > g(x) the go to i 6

v Decide a sig either by geeratig a ew radom umber, or by usig ξ for which < ξ g(x) here, ad exit with x with this sig. The secod method is origially proposed by J. C. Butcher [3] ad uses two terms α = f π (x) = g (x) = e x for x α = / π f (x) = e (x ) g (x) = e (x ) for x > i Geerate ξ ad ξ ii If ξ 3 > the determie x = l(3ξ ) ad z = (x ) else determie x = 3ξ / ad z = x / iii Determie g = e z iv If ξ > g the go to i v Determie the sig of ξ g/ ad exit with x with this sig. 34.8.7 Method by Marsaglia A ice method proposed by G. Marsaglia is based o iscribig a splie fuctio beeath the stadard ormal curve ad subsequetly a triagular distributio beeath the remaiig differece. See figure 7 for a graphical presetatio of the method. The algorithm used is described below. Figure 7: Marsaglia method The sum of three uiform radom umbers ξ, ξ, ad ξ 3 follow a parabolic splie fuctio. Usig x = (ξ + ξ + ξ 3 3 ) we obtai a distributio (3 x )/8 if x f (x) = (3 x ) /6 if < x 3 if x > 3 Maximizig α with the costrait f(x) α f (x) i the full iterval x 3 gives α = 6e / π.8638554 i.e. i about 86% of all cases such a combiatio is made. 7

Moreover, a triagular distributio give by makig the combiatio x = 3(ξ +ξ ) leadig to a fuctio ( f (x) = 4 3 9 x ) for x < 3 ad zero elsewhere. This fuctio may be iscribed uder the remaiig curve f(x) f (x) maximizig α such that f 3 (x) = f(x) α f (x) α f (x) i the iterval x 3. This leads to a value α.8 i.e. i about % of all cases this combiatio is used The maximum value of f 3 (x) i the regio x 3 is.8 ad here we use a straightforward reject-accept techique. This is doe i about.6% of all cases. Fially, the tails outside x > 3, coverig about.7% of the total area is dealt with with a stadard tail-method where a Put x = 9 l ξ b If xξ > 9 the go to a c Else geerate a sig s = + or s = with equal probability ad exit with x = s x 34.8.8 Histogram Techique Yet aother method due to G. Marsaglia ad collaborators [37] is oe where a histogram with k bis ad bi-width c is iscribed uder the (folded) ormal curve. The differece betwee the ormal curve ad the histogram is treated with a combiatio of triagular distributios ad accept-reject techiques as well as the usual techique for the tails. Tryig to optimize fast geeratio we foud k = 9 ad c = to be a fair choice. This may, however, 3 ot be true o all computers. See figure 8 for a graphical presetatio of the method. The algorithm is as follows: Figure 8: Histogram method i Geerate ξ ad chose which regio to geerate from. This is doe e.g. with a sequetial search i a cumulative vector where the areas of the regios have bee 8

sorted i descedig order. The umber of elemets i this vector is k + c + which for the parameters metioed above becomes. ii If a histogram bi i (i =,,..., k) is selected the determie x = (ξ + i )c ad go to vii. iii If a iscribed triagle i (i =,,..., c ) the determie x = (mi(ξ, ξ 3 ) + i)c ad go to vii. iv If subscribed triagle i (i = c +,..., k) the determie x = (mi(ξ, ξ 3 )+i )c ad accept this value with a probability equal to the ratio betwee the ormal curve ad the triagle at this x-value (histogram subtracted i both cases) else iterate. Whe a value is accepted the go to vii. v For the remaiig regios betwee the iscribed triagles ad the ormal curve for c x < use a stadard reject accept method i each bi ad the go to vii. vi If the tail regio is selected the use a stadard techique e.g. (a) x = (kc) l ξ, (b) if xξ 3 > (kc) the go to a else use x = x. vii Attach a radom sig to x ad exit. This is doe by either geeratig a ew uiform radom umber or by savig the first bit of ξ i step i. The latter is faster ad the degradatio i precisio is egligible. 34.8.9 Ratio of Uiform Deviates A techique usig the ratio of two uiform deviates was propose by A. J. Kiderma ad J. F. Moaha i 977 [38]. It is based o selectig a acceptace regio such that the ratio of two uiform pseudoradom umbers follow the stadard ormal distributio. With u ad v uiform radom umbers, u betwee ad ad v betwee /e ad /e, such a regio is defied by v < 4 u l u as is show i the left-had side of figure 9. Note that it is eough to cosider the upper part (v > ) of the acceptace limit due to the symmetry of the problem. I order to avoid takig the logarithm, which may slow the algorithm dow, simpler boudary curves were desiged. A improvemet to the origial proposal was made by Joseph L. Leva i 99 [39,4] choosig the same quadratic form for both the lower ad the upper boudary amely Q(u, v) = (u s) b(u s)(v t) + (a v) Here (s, t) = (.44987, -.386595) is the ceter of the ellipses ad a =.96 ad b =.547 are suitable costats to obtai tight boudaries. I the right-had side of figure 9 we show the value of the quadratic form at the acceptace limit Q(u, u l u) as a fuctio of u. It may be deduced that oly i the iterval r < Q < r with r =.7597 ad r =.7846 we still have to evaluate the logarithm. The algorithm is as follows: 9

Figure 9: Method usig ratio betwee two uiform deviates i Geerate uiform radom umbers u = ξ ad v = /e(ξ ). ii Evaluate the quadratic form Q = x + y(ay bx) with x = u s ad y = v t. iii Accept if iside ier boudary, i.e. if Q < r, the go to vi. iv Reject if outside upper boudary, i.e. if Q > r, the go to i. v Reject if outside acceptace regio, i.e. if v > 4u l u, the go to i. vi Retur the ratio v/u as a pseudoradom umber from a stadard ormal distributio. O average.738 uiform radom umbers are cosumed ad. logarithms are computed per each stadard ormal radom umber obtaied by this algorithm. As a compariso the umber of logarithmic evaluatios without cuttig o the boudaries, skippig steps ii through iv above, would be.369. The pealty whe usig logarithms o moder computers is ot as severe as it used to be but still some efficiecy is gaied by usig the proposed algorithm. 34.8. Compariso of radom umber geerators Above we described several methods to achieve pseudoradom umbers from a stadard ormal distributio. Which oe is the most efficiet may vary depedig o the actual implemetatio ad the computer it is used at. To give a rough idea we foud the followig times per radom umber 8 (i the table are also give the average umber of uiform 8 The timig was doe o a Digital Persoal Workstatio 433au workstatio ruig Uix versio 4.D ad all methods were programmed i stadard Fortra as fuctios givig oe radom umber at each call. 3

pseudoradom umbers cosumed per radom umber i our implemetatios) Method sectio µs/r.. N ξ /r.. commet Trapezoidal method 34.8.4.39.46 Polar method 34.8.3.4.73 pair Histogram method 34.8.8.4. Box-Muller trasformatio 34.8..44. pair Splie fuctios 34.8.7.46 3.55 Ratio of two uiform deviates 34.8.9.55.738 Compositio-rejectio, two terms 34.8.6.68.394 Ceter-tail method 34.8.5.88 5.844 Compositio-rejectio, oe term 34.8.6.9.63 Cetral limit method approach 34.8..6. iaccurate The trapezoidal method is thus fastest but the differece is ot great as compared to some of the others. The cetral limit theorem method is slow as well as iaccurate although it might be the easiest to remember. The other methods are all exact except for possible umerical problems. Pair idicates that these geerators give two radom umbers at a time which may implies that either oe is ot used or oe is left pedig for the ext call (as is the case i our implemetatios). 3

34.9 Tests o Parameters of a Normal Distributio For observatios from a ormal sample differet statistical distributios are applicable i differet situatios whe it comes to estimatig oe or both of the parameters µ ad σ. I the table below we try to summarize this i a codesed form. TESTS OF MEAN AND VARIANCE OF NORMAL DISTRIBUTION H Coditio Statistic Distributio µ = µ σ kow x µ σ/ N(, ) σ ukow σ = σ µ kow µ ukow µ = µ = µ σ = σ = σ kow σ σ kow ( )s σ ( )s σ x µ s/ t = (x i µ) i= σ = (x i x) i= σ x y σ + m x y χ χ N(, ) σ + σ m N(, ) σ = σ = σ ukow x y s + m s = ( )s +(m )s +m t +m σ = σ σ σ ukow µ µ kow µ µ ukow s s s s = = x y m m s + s m N(, ) (x i µ ) i= m F,m (y i µ ) (x i x) i= i= m F,m (y i y) i= 3

35 Pareto Distributio 35. Itroductio The Pareto distributio is give by f(x; α, k) = αk α /x α+ where the variable x k ad the parameter α > are real umbers. As is see k is oly a scale factor. The distributio has its ame after its ivetor the italia Vilfredo Pareto (848 93) who worked i the fields of atioal ecoomy ad sociology (professor i Lausae, Switzerlad). It was itroduced i order to explai the distributio of wages i society. 35. Cumulative Distributio The cumulative distributio is give by 35.3 Momets Algebraic momets are give by E(x ) = k F (x) = x f(x) = x k k f(u)du = ( ) α k x [ ] x αkα x = αkα = αkα α+ x α + α k which is defied for α >. Especially the expectatio value ad variace are give by 35.4 Radom Numbers E(x) = αk for α > α αk V (x) = for α > (α )(α ) To obtai a radom umber from a Pareto distributio we use the straightforward way of solvig the equatio F (x) = ξ with ξ a radom umber uiformly distributed betwee zero ad oe. This gives F (x) = ( ) α k = ξ x = x k ( ξ) α 33

36 Poisso Distributio 36. Itroductio The Poisso distributio is give by p(r; µ) = µr e µ where the variable r is a iteger (r ) ad the parameter µ is a real positive quatity. It is amed after the frech mathematicia Siméo Deis Poisso (78 84) who was the first to preset this distributio i 837 (implicitly the distributio was kow already i the begiig of the 8th cetury). As is easily see by comparig two subsequet r-values the distributio icreases up to r + < µ ad the declies to zero. For low values of µ it is very skewed (for µ < it is J-shaped). The Poisso distributio describes the probability to fid exactly r evets i a give legth of time if the evets occur idepedetly at a costat rate µ. A ubiased ad efficiet estimator of the Poisso parameter µ for a sample with observatios x i is ˆµ = x, the sample mea, with variace V (ˆµ) = µ/. For µ the distributio teds to a ormal distributio with mea µ ad variace µ. The Poisso distributio is oe of the most importat distributios i statistics with may applicatios. Alog with the properties of the distributio we give a few examples here but for a more thorough descriptio we refer to stadard text-books. 36. Momets The expectatio value, variace, third ad fourth cetral momets of the Poisso distributio are E(r) = µ V (r) = µ µ 3 = µ r! µ 4 = µ( + 3µ) The coefficiets of skewess ad kurtosis are γ = / µ ad γ = /µ respectively, i.e. they ted to zero as µ i accordace with the distributio becomig approximately ormally distributed for large values of µ. Algebraic momets may be foud by the recursive formula { } µ k+ = µ ad cetral momets by a similar formula { µ k+ = µ µ k + dµ k dµ kµ k + dµ k dµ 34 }

For a Poisso distributio oe may ote that factorial momets g k cumulats κ k (see sectio.5) become especially simple g k = E(r(r ) (r k + )) = µ k κ r = µ for all r (cf page 6) ad 36.3 Probability Geeratig Fuctio The probability geeratig fuctio is give by G(z) = E(z r ) = z r µr e µ = e µ r! r= r= (µz) r! = e µ(z ) Although we mostly use the probability geeratig fuctio i the case of a discrete distributio we may also defie the characteristic fuctio φ(t) = E(e ıtr ) = e µ r= ıtr µr e r! = exp { µ ( e ıt )} a result which could have bee give directly sice φ(t) = G(e ıt ). 36.4 Cumulative Distributio Whe calculatig the probability cotet of a Poisso distributio we eed the cumulative, or distributio, fuctio. This is easily obtaied by fidig the idividual probabilities e.g. by the recursive formula p(r) = p(r ) µ startig with p() = r e µ. There is, however, also a iterestig coectio to the icomplete Gamma fuctio [] r µ k e µ P (r) = = P (r +, µ) k! k= with P (a, x) the icomplete Gamma fuctio ot to be cofused with P (r). Sice the cumulative chi-square distributio also has a relatio to the icomplete Gamma fuctio oe may obtai a relatio betwee these cumulative distributios amely P (r) = r µ k e µ = k! k= µ f(x; ν = r + )dx where f(x; ν = r + ) deotes the chi-square distributio with ν degrees of freedom. 36.5 Additio Theorem The so called additio theorem states that the sum of ay umber of idepedet Poissodistributed variables is also distributed accordig to a Poisso distributio. For variables each distributed accordig to the Poisso distributio with parameters (meas) µ i we fid characteristic fuctio φ(r + r +... + r ) = exp { ( µ i e ıt )} { = exp i= ( µ i e ıt )} i= which is the characteristic fuctio for a Poisso variable with parameter µ = µ i. 35

36.6 Derivatio of the Poisso Distributio For a biomial distributio the rate of success p may be very small but i a log series of trials the total umber of successes may still be a cosiderable umber. I the limit p ad N but with Np = µ a fiite costat we fid p(r) = ( ) N p r ( p) N r r r! = N ( r! N r r N πnn N e N π(n r)(n r) N r e (N r) ( ) N e r µ r µ ) N r µ r e µ N r! ( µ N ) r ( µ N ) N r = as N ad where we have used that lim ( x ) = e x ad Stirlig s formula (se sectio 4.) for the factorial of a large umber! π e. It was this approximatio to the biomial distributio which S. D. Poisso preseted i his book i 837. 36.7 Histogram I a histogram of evets we would regard the distributio of the bi cotets as multiomially distributed if the total umber of evets N were regarded as a fixed umber. If, however, we would regard the total umber of evets ot as fixed but as distributed accordig to a Poisso distributio with mea ν we obtai (with k bis i the histogram ad the multiomial probabilities for each bi i the vector p) Give a multiomial distributio, deoted M(r; N, p), for the distributio of evets ito bis for fixed N ad a Poisso distributio, deoted P (N; ν), for the distributio of N we write the joit distributio P(r, N) = M(r; N, p)p (N; ν) = = where we have used that ( ) ( r! (νp ) r e νp r! (νp ) r e νp k p i = i= ( N! r!r!... r k! pr p r... p r k k ad k r i = N i= ) ( ν N e ν ) = N! ) ) (... r k! (νp k) r k e νp k i.e. we get a product of idepedet Poisso distributios with meas νp i for each idividual bi. A simpler case leadig to the same result would be the classificatio ito oly two groups usig a biomial ad a Poisso distributio. The assumptio of idepedet Poisso distributios for the umber evets i each bi is behid the usual rule of usig N as the stadard deviatio i a bi with N etries ad eglectig correlatios betwee bis i a histogram. 36

36.8 Radom Number Geeratio By use of the cumulative techique e.g. formig the cumulative distributio by startig with P () = e µ ad usig the recursive formula P (r) = P (r ) µ r a radom umber from a Poisso distributio is easily obtaied usig oe uiform radom umber betwee zero ad oe. If µ is a costat the by far fastest geeratio is obtaied if the cumulative vector is prepared oce for all. A alterative is to obtai, i ρ, a radom umber from a Poisso distributio by multiplyig idepedet uiform radom umbers ξ i util ρ ξ i e µ i= For large values of µ use the ormal approximatio but beware of the fact that the Poisso distributio is a fuctio i a discrete variable. 37

37 Rayleigh Distributio 37. Itroductio The Rayleigh distributio is give by f(x; α) = x x e α α for real positive values of the variable x ad a real positive parameter α. It is amed after the british physicist Lord Rayleigh (84 99), also kow as Baro Joh William Strutt Rayleigh of Terlig Place ad Nobel prize wier i physics 94. Note that the parameter α is simply a scale factor ad that the variable y = x/α has the simplified distributio g(y) = ye y /. Figure 3: The Rayleigh distributio The distributio, show i figure 3, has a mode at x = α ad is positively skewed. 37. Momets Algebraic momets are give by E(x ) = x f(x)dx = α x + e x /α i.e. we have a coectio to the absolute momets of the Gauss distributio. Usig these (see sectio 34 o the ormal distributio) the result is E(x ) = { π!!α for odd k k!α k for = k 38

Specifically we ote that the expectatio value, variace, ad the third ad fourth cetral momets are give by ( π E(x) = α, V (x) = α π ) ) π, µ 3 = α 3 (π 3), ad µ 4 = α (8 4 3π 4 The coefficiets of skewess ad kurtosis is thus γ = (π 3) π ( ) 3.63 ad γ = 8 3π 4 ( ) 3.459 π π 37.3 Cumulative Distributio The cumulative distributio, or the distributio fuctio, is give by F (x) = x f(y)dy = a x ye y α dy = x α e z dz = e x α where we have made the substitutio z = y i order to simplify the itegratio. As it α should we see that F () = ad F ( ) =. Usig this we may estimate the media M by F (M) = M = α l.774α ad the lower ad upper quartiles becomes Q = α l 3.75853α 4 ad Q 3 = α l 4.665α ad the same techique is useful whe geeratig radom umbers from the Rayleigh distributio as is described below. 37.4 Two-dimesioal Kietic Theory Give two idepedet coordiates x ad y from ormal distributios with zero mea ad the same variace σ the distace z = x + y is distributed accordig to the Rayleigh distributio. The x ad y may e.g. be regarded as the velocity compoets of a particle movig i a plae. To realize this we first write w = z σ = x σ + y σ Sice x/σ ad y/σ are distributed as stadard ormal variables the sum of their squares has the chi-squared distributio with degrees of freedom i.e. g(w) = e w/ / from which we fid f(z) = g(w) dw dz ( ) z z = g σ σ = z σ e which we recogize as the Rayleigh distributio. This may be compared to the threedimesioal case where we ed up with the Maxwell distributio. 39 z σ

37.5 Radom Number Geeratio To obtai radom umbers from the Rayleigh distributio i a efficiet way we make the trasformatio y = x /α a variable which follow the expoetial distributio g(y) = e y. A radom umber from this distributio is easily obtaied by takig mius the atural logarithm of a uiform radom umber. We may thus fid a radom umber r from a Rayleigh distributio by the expressio r = α l ξ where ξ is a radom umber uiformly distributed betwee zero ad oe. This could have bee foud at oce usig the cumulative distributio puttig F (x) = ξ e x α = ξ x = α l( ξ) a result which is idetical sice if ξ is uiformly distributed betwee zero ad oe so is ξ. Followig the examples give above we may also have used two idepedet radom umbers from a stadard ormal distributio, z ad z, ad costruct r = α z + z However, this techique is ot as efficiet as the oe outlied above. 4

38 Studet s t-distributio 38. Itroductio The Studet s t-distributio is give by f(t; ) = Γ ( ) + ( ( ) πγ + t ) + = ( + t ) + B (, ) where the parameter is a positive iteger ad the variable t is a real umber. The fuctios Γ ad B are the usual Gamma ad Beta fuctios. I figure 3 we show the t-distributio for values of (lowest maxima),, 5 ad (fully draw ad idetical to the stadard ormal distributio). Figure 3: Graph of t-distributio for some values of If we chage variable to x = t/ ad put m = + the Studet s t-distributio becomes f(x; m) = k ( + x ) with k = Γ(m) m Γ ( ) ( ) = Γ m B (, m ) where k is simply a ormalizatio costat ad m is a positive half-iteger. 38. History A brief history behid this distributio ad its ame is the followig. William Sealy Gosset (876-937) had a degree i mathematics ad chemistry from Oxford whe he i 899 bega workig for Messrs. Guiess brewery i Dubli. I his work at the brewery he developed a small-sample theory of statistics which he eeded i makig small-scale experimets. 4

Due to compay policy it was forbidde for employees to publish scietific papers ad his work o the t-ratio was published uder the pseudoym Studet. It is a very importat cotributio to statistical theory. 38.3 Momets The Studet s t-distributio is symmetrical aroud t = ad thus all odd cetral momets vaish. I calculatig eve momets (ote that algebraic ad cetral momets are equal) we make use of the somewhat simpler f(x; m) form give above with x = t which implies the followig relatio betwee expectatio values E(t r ) = r E(x r ). Cetral momets of eve order are give by, with r a iteger, µ r (x) = f(x; m)dx = k If we make the substitutio y = x dx ad we obtai (+x ) µ r (x) = k = k x r x r x r dx = k ( + x ) m ( + x ) dx m x implyig = y ad x = y +x +x y ( + x ) ( + x ) m x ( y) m ( y y dy = k ) r dy = k x r dy = ( + x ) m = kb(r +, m r ) = B(r +, m r ) B(, m ) ( y) m r 3 y r dy = the dy = The ormalizatio costat k was give above ad we may ow verify this expressio by lookig at µ = givig k = /B(, m ) ad thus fially, icludig the r factor givig momets i t we have µ r (t) = r µ r (x) = r B(r +, m r ) B(, m ) = r B(r +, r) B(, ) As ca be see from this expressio we get ito problems for r ad ideed those momets are udefied or diverget 9. The formula is thus valid oly for r <. A recursive formula to obtai eve algebraic momets of the t-distributio is µ r = µ r r r startig with µ =. Especially we ote that, whe is big eough so that these momets are defied, the secod cetral momet (i.e. the variace) is µ = V (t) = ad the fourth cetral 3 momet is give by µ 4 = ( )( 4) γ = ad γ = 6, respectively. 4. The coefficiets of skewess ad kurtosis are give by 9 See e.g. the discussio i the descriptio of the momets for the Cauchy distributio which is the special case where m = =. 4

38.4 Cumulative Fuctio I calculatig the cumulative fuctio for the t-distributio it turs out to be simplifyig to first estimate the itegral for a symmetric regio t t f(u)du = = = = = t ( B, ) ( t ( t ( B, ) B (, ) B (, ) B (, ) +t + u + u ( x) +t ( B (, = I +t (, ) + ) + du = du = x + x x x dx = x dx = ) ( B, )) +t = ) = I t +t (, ) where we have made the substitutio x = /( + u ) i order to simplify the itegratio. From this we fid the cumulative fuctio as I ( t, ) for < x < +t F (t) = + I ( x, ) +x for x < 38.5 Relatios to Other Distributios The distributio i F = t is give by ( dt f(f ) = df f(t) = + F F ) + B(, ) = F B(, + )(F + ) which we recogize as a F -distributio with ad degrees of freedom. As the Studet s t-distributio approaches the stadard ormal distributio. However, a better approximatio tha to create a simplemided stadardized variable, dividig by the square root of the variace, is to use z = t ( ) 4 + t which is more closely distributed accordig to the stadard ormal distributio. 43

38.6 t-ratio Regard t = x/ y where x ad y are idepedet variables distributed accordig to the stadard ormal ad the chi-square distributio with degrees of freedom, respectively. The idepedece implies that the joit probability fuctio i x ad y is give by f(x, y; ) = ( e x π ) y e y Γ ( ) where < x < ad y >. If we chage variables to t = x/ y distributio i t ad u, with < t < ad u >, becomes The determiat is u u f(t, u; ) = (x, y) f(t, u; ) = f(x, y; ) (t, u) ad thus we have ( e ut π ) u e u Γ ( ) = u (+) e ( ) u + t πγ ( ) + ad u = y the Fially, sice we are iterested i the margial distributio i t we itegrate over u ( ) f(t; ) = f(t, u; )du = ( ) πγ + u + e u + t du = = = πγ ( ) + ( + t ) + πγ ( ) ( v + t ) + v + e v dv = ( + t where we made the substitutio v = u the last step is recogized as beig equal to Γ ( ) +. 38.7 Oe Normal Sample e v dv ( ) = + t ( + t ) + πγ ( ) Γ ( + ) + ( ) + t = ( B, ) ) i order to simplify the itegral which i Regard a sample from a ormal populatio N(µ, σ ) where the mea value x is distributed as N(µ, σ ( )s ) ad is distributed accordig to the chi-square distributio with σ degrees of freedom. Here s is the usual ubiased variace estimator s = (x i x) i= which i the case of a ormal distributio is idepedet of x. This implies that t = x µ σ/ ( )s σ /( ) = x µ s/ is distributed accordig to Studet s t-distributio with degrees of freedom. We may thus use Studet s t-distributio to test the hypothesis that x = µ (see below). 44

38.8 Two Normal Samples Regard two samples {x, x,..., x m } ad {y, y,..., y } from ormal distributios havig the same variace σ but possibly differet meas µ x ad µ y, respectively. The the quatity (x y) (µ x µ y ) has a ormal distributio with zero mea ad variace equal to σ ( + ) m. Furthermore the pooled variace estimate m s = (m )s x + ( )s (x i x) + (y i y) y i= i= = m + m + is a ormal theory estimate of σ with m + degrees of freedom. Sice s is idepedet of x for ormal populatios the variable t = (x y) (µ x µ y ) s m + has the t-distributio with m + degrees of freedom. We may thus use Studet s t-distributio to test the hypotheses that x y is cosistet with δ = µ x µ y. I particular we may test if δ = i.e. if the two samples origiate from populatio havig the same meas as well as variaces. 38.9 Paired Data If observatios are made i pairs (x i, y i ) for i =,,..., the appropriate test statistic is t = d s d = d s d / = d (d i d) i= ( ) where d i = x i y i ad d = x y. This quatity has a t-distributio with degrees of freedom. We may also write this t-ratio as d t = s x + s y C xy where s x ad s y are the estimated variaces of x ad y ad C xy is the covariace betwee them. If we would ot pair the data the covariace term would be zero but the umber of degrees of freedom i.e. twice as large. The smaller umber of degrees of freedom i the paired case is, however, ofte compesated for by the iclusio of the covariace. 38. Cofidece Levels I determiig cofidece levels or testig hypotheses usig the t-distributio we defie the quatity t α, from F (t α, ) = t α, f(t; )dt = α If y is a ormal theory estimate of σ with k degrees of freedom the ky/σ is distributed accordig to the chi-square distributio with k degrees of freedom. 45

i.e. α is the probability that a variable distributed accordig to the t-distributio with degrees of freedom exceeds t α,. Note that due to the symmetry about zero of the t-distributio t α, = t α,. I the case of oe ormal sample described above we may set a α cofidece iterval for µ x s t α/, µ x + s t α/, Note that i the case where σ is kow we would ot use the t-distributio. The appropriate distributio to use i order to set cofidece levels i this case would be the ormal distributio. 38. Testig Hypotheses As idicated above we may use the t-statistics i order to test hypotheses regardig the meas of populatios from ormal distributios. I the case of oe sample the ull hypotheses would be H : µ = µ ad the alterative hypothesis H : µ µ. We would the use t = x µ s/ as outlied above ad reject H at the α cofidece level of sigificace if t > s t α/,. This test is two-tailed sice we do ot assume ay a priori kowledge of i which directio a evetual differece would be. If the alterate hypothesis would be e.g. H : µ > µ the a oe-tailed test would be appropriate. The probability to reject the hypothesis H if it is ideed true is thus α. This is a so called Type I error. However, we might also be iterested i the probability of committig a Type II error implyig that we would accept the hypothesis although it was wrog ad the distributio istead had a mea µ. I addressig this questio the t-distributio could be modified yieldig the o-cetral t-distributio. The probability cotet β of this distributio i the cofidece iterval used would the be the probability of wrogly acceptig the hypothesis. This calculatio would deped o the choice of α as well as o the umber of observatios. However, we do ot describe details about this here. I the two sample case we may wat to test the ull hypothesis H : µ x = µ y as compared to H : µ x µ y. Oce agai we would reject H if the absolute value of the quatity t = (x y)/s + would exceed t m α/,+m. 38. Calculatio of Probability Cotet I order to fid cofidece itervals or to test hypotheses we must be able to calculate itegrals of the probability desity fuctio over certai regios. We recall the formula F (t α, ) = t α, f(t; )dt = α which defies the quatity t α, for a specified cofidece level α. The probability to get a value equal to t α, or higher is thus α. Classically all text-books i statistics are equipped with tables givig values of t α, for specific α-values. This is sometimes useful ad i table 8 o page 78 we show such a 46

table givig poits where the distributio has a cumulative probability cotet of α for differet umber of degrees of freedom. However, it is ofte preferable to calculate directly the exact probability that oe would observe the actual t-value or worse. To calculate the itegral o the left-had side we differ betwee the case where the umber of degrees of freedom is a odd or eve iteger. The equatio above may either be adjusted such that a required α is obtaied or we may replace t α, with the actual t-value foud i order to calculate the probability for the preset outcome of the experimet. The algorithm proposed for calculatig the probability cotet of the t-distributio is described i the followig subsectios. 38.. Eve umber of degrees of freedom For eve we have puttig m = ad makig the substitutio x = t α = t α, f(t; )dt = Γ ( ) + ( ) πγ t α, dt ( + t ) + ) = Γ ( m + πγ(m) tα,/ dx ( + x ) m+ For coveiece (or maybe it is rather laziess) we make use of stadard itegral tables where we fid the itegral dx (ax + c) m+ = m x m r (m )!m!(r)! ax + c r= (m)!(r!) c m r (ax + c) r where i our case a = c =. Itroducig x α = t α, / for coveiece this gives α = Γ ( ) m + (m )!m! m x α πγ(m)(m)! + x α m r= (r)! r (r!) ( + x α) r + The last term iside the brackets is the value of the itegrad at which is see to equal. Lookig at the factor outside the brackets usig that Γ() = ( )! for a positive iteger, Γ ( ) m + = (m )!! π, ad rewritig (m)! = (m)!!(m )!! = m m!(m )!! m we fid that it i fact is equal to oe. We thus have α = x α + x α m r= (r)! r (r!) ( + x α) r + I evaluatig the sum it is useful to look at the idividual terms. Deotig these by u r we fid the recurrece relatio where we start with u =. u r = u r r(r ) r ( + x α) = u r r + x α 47

To summarize: i order to determie the probability α to observe a value t or bigger from a t-distributio with a eve umber of degrees of freedom we calculate where u = ad u r = u r α = r +t /. t + t 38.. Odd umber of degrees of freedom m r= u r + For odd we have puttig m = ad makig the substitutio x = t ) α = t α, f(t; )dt = Γ ( + ( ) πγ t α, dt ( + t ) + = Γ(m + ) ( πγ m + ) x α dx ( + x ) m+ where we agai have itroduced x α = t α, /. Oce agai we make use of stadard itegral tables where we fid the itegral [ dx (m)! x m r!(r )! (a + bx m+ = ) (m!) a (4a) m r (r)! (a + bx ) r + ] dx (4a) m a + bx r= where i our case a = b =. We obtai α = [ Γ(m + )(m)! xα ( ) πγ m + m! 4 m m r= 4 r r!(r )! (r)! ( + x α) r + arcta x α + π ] where the last term iside the brackets is the value of the itegrad at. The factor outside the brackets is equal to which is foud usig that Γ() = ( )! for a positive π iteger, Γ ( ) m + = (m )!! π, ad (m)! = (m)!!(m )!! = m (m)!(m )!!. We m get α = [ xα m 4 r r!(r )! π r= (r)! ( + x α) r + arcta x α + π ] = = [ xα m r r!(r )! π + x α (r)!( + x α) + arcta x r α + π ] r= To compute the sum we deote the terms by v r ad fid the recurrece relatio ( ) 4r(r ) v r = v r r(r )( + x α) = v r r ( + x α) startig with v =. To summarize: i order to determie the probability α to observe a value t or bigger from a t-distributio with a odd umber of degrees of freedom we calculate t α = π + t where v = ad v r = v r r +t /. r= 48 v r + arcta t +

38..3 Fial algorithm The fial algorithm to evaluate the probability cotet from to t for a t-distributio with degrees of freedom is Calculate x = For eve: t Put m = Set u =, s = ad i =. For i =,,,..., m set s = s + u i, i = i + ad u i = u i i +x. α = For odd: x +x s. Put m =. Set v =, s = ad i =. For i =,,..., m set s = s + v i, i = i + ad v i = v i i +x. α = π ( x +x s + arcta x). 38.3 Radom Number Geeratio Followig the defiitio we may defie a radom umber t from a t-distributio, usig radom umbers from a ormal ad a chi-square distributio, as t = z y / where z is a stadard ormal ad y a chi-squared variable with degrees of freedom. To obtai radom umbers from these distributios see the appropriate sectios. 49

39 Triagular Distributio 39. Itroductio The triagular distributio is give by f(x; µ, Γ) = x µ Γ where the variable x is bouded to the iterval µ Γ x µ + Γ ad the locatio ad scale parameters µ ad Γ (Γ > ) all are real umbers. 39. Momets The expectatio value of the distributio is E(x) = µ. Due to the symmetry of the distributio odd cetral momets vaishes while eve momets are give by µ = Γ ( + )( + ) for eve values of. I particular the variace V (x) = µ = Γ /6 ad the fourth cetral momet µ 4 = Γ 4 /5. The coefficiet of skewess is zero ad the coefficiet of kurtosis γ =.6. 39.3 Radom Number Geeratio The sum of two pseudoradom umbers uiformly distributed betwee (µ Γ)/ ad (µ + Γ)/ is distributed accordig to the triagular distributio. If ξ ad ξ are uiformly distributed betwee zero ad oe the + Γ x = µ + (ξ + ξ )Γ or x = µ + (ξ ξ )Γ follow the triagular distributio. Note that this is a special case of a combiatio x = (a + b)ξ + (b a)ξ b with b > a which gives a radom umber from a symmetric trapezoidal distributio with vertices at (±b, ) ad (±a, a+b ). 5

4 Uiform Distributio 4. Itroductio The uiform distributio is, of course, a very simple case with f(x; a, b) = b a for a x b The cumulative, distributio, fuctio is thus give by 4. Momets if x a x a F (x; a, b) = if a x b b a if b x The uiform distributio has expectatio value E(x) = (a + b)/, variace V (x) = (b a) /, µ 3 =, µ 4 = (b a) 4 /8, coefficiet of skewess γ = ad coefficiet of kurtosis γ =.. More geerally all odd cetral momets vaish ad for a eve iteger µ = (b a) ( + ) 4.3 Radom Number Geeratio Sice we assume the presece of a pseudoradom umber geerator givig radom umbers ξ betwee zero ad oe a radom umber from the uiform distributio is simply give by x = (b a)ξ + a 5

4 Weibull Distributio 4. Itroductio The Weibull distributio is give by f(x; η, σ) = η σ ( x σ ) η e ( x σ) η where the variable x ad the parameters η ad σ all are positive real umbers. The distributio is amed after the swedish physicist Waloddi Weibull (887 979) a professor at the Techical Highschool i Stockholm 94 953. The parameter σ is simply a scale parameter ad the variable y = x/σ has the distributio g(y) = η y η e yη I figure 3 we show the distributio for a few values of η. For η < the distributio has its mode at y =, at η = it is idetical to the expoetial distributio, ad for η > the distributio has a mode at x = ( η which approaches x = as η icreases (at the same time the distributio gets more symmetric ad arrow). η ) η Figure 3: The Weibull distributio 5

4. Cumulative Distributio The cumulative distributio is give,by F (x) = x f(u)du = x ( ) η u η e ( σ) u η du = σ σ (x/σ) η e y dy = e ( x σ) η where we have made the substitutio y = (u/σ) η i order to simplify the itegratio. 4.3 Momets Algebraic momets are give by E(x k ) = x k f(x)dx = σ k y k η e y dy = σ k Γ ( ) k η + where we have made the same substitutio as was used whe evaluatig the cumulative distributio above. Especially the expectatio value ad the variace are give by E(x) = σγ ( ) η + 4.4 Radom Number Geeratio ( ) ( ) ad V (x) = σ Γ η + Γ η + To obtai radom umbers from Weibull s distributio usig ξ, a radom umber uiformly distributed from zero to oe, we may solve the equatio F (x) = ξ to obtai a radom umber i x. F (x) = e ( x σ) η = ξ x = σ( l ξ) η 53

4 Appedix A: The Gamma ad Beta Fuctios 4. Itroductio I statistical calculatios for stadard statistical distributios such as the ormal (or Gaussia) distributio, the Studet s t-distributio, the chi-squared distributio, ad the F - distributio oe ofte ecouters the so called Gamma ad Beta fuctios. More specifically i calculatig the probability cotet for these distributios the icomplete Gamma ad Beta fuctios occur. I the followig we briefly defie these fuctios ad give umerical methods o how to calculate them. Also coectios to the differet statistical distributios are give. The mai refereces for this has bee [4,4,43] for the formalism ad [] for the umerical methods. 4. The Gamma Fuctio The Gamma fuctio is ormally defied as Γ(z) = t z e t dt where z is a complex variable with Re(z) >. This is the so called Euler s itegral form for the Gamma fuctio. There are, however, two other defiitios worth metioig. Firstly Euler s ifiite limit form 3 Γ(z) = lim z(z + )(z + ) (z + ) z z,,,... ad secodly the ifiite product form sometimes attributed to Euler ad sometimes to Weierstrass ( Γ(z) = zγz + z ) e z z < = where γ.57756649 is Euler s costat. I figure 33 we show the Gamma fuctio for real argumets from 5 to 5. Note the sigularities at x =,,,.... For z a positive real iteger we have the well kow relatio to the factorial fuctio! = Γ( + ) ad, as the factorial fuctio, the Gamma fuctio satisfies the recurrece relatio Γ(z + ) = zγ(z) I the complex plae Γ(z) has a pole at z = ad at all egative iteger values of z. The reflectio formula Γ( z) = π Γ(z) si(πz) = πz Γ(z + ) si(πz) may be used i order to get fuctio values for Re(z) < from values for Re(z) >. 54

Figure 33: The Gamma fuctio A well kow approximatio to the Gamma fuctio is Stirlig s formula Γ(z) = z z e z π z ( + z + 88z 39 584z 3 57 4883z 4 +... ) for arg z < π ad z ad where ofte oly the first term () i the series expasio is kept i approximate calculatios. For the faculty of a positive iteger oe ofte uses the approximatio! π e which has the same origi ad also is called Stirlig s formula. 4.. Numerical Calculatio There are several methods to calculate the Gamma fuctio i terms of series expasios etc. For umerical calculatios, however, the formula by Laczos is very useful [] Γ(z + ) = ( z + γ + ) z+ e (z+γ+ ) [ π c + c z + + c z + + + c ] z + + ɛ for z > ad a optimal choice of the parameters γ,, ad c to c. For γ = 5, = 6 ad a certai set of c s the error is smaller tha ɛ <. This boud is true for all complex z i the half complex plae Re(z) >. The coefficiets ormally used are c =, c = 76.8973, c = -86.55333, c 3 = 4.498, c 4 = -.373956, c 5 =.8583, ad c 6 = -.53638. Use the reflectio formula give above to obtai results for Re(z) < e.g. for egative real argumets. Beware, however, to avoid the sigularities. While implemetig routies for the Gamma fuctio it is recommedable to evaluate the atural logarithm i order to avoid umerical overflow. 55

A alterative way of evaluatig l Γ(z) is give i refereces [44,45], givig formulæ which also are used i order to evaluate the Digamma fuctio below. The expressios used for l Γ(z) are ( ) z l z z + l π + z K B k k(k ) z k + R K (z) for < x x k= l Γ(z) = l Γ(z + ) l (z + k) for x < x k= l π + l Γ( z) l si πz for x < Here = [x ] [x] (the differece of iteger parts, where x is the real part of z = x + ıy) ad e.g. K = ad x = 7. gives excellet accuracy i.e. small R K. Note that Kölbig [45] gives the wrog sig o the (third) costat term i the first case above. 4.. Formulæ Below we list some useful relatios cocerig the Gamma fuctio, faculties ad semifaculties (deoted by two exclamatio marks here). For a more complete list cosult e.g. [4]. Γ(z) = ( l t ) z dt Γ(z + ) = zγ(z) = z! Γ(z) = α z t z e αt dt for Re(z) >, Re(α) > Γ(k) = (k )! for k (iteger,! = ) z! = Γ(z + ) = Γ ( ) = π Γ ( ) + ( )!! = π π Γ(z)Γ( z) = si πz πz z!( z)! = si πz (m)!! = 4 6 m = m m! (m )!! = 3 5 (m ) 4.3 Digamma Fuctio e t t z dt for Re(z) > (m)! = (m)!!(m )!! = m m!(m )!! It is ofte coveiet to work with the logarithm of the Gamma fuctio i order to avoid umerical overflow i the calculatios. The first derivatives of this fuctio ψ(z) = d dz dγ(z) l Γ(z) = Γ(z) dz 56

is kow as the Digamma, or Psi, fuctio. A series expasio of this fuctio is give by ( ψ(z + ) = γ z + ) for z,,, 3,... = where γ.57756649 is Euler s costat which is see to be equal to ψ(). If the derivative of the Gamma fuctio itself is required we may thus simply use dγ(z)/dz = Γ(z) ψ(z). Note that some authors write ψ(z) = d l Γ(z + ) = d z! for the Digamma dz dz fuctio, ad similarly for the polygamma fuctios below, thus shiftig the argumet by oe uit. I figure 34 we show the Digamma fuctio for real argumets from 5 to 5. Note the sigularities at x =,,,.... For iteger values of z we may write Figure 34: The Digamma, or Psi, fuctio ψ() = γ + which is efficiet eough for umerical calculatios for ot too large values of. Similarly for half-iteger values we have ψ ( + m= m ) = γ l + m= m However, for arbitrary argumets the series expasio above is uusable. Followig the recipe give i a article by K. S. Kölbig [45] we use l z K B k z k z k + R K (z) for < x x k= ψ(z) = ψ(z + ) for x < x z+k k= ψ( z) + + π cot πz for x < z 57

Here = [x ] [x] (the differece of iteger parts, where x is the real part of z = x + ıy) ad we have chose K = ad x = 7. which gives a very good accuracy (i.e. small R K, typically less tha 5 ) for double precisio calculatios. The mai iterest i statistical calculatios is ormally fuctio values for ψ(x) for real positive argumets but the formulæ above are valid for ay complex argumet except for the sigularities alog the real axis at z =,,, 3,.... The B k are Beroulli umbers give by B =, B =, B =, B 6 4 =, B 3 6 =, B 4 8 =, B 3 = 5, B 66 = 69, B 73 4 = 7, B 6 6 = 367, B 5 8 = 43867, B 798 = 746... 33 4.4 Polygamma Fuctio Higher order derivatives of l Γ(z) are called Polygamma fuctios ψ () (z) = d d+ ψ(z) = l Γ(z) for =,, 3,... dz dz+ Here a series expasio is give by ψ () (z) = ( ) +! k= (z + k) + for z,,,... For umerical calculatios we have adopted a techique similar to what was used to evaluate l Γ(z) ad ψ(z). ( ) [t +! + K ψ () z (z) = + k= ψ () (z + m) ( )! m k= ] (k+ )! B k + R (k)!z k+ K (z) for < x x (z+k) + for x < x where t = l z for = ad t = ( )!/z for >. Here m = [x ] [x] i.e. the differece of iteger parts, where x is the real part of z = x + ıy. We treat primarily the case for real positive argumets x ad if complex argumets are required oe ought to add a third reflectio formula as was doe i the previous cases. Without ay special optimizatio we have chose K = 4 ad x = 7. which gives a very good accuracy, i.e. small R K, typically less tha 5, eve for double precisio calculatios except for higher orders ad low values of x where the fuctio value itself gets large. For more relatios o the Polygamma (ad the Digamma) fuctios see e.g. [4]. Two useful relatios used i this documet i fidig cumulats for some distributios are ψ () () = ( ) +!ζ( + ) ψ () ( ) = ( )+!( + )ζ( + ) = ( + )ψ () () where ζ is Riema s zeta fuctio (see page 59 ad [3]). Sometimes the more specific otatio tri-, tetra-, peta- ad hexagamma fuctios are used for ψ, ψ, ψ (3) ad ψ (4), respectively. For this calculatio we eed a few more Beroulli umbers ot give o page 58 above amely B = 85453 38, B 4 = 363649 73, B 6 = 85533 6, ad B 8 = 3749469 87 58

4.5 The Icomplete Gamma Fuctio For the icomplete Gamma fuctio there seem to be several defiitios i the literature. Defiig the two itegrals γ(a, x) = x t a e t dt ad Γ(a, x) = x t a e t dt with Re(a) > the icomplete Gamma fuctio is ormally defied as P (a, x) = γ(a, x) Γ(a) but sometimes also γ(a, x) ad Γ(a, x) is referred to uder the same ame as well as the complemet to P (a, x) Γ(a, x) Q(a, x) = P (a, x) = Γ(a) Note that, by defiitio, γ(a, x) + Γ(a, x) = Γ(a). I figure 35 the icomplete Gamma fuctio P (a, x) is show for a few a-values (.5,, 5 ad ). Figure 35: The icomplete Gamma fuctio 4.5. Numerical Calculatio For umerical evaluatios of P two formulæ are useful []. For values x < a + the series γ(a, x) = e x x a = 59 Γ(a) Γ(a + + ) x

coverges rapidly while for x a + the cotiued fractio ( Γ(a, x) = e x x a a a ) x+ + x+ + x+ is a better choice. 4.5. Formulæ Below we list some relatios cocerig the icomplete Gamma fuctio. complete list cosult e.g. [4]. Γ(a) = γ(a, x) + Γ(a, x) γ(a, x) = x e t t a dt for Re(a) > γ(a +, x) = aγ(a, x) x a e x γ(, x) = ( )! Γ(a, x) = x e t t a dt [ e x Γ(a +, x) = aγ(a, x) x a e x Γ(, x) = ( )!e x r= x r r! r= x r ] r! =,,... For a more 4.5.3 Special Cases The usage of the icomplete Gamma fuctio P (a, x) i calculatios made i this documet ofte ivolves iteger or half-iteger values for a. These cases may be solved by the followig formulæ P (, x) = e x P (, x) = erf x k= P (a +, x) = P (a, x) xa e x Γ(a + ) = P (a, x) xa e x aγ(a) P (, x) = erf x the last formula for odd values of. 4.6 The Beta Fuctio x k k! k= x k e x Γ ( k+ ) = erf x e x x π The Beta fuctio is defied through the itegral formula B(a, b) = B(b, a) = 6 t a ( t) b dt k= (x) k (k )!!

ad is related to the Gamma fuctio by B(a, b) = Γ(a)Γ(b) Γ(a + b) The most straightforward way to calculate the Beta fuctio is by usig this last expressio ad a well optimized routie for the Gamma fuctio. I table 9 o page 79 expressios for the Beta fuctio for low iteger ad half-iteger argumets are give. Aother itegral, obtaied by the substitutio x = t/( t), yieldig the Beta fuctio is x a B(a, b) = dx ( + x) a+b 4.7 The Icomplete Beta Fuctio The icomplete Beta fuctio is defied as I x (a, b) = B x x(a, b) B(a, b) = t a ( t) b dt B(a, b) for a, b > ad x. The fuctio B x (a, b), ofte also called the icomplete Beta fuctio, satisfies the followig formula B x (a, b) = x x u a ( + u) a+b du = B (b, a) B x (b, a) = [ = x a a + b ( b)( b) x + x + a +!(a + ) + ( b)( b) ( b) x +!(a + ) I figure 36 the icomplete Beta fuctio is show for a few (a, b)-values. Note that by symmetry the (, 5) ad (5, ) curves are reflected aroud the diagoal. For large values of a ad b the curve rises sharply from ear zero to ear oe aroud x = a/(a + b). 4.7. Numerical Calculatio I order to obtai I x (a, b) the series expasio I x (a, b) = xa ( x) b ab(a, b) [ + = ] ] B(a +, + ) B(a + b, + ) x+ is ot the most useful formula for computatios. The cotiued fractio formula I x (a, b) = xa ( x) b ab(a, b) 6 [ ] d d + + +

Figure 36: The icomplete Beta fuctio turs out to be a better choice []. Here (a + m)(a + b + m)x d m+ = (a + m)(a + m + ) ad d m = m(b m)x (a + m )(a + m) ad the formula coverges rapidly for x < (a + )/(a + b + ). For other x-values the same formula may be used after applyig the symmetry relatio 4.7. Approximatio I x (a, b) = I x (b, a) For higher values of a ad b, well already from a + b > 6, the icomplete Beta fuctio may be approximated by For (a + b + )( x).8 usig a approximatio to the chi-square distributio i the variable χ = (a + b )( x)(3 x) ( x)(b ) with = b degrees of freedom. For (a+b+)( x).8 usig a approximatio to the stadard ormal distributio i the variable z = 3 [ ( ) ( )] w 9b w 9a w + w b a where w = 3 bx ad w = 3 a( x) I both cases the maximum differece to the true cumulative distributio is below.5 all way dow to the limit where a + b = 6 [6]. 6

4.8 Relatios to Probability Desity Fuctios The icomplete Gamma ad Beta fuctios, P (a, x) ad I x (a, b) are related to may stadard probability desity fuctios or rather to their cumulative (distributio) fuctios. We give very brief examples here. For more details o these distributios cosult ay book i statistics. 4.8. The Beta Distributio The cumulative distributio for the Beta distributio with parameters p ad q is give by F (x) = i.e. simply the icomplete Beta fuctio. 4.8. The Biomial Distributio x t p ( t) q dt = B x(p, q) B(p, q) B(p, q) = I x(p, q) For the biomial distributio with parameters ad p j=k ( ) p j ( p) j = I p (k, k+) j i.e. the cumulative distributio may be obtaied by P (k) = k i= ( ) p i ( p) i = I p ( k, k+) i However, be careful to evaluate P (), which obviously is uity, usig the icomplete Beta fuctio sice this is ot defied for argumets which are less or equal to zero. 4.8.3 The Chi-squared Distributio The cumulative chi-squared distributio for degrees of freedom is give by F (x) = Γ ( ) x ) = γ (, x Γ ( ) = P (, x ( x ) e x dx = ) Γ ( ) x y e y dy = where x is the chi-squared value sometimes deoted χ. I this calculatio we made the simple substitutio y = x/ i simplifyig the itegral. 63

4.8.4 The F -distributio The cumulative F -distributio with m ad degrees of freedom is give by F (x) = = B ( m, ) x B ( m, ) ( m, ) m m F m (mf + ) m+ mx mx+ y m ( y) = B z ( B ( m, ) = I m z, ) df = B ( m, ) x dy y( y) = B ( m, ) ( ) mf m ( ) df mf + mf + F = mx mx+ y m ( y) dy = with z = mx/( + mx). Here we have made the substitutio y = mf/(mf + ), leadig to df/f = dy/y( y), i simplifyig the itegral. 4.8.5 The Gamma Distributio Not surprisigly the cumulative distributio for the Gamma distributio with parameters a ad b is give by a icomplete Gamma fuctio. F (x) = = x Γ(b) f(x)dx = ax ab x u b e au du = Γ(b) v b e v dv = γ(b, ax) Γ(b) ab Γ(b) = P (b, ax) ax ( v a ) b e v dv a = 4.8.6 The Negative Biomial Distributio The egative biomial distributio with parameters ad p is related to the icomplete Beta fuctio via the relatio ( ) + s p ( p) s = I p (a, ) s s=a Also the geometric distributio, a special case of the egative biomial distributio, is coected to the icomplete Beta fuctio, see summary below. 4.8.7 The Normal Distributio The cumulative ormal, or Gaussia, distributio is give by 3 F (x) = + P (, ) x if x P (, ) x if x < 3 Without loss of geerality it is eough to regard the stadard ormal desity. 64

where P (, ) x is the icomplete Gamma fuctio occurrig as twice the itegral of the stadard ormal curve from to x sice x π e t dt = π = γ ( x, ) x Γ ( e u u du = ) = P ( Γ ( ), ) x x u e u du = The so called error fuctio may be expressed i terms of the icomplete Gamma fuctio erf x = x e t dt = P (, π x) as is the case for the complemetary error fuctio erfc x = erf x = π x e t dt = P (, x) defied for x, for x < use erf( x) = erf(x) ad erfc( x) = + erf(x). See also sectio 3. There are also other series expasios for erf x like erf x = [ ] x x3 π 3! + x5 5! x7 7 3! +... = 4.8.8 The Poisso Distributio [ = e x πx x + 3 (x ) 3 5 (x ) +... 3 Although the Poisso distributio is a probability desity fuctio i a discrete variable the cumulative distributio may be expressed i terms of the icomplete Gamma fuctio. The probability for outcomes from zero to k iclusive for a Poisso distributio with parameter (mea) µ is ] P µ (< k) = k = µ e µ! = P (k, µ) for k =,,... 4.8.9 Studet s t-distributio The symmetric itegral of the t-distributio with degrees of freedom, ofte deoted A(t ), is give by A(t ) = t ( B, ) t ( + x ) + dx = t ( B, ) ( ) + dx = + x 65

= = B (, ) B (, ) t +t t +t ( y) + y ( y) dy = B z ( ) y y y dy = (, ) B (, ) = I z (, ) with z = t /( + t ). 4.8. Summary The followig table summarizes the relatios betwee the cumulative, distributio, fuctios of some stadard probability desity fuctios ad the icomplete Gamma ad Beta fuctios. Distributio Parameters Cumulative distributio Rage Beta p, q F (x) = I x (p, q) x Biomial, p P (k) = I p ( k, k+) k =,,..., Chi-squared F (x) = P (, ) x x ( F m, F (x) = I mx m, ) x +mx Gamma a, b F (x) = P (b, ax) x Geometric p P (k) = I p (, k) k =,,... Negative biomial, p P (k) = I p (, k+) k =,,... Stadard ormal F (x) = P (, ) x < x < F (x) = + P (, ) x x < Poisso µ P (k) = P (k+, µ) k =,,... Studet F (x) = I ( x, ) < x < +x F (x) = + I ( x, ) x < +x 66

43 Appedix B: Hypergeometric Fuctios 43. Itroductio The hypergeometric ad the cofluet hypergeometric fuctios has a cetral role iasmuch as may stadard fuctios may be expressed i terms of them. This appedix is based o iformatio from [4,46,47] i which much more detailed iformatio o the hypergeometric ad cofluet hypergeometric fuctio may be foud. 43. Hypergeometric Fuctio The hypergeometric fuctio, sometimes called Gauss s differetial equatio, is give by [4,46] x( x) f(x) + [c (a + b + )x] f(x) x x abf(x) = Oe solutio is f(x) = F (a, b, c; x) = + ab c x! a(a + )b(b + ) x + + c,,, 3,... c(c + )! The rage of covergece is x < ad x =, for c > a + b, ad x =, for c > a + b. Usig the so called Pochhammer symbol (a) = a(a + )(a + ) (a + ) = (a + )! (a )! = Γ(a + ) Γ(a) with (a) = this solutio may be writte 4 as F (a, b, c; x) = = (a) (b) (c) x! = Γ(c) Γ(a)Γ(b) = Γ(a + )Γ(b + ) x Γ(c + )! By symmetry F (a, b, c; x) = F (b, a, c; x) ad sometimes the idices are dropped ad whe the risk for cofusio is egligible oe simply writes F (a, b, c; x). Aother idepedet solutio to the hypergeometric equatio is ad f(x) = x c F (a+ c, b+ c, c; x) c, 3, 4,... The :th derivative of the hypergeometric fuctio is give by d dx F (a, b, c; x) = (a) (b) (c) F (a+, b+, c+; x) F (a, b, c; x) = ( x) c a b F (c a, c b, c; x) Several commo mathematical fuctio may be expressed i terms of the hypergeometric fuctio such as, the icomplete Beta fuctio B x (a, b), the complete elliptical itegrals 4 The otatio F idicates the presece of two Pochhammer symbols i the umerator ad oe i the deomiator. 67

K ad E, the Gegebauer fuctios T β (x), the Legedre fuctios P (x), P m (x) ad Q ν (x) (secod kid), ad the Chebyshev fuctios T (x), U (x) ad V (x) ( z) a = F (a, b, b; z) l( + z) = x F (,, ; z) arcta z = z F (,, 3 ; z) arcsi z = z F (,, 3 ; z) = z z F (,, 3 ; z) B x (a, b) = xa a F (a, b, a+; x) K = E = T β (x) = π π ( k si θ) π ( dθ = F,, ; k) ( k si θ) π ( dθ = F,, ; k) ( + β)! β!β! P (x) = F (, +, ; x P m (x) = ( + m)! ( x ) m ( m)! m m! ( )!! P (x) = ( ) ()!! ( + )!! P + (x) = ( ) ()!! Q ν (x) = ( ) F, +β+, +β; x ) ( ) F m, m++, m+; x F (, +, ; x) πν! ( ( ) ν +!(x) ν+ F ν+, ν ) ( T (x) = F,, ; x x F (, + 3, 3 ; x) ( U (x) = ( + ) F, +, 3 ; x +, ν+3 ; x ) V (x) = ( x F +, +, 3 ; x for Q ν (x) the coditios are x >, arg x < π, may more similar ad additioal formulæ. 43.3 Cofluet Hypergeometric Fuctio ) ) ad ν,, 3,.... See [46] for The cofluet hypergeometric equatio, or Kummer s equatio as it is ofte called, is give by [4,47] x f(x) + (c x) f(x) x x af(x) = Oe solutio to this equatio is f(x) = F (a, c; x) = M(a, c; x) = 68 = (a) x (c)! c,,,...

This solutio is coverget for all fiite real x (or complex z). Aother solutio is give by f(x) = x c M(a+ c, c; x) c, 3, 4,... Ofte a liear combiatio of the first ad secod solutio is used U(a, c; x) = π [ ] M(a, c; x) si πc (a c)!(c )! x c M(a+ c, c; x) (a )!( c)! The cofluet hypergeometric fuctios M ad U may be expressed i itegral form as M(a, c; x) = U(a, c; x) = Γ(c) e xt t a ( t) c a dt Re c >, Re a > Γ(a)Γ(c a) e xt t a ( + t) c a dt Re x >, Re a > Γ(a) Useful formulæ are the Kummer trasformatios M(a, c; x) = e x M(c a, c; x) U(a, c; x) = x c U(a c+, c; x) The :th derivatives of the cofluet hypergeometric fuctios are give by d dz M(a, b; z) = (a) M(a+, b+; z) (b) d dz U(a, b; z) = ( ) (a) U(a+, b+; z) Several commo mathematical fuctio may be expressed i terms of the hypergeometric fuctio such as the error fuctio, the icomplete Gamma fuctio γ(a, x), Bessel fuctios J ν (x), modified Bessel fuctios of the first kid I ν (x), Hermite fuctios H (x), Laguerre fuctios L (x), associated Laguerre fuctios L m (x), Whittaker fuctios M kµ (x) ad W kµ (x), Fresel itegrals C(x) ad S(x), modified Bessel fuctio of the secod kid K ν (x) e z = M(a, a; z) erf(x) = π xm (, 3 ; x) = π xe x M (, 3 ; x) γ(a, x) = xa a J ν (x) = e ıx ν! I ν (x) = e x ν! H (x) = ( ) ()! M(a, a+; x) Re a > ( ) x ν ( M ν+, ν+; ıx) ) ν ( M ν+, ν+; x) ( x M (,! ; x) ( + )! H + (x) = ( ) xm (, 3! ; x) 69

L (x) = M(, ; x) L m (x) = ( ) m m x m L +m(x) = M kµ (x) = e x x µ+ M ( µ k+, µ+; x) W kµ (x) = e x x µ+ U ( µ k+, µ+; x) ( C(x) + ıs(x) = xm x, 3 ) ; ıπx K ν (x) = πe x (x) ν U ( ν+, ν+; x) See [47] for may more similar ad additioal formulæ. ( + m)! M(, m+; x)!m! 7

Table : Percetage poits of the chi-square distributio α.5.8.9.95.975.99.995.999.4549.644.755 3.845 5.39 6.6349 7.8794.88.3863 3.89 4.65 5.995 7.3778 9.3.597 3.86 3.366 4.646 6.54 7.847 9.3484.345.838 6.66 4 3.3567 5.9886 7.7794 9.4877.43 3.77 4.86 8.467 5 4.355 7.893 9.364.7.833 5.86 6.75.55 6 5.348 8.558.645.59 4.449 6.8 8.548.458 7 6.3458 9.83.7 4.67 6.3 8.475.78 4.3 8 7.344.3 3.36 5.57 7.535.9.955 6.4 9 8.348.4 4.684 6.99 9.3.666 3.589 7.877 9.348 3.44 5.987 8.37.483 3.9 5.88 9.588.34 4.63 7.75 9.675.9 4.75 6.757 3.64.34 5.8 8.549.6 3.337 6.7 8.3 3.99 3.34 6.985 9.8.36 4.736 7.688 9.89 34.58 4 3.339 8.5.64 3.685 6.9 9.4 3.39 36.3 5 4.339 9.3.37 4.996 7.488 3.578 3.8 37.697 6 5.338.465 3.54 6.96 8.845 3. 34.67 39.5 7 6.338.65 4.769 7.587 3.9 33.49 35.78 4.79 8 7.338.76 5.989 8.869 3.56 34.85 37.56 4.3 9 8.338 3.9 7.4 3.44 3.85 36.9 38.58 43.8 9.337 5.38 8.4 3.4 34.7 37.566 39.997 45.35.337 6.7 9.65 3.67 35.479 38.93 4.4 46.797.337 7.3 3.83 33.94 36.78 4.89 4.796 48.68 3.337 8.49 3.7 35.7 38.76 4.638 44.8 49.78 4 3.337 9.553 33.96 36.45 39.364 4.98 45.559 5.79 5 4.337 3.675 34.38 37.65 4.646 44.34 46.98 5.6 6 5.336 3.795 35.563 38.885 4.93 45.64 48.9 54.5 7 6.336 3.9 36.74 4.3 43.95 46.963 49.645 55.476 8 7.336 34.7 37.96 4.337 44.46 48.78 5.993 56.89 9 8.336 35.39 39.87 4.557 45.7 49.588 5.336 58.3 3 9.336 36.5 4.56 43.773 46.979 5.89 53.67 59.73 4 39.335 47.69 5.85 55.758 59.34 63.69 66.766 73.4 5 49.335 58.64 63.67 67.55 7.4 76.54 79.49 86.66 6 59.335 68.97 74.397 79.8 83.98 88.379 9.95 99.67 7 69.334 79.75 85.57 9.53 95.3.43 4..3 8 79.334 9.45 96.578.88 6.63.33 6.3 4.84 9 89.334.5 7.57 3.5 8.4 4. 8.3 37. 99.334.67 8.5 4.34 9.56 35.8 4.7 49.45 7

Table : Extreme cofidece levels for the chi-square distributio Chi-square Cofidece Levels (as χ values) d.f... 3 4 5 6 7 8 9.7 6.63.8 5. 9.5 3.9 8.4 3.8 37.3 4.8 46.3 5.8 4.6 9. 3.8 8.4 3. 7.6 3. 36.8 4.4 46. 5.7 55.3 3 6.5.3 6.3. 5.9 3.7 35.4 4. 44.8 49.5 54. 58.9 4 7.78 3.3 8.5 3.5 8.5 33.4 38. 43. 47.9 5.7 57.4 6. 5 9.4 5..5 5.7 3.9 35.9 4.9 45.8 5.7 55.6 6.4 65. 6.6 6.8.5 7.9 33. 38.3 43.3 48.4 53.3 58.3 63. 68. 7. 8.5 4.3 9.9 35.3 4.5 45.7 5.8 55.9 6.9 65.9 7.8 8 3.4. 6. 3.8 37.3 4.7 48. 53. 58.3 63.4 68.4 73.5 9 4.7.7 7.9 33.7 39.3 44.8 5. 55.4 6.7 65.8 7.9 76. 6. 3. 9.6 35.6 4.3 46.9 5.3 57.7 6.9 68. 73.3 78.5 7.3 4.7 3.3 37.4 43. 48.9 54.4 59.8 65. 7.5 75.7 8.9 8.5 6. 3.9 39. 45. 5.8 56.4 6.9 67.3 7.7 78. 83. 3 9.8 7.7 34.5 4.9 46.9 5.7 58.4 64. 69.5 74.9 8. 85.5 4. 9. 36. 4.6 48.7 54.6 6.4 66. 7.6 77. 8.4 87.8 5.3 3.6 37.7 44.3 5.5 56.5 6.3 68. 73.6 79. 84.6 9. 6 3.5 3. 39.3 45.9 5. 58.3 64. 7. 75.7 8. 86.7 9. 7 4.8 33.4 4.8 47.6 54. 6. 66. 7.9 77.6 83.3 88.8 94.3 8 6. 34.8 4.3 49. 55.7 6.9 68. 73.8 79.6 85.3 9.9 96.4 9 7. 36. 43.8 5.8 57.4 63.7 69.8 75.7 8.6 87.3 9.9 98.5 8.4 37.6 45.3 5.4 59. 65.4 7.6 77.6 83.5 89.3 94.9 5 34.4 44.3 5.6 6. 67. 73.9 8.4 86.6 9.8 98.8 5 3 4.3 5.9 59.7 67.6 75. 8. 88.8 95.3 8 4 35 46. 57.3 66.6 74.9 8.6 89.9 97. 4 7 3 9 4 5.8 63.7 73.4 8. 9. 97.7 5 9 5 3 38 45 57.5 7. 8. 89. 97.4 5 3 7 34 4 47 5 63. 76. 86.7 96. 5 3 8 35 4 49 55 6 74.4 88.4 99.6 9 7 35 43 5 58 65 7 7 85.5 3 3 4 5 58 66 73 8 88 8 96.6 5 36 46 55 64 7 8 88 96 4 9 8 4 37 49 59 69 78 87 95 3 9 8 36 49 6 7 8 9 9 8 6 34 4 59 74 86 98 9 9 8 37 46 55 63 5 73 93 9 3 36 47 58 68 78 88 97 36 6 49 68 83 97 3 3 333 344 355 365 374 3 33 36 38 4 46 43 445 458 47 483 495 56 4 437 469 493 54 53 549 565 58 594 67 6 63 5 54 576 63 66 646 665 68 698 74 78 74 756 6 645 684 73 737 759 779 798 85 83 847 86 877 8 85 896 99 957 98 5 6 45 64 8 98 4 58 7 44 75 7 5 7 9 3 33 348 7

Table 3: Extreme cofidece levels for the chi-square distributio (as χ /d.f. values) Chi-square Cofidece Levels (as χ /d.f. values) d.f... 3 4 5 6 7 8 9.7 6.63.83 5.4 9.5 3.93 8.37 3.84 37.3 4.8 46.33 5.84.3 4.6 6.9 9..5 3.8 6. 8.4.7 3.3 5.33 7.63 3.8 3.78 5.4 7.4 8.63..8 3.38 4.95 6.5 8.8 9.64 4.94 3.3 4.6 5.88 7. 8.34 9.56.77.97 3.7 4.36 5.55 5.85 3. 4. 5.5 6.7 7.8 8.7 9.6.4..8 3.5 6.77.8 3.74 4.64 5.5 6.38 7. 8.6 8.89 9.7.54.35 7.7.64 3.47 4.7 5.4 5.79 6.53 7.6 7.98 8.7 9.4. 8.67.5 3.7 3.98 4.67 5.34 6. 6.65 7.9 7.9 8.56 9.8 9.63.4 3. 3.75 4.37 4.98 5.57 6.6 6.74 7.3 7.88 8.45.6.3.96 3.56 4.3 4.69 5.3 5.77 6.9 6.8 7.33 7.85.57.5.84 3.4 3.93 4.44 4.94 5.44 5.9 6.4 6.88 7.35.55.8.74 3.6 3.76 4.4 4.7 5.6 5.6 6.6 6.5 6.93 3.5.3.66 3.4 3.6 4.6 4.49 4.9 5.34 5.76 6.7 6.58 4.5.8.58 3.4 3.48 3.9 4.3 4.7 5. 5.5 5.89 6.7 5.49.4.5.95 3.37 3.77 4.6 4.54 4.9 5.8 5.64 6. 6.47..45.87 3.7 3.65 4. 4.37 4.73 5.8 5.4 5.76 7.46.97.4.8 3.7 3.54 3.89 4.3 4.57 4.9 5. 5.55 8.44.93.35.73 3.9 3.44 3.78 4. 4.4 4.74 5.5 5.36 9.43.9.3.67 3. 3.35 3.67 3.99 4.9 4.59 4.89 5.8.4.88.7.6.95 3.7 3.58 3.88 4.7 4.46 4.75 5.3 5.38.77..4.69.96 3. 3.47 3.7 3.95 4.9 4.4 3.34.7.99.5.5.73.96 3.8 3.39 3.6 3.8 4. 35.3.64.9.4.36.57.77.96 3.5 3.34 3.5 3.69 4.3.59.84.5.5.44.6.8.97 3.3 3.9 3.45 45.8.55.78.98.6.34.5.66.8.97 3. 3.6 5.6.5.73.9.9.5.4.55.7.84.97 3. 6.4.47.66.83.98..5.38.5.63.75.86 7..43.6.75.89..4.5.37.48.58.68 8..4.56.7.8.94.5.5.6.35.45.54 9..38.5.65.77.87.98.7.7.6.35.43.8.36.49.6.7.8.9..9.8.6.34.7.3.45.55.65.74.8.9.98.5..9 5.5.9.4.49.57.65.7.79.85.9.98.4.3.5.34.4.48.55.6.67.7.77.8.87 3...7.33.39.44.48.53.57.6.65.69 4.9.7.3.8.33.37.4.45.48.5.55.58 5.8.5..5.9.33.36.4.43.46.48.5 6.7.4.9.3.7.3.33.36.39.4.44.46 8.6..6..3.6.8.3.33.35.37.39.6..4.7..3.5.7.9.3.33.35 73

Table 4: Exact ad approximate values for the Beroulli umbers Beroulli umbers N/D = B / k k / =. / = 5. /6 =.66666 66667 4 /3 = 3.33333 33333 6 /4 =.3895 38 8 /3 = 3.33333 33333 5/66 = 7.57575 75758 69/73 =.533 553 4 7/6 =.6666 66667 6 3 67/5 = 7.95 6867 8 43 867/798 = 5.497 77945 74 6/33 = 5.94 44 854 53/38 = 6.9 3884 3 4 36 364 9/ 73 = 8.658 534 4 6 8 553 3/6 =.455 7667 6 8 3 749 46 9/87 =.798 368 7 3 8 65 84 76 5/4 3 = 6.58 8739 8 3 7 79 3 4 7/5 =.563 5767 34 577 687 858 367/6 = 4.964 6436 36 6 35 7 553 53 477 373/ 99 9 =.376 555 3 38 99 993 93 84 559/6 = 4.8833 3897 4 4 6 8 78 496 449 5/3 53 =.9965 7934 6 4 5 97 643 98 7 8 69/ 86 = 8.4693 4757 7 44 7 833 69 579 3 4 35 3/69 = 4.338 7854 9 46 596 45 593 9 63 77 96/8 =.57 48638 48 5 69 43 368 997 87 686 49 7 547/46 4 =.866 65 3 5 495 57 5 4 79 648 477 55/66 = 7.586 6746 4 5 8 65 78 35 489 957 347 94 99 853/ 59 = 5.3877 85 6 54 9 49 963 634 884 86 4 48 3 8 69/798 = 3.6587 76485 8 56 479 39 99 33 6 753 685 45 739 663 9/87 =.84987 693 3 58 84 483 63 348 88 4 86 46 775 994 36 /354 =.38654 75 3 6 5 33 4 483 755 57 4 34 994 79 8 46 4 49/56 786 73 =.3999 4957 34 6 3 585 434 86 858 54 953 39 857 43 386 5/6 =.59 7573 36 64 6 783 83 47 866 59 886 385 444 979 4 647 94 7/5 =.938 59 38 66 47 6 6 335 654 5 69 48 55 93 34 4 899 /64 7 =.756 96488 4 68 78 773 3 858 78 78 4 99 49 8 474 66 44 347 /3 =.6577 86 4 7 55 38 347 333 367 3 83 76 567 377 857 8 5 438 6 35/4 686 = 3.5 83 44 74

Table 5: Percetage poits of the F -distributio α=. m 3 4 5 5 39.86 49.5 53.59 55.83 57.4 6.9 6.74 6.69 63. 63.33 8.56 9. 9.6 9.43 9.93 9.39 9.44 9.47 9.48 9.49 3 5.538 5.46 5.39 5.343 5.39 5.3 5.84 5.55 5.44 5.34 4 4.545 4.35 4.9 4.7 4.5 3.9 3.844 3.795 3.778 3.76 5 4.6 3.78 3.69 3.5 3.453 3.97 3.7 3.47 3.6 3.5 3.85.94.78.65.5.33..7.87.55.975.589.38.49.58.937.794.69.65.67 5.89.4.97.6.966.79.568.44.388.37.756.356.39..96.663.494.355.93.4.76.33.84.945.847.599.4.63.85. α=.5 m 3 4 5 5 6.4 99.5 5.7 4.6 3. 4.9 48. 5.8 53. 54.3 8.5 9. 9.6 9.5 9.3 9.4 9.45 9.48 9.49 9.5 3.3 9.55 9.77 9.7 9.3 8.786 8.66 8.58 8.554 8.56 4 7.79 6.944 6.59 6.388 6.56 5.964 5.83 5.699 5.664 5.68 5 6.68 5.786 5.49 5.9 5.5 4.735 4.558 4.444 4.45 4.365 4.965 4.3 3.78 3.478 3.36.978.774.637.588.538 4.35 3.493 3.98.866.7.348.4.966.97.843 5 4.34 3.83.79.557.4.6.784.599.55.438 3.936 3.87.696.463.35.97.676.477.39.83 3.84.996.65.37.4.83.57.35.43. α=. m 3 4 5 5 45 5 543 565 5764 656 69 633 6334 6366 98.5 99. 99.7 99.5 99.3 99.4 99.45 99.48 99.49 99.5 3 34. 3.8 9.46 8.7 8.4 7.3 6.69 6.35 6.4 6.3 4. 8. 6.69 5.98 5.5 4.55 4. 3.69 3.58 3.46 5 6.6 3.7.6.39.97.5 9.553 9.38 9.3 9..4 7.559 6.55 5.994 5.636 4.849 4.45 4.5 4.4 3.99 8.96 5.849 4.938 4.43 4.3 3.368.938.643.535.4 5 7.7 5.57 4.99 3.7 3.48.698.65.949.85.683 6.895 4.84 3.984 3.53 3.6.53.67.735.598.47 6.635 4.65 3.78 3.39 3.7.3.878.53.358. 75

Table 6: Probability cotet from z to z of Gauss distributio i % z....3.4.5.6.7.8.9...8.6.39 3.9 3.99 4.78 5.58 6.38 7.7. 7.97 8.76 9.55.34.3.9.7 3.5 4.8 5.7. 5.85 6.63 7.4 8.9 8.97 9.74.5.8.5.8.3 3.58 4.34 5. 5.86 6.6 7.37 8. 8.86 9.6 3.35.4 3.8 3.8 3.55 33.8 34. 34.73 35.45 36.6 36.88 37.59.5 38.9 38.99 39.69 4.39 4.8 4.77 4.45 43.3 43.8 44.48.6 45.5 45.8 46.47 47.3 47.78 48.43 49.7 49.7 5.35 5.98.7 5.6 5.3 5.85 53.46 54.7 54.67 55.7 55.87 56.46 57.5.8 57.63 58. 58.78 59.35 59.9 6.47 6. 6.57 6. 6.65.9 63.9 63.7 64.4 64.76 65.8 65.79 66.9 66.8 67.9 67.78. 68.7 68.75 69.3 69.7 7.7 7.63 7.9 7.54 7.99 7.43. 7.87 73.3 73.73 74.5 74.57 74.99 75.4 75.8 76. 76.6. 76.99 77.37 77.75 78.3 78.5 78.87 79.3 79.59 79.95 8.9.3 8.64 8.98 8.3 8.65 8.98 8.3 8.6 8.93 83.4 83.55.4 83.85 84.5 84.44 84.73 85. 85.9 85.57 85.84 86. 86.38.5 86.64 86.9 87.5 87.4 87.64 87.89 88. 88.36 88.59 88.8.6 89.4 89.6 89.48 89.69 89.9 9. 9.3 9.5 9.7 9.9.7 9.9 9.7 9.46 9.64 9.8 9.99 9.6 9.33 9.49 9.65.8 9.8 9.97 93. 93.7 93.4 93.57 93.7 93.85 93.99 94..9 94.6 94.39 94.5 94.64 94.76 94.88 95. 95. 95.3 95.34. 95.45 95.56 95.66 95.76 95.86 95.96 96.6 96.5 96.5 96.34. 96.43 96.5 96.6 96.68 96.76 96.84 96.9 97. 97.7 97.5. 97. 97.9 97.36 97.43 97.49 97.56 97.6 97.68 97.74 97.8.3 97.86 97.9 97.97 98. 98.7 98. 98.7 98. 98.7 98.3.4 98.36 98.4 98.45 98.49 98.53 98.57 98.6 98.65 98.69 98.7.5 98.76 98.79 98.83 98.86 98.89 98.9 98.95 98.98 99. 99.4.6 99.7 99.9 99. 99.5 99.7 99. 99. 99.4 99.6 99.9.7 99.3 99.33 99.35 99.37 99.39 99.4 99.4 99.44 99.46 99.47.8 99.49 99.5 99.5 99.53 99.55 99.56 99.58 99.59 99.6 99.6.9 99.63 99.64 99.65 99.66 99.67 99.68 99.69 99.7 99.7 99.7 3. 99.73 99.74 99.75 99.76 99.76 99.77 99.78 99.79 99.79 99.8 3. 99.8 99.8 99.8 99.83 99.83 99.84 99.84 99.85 99.85 99.86 3. 99.86 99.87 99.87 99.88 99.88 99.88 99.89 99.89 99.9 99.9 3.3 99.9 99.9 99.9 99.9 99.9 99.9 99.9 99.9 99.93 99.93 3.4 99.93 99.94 99.94 99.94 99.94 99.94 99.95 99.95 99.95 99.95 3.5 99.95 99.96 99.96 99.96 99.96 99.96 99.96 99.96 99.97 99.97 3.6 99.97 99.97 99.97 99.97 99.97 99.97 99.97 99.98 99.98 99.98 3.7 99.98 99.98 99.98 99.98 99.98 99.98 99.98 99.98 99.98 99.98 3.8 99.99 99.99 99.99 99.99 99.99 99.99 99.99 99.99 99.99 99.99 3.9 99.99 99.99 99.99 99.99 99.99 99.99 99.99 99.99 99.99 99.99 76

Table 7: Stadard ormal distributio z-values for a specific probability cotet from z to z. Read colum-wise ad add margial colum ad row z. Read colum-wise ad add margial colum ad row figures to fid probabilities. Prob.....3.4.5.6.7.8.9...5.53.385.54.674.84.36.8.645...8.56.388.57.677.845.4.87.655.4.5.3.58.39.53.68.849.45.93.665.6.7.33.6.393.533.684.85.49.99.675.8..35.63.396.536.687.856.54.35.685...38.66.398.538.69.859.58.3.696..5.4.68.4.54.693.863.63.37.76.4.7.43.7.44.544.696.867.67.33.77.6..46.74.47.547.7.87.7.39.78.8..48.76.49.55.73.874.76.335.74..5.5.79.4.553.76.878.8.34.75..7.53.8.45.556.79.88.85.347.763.4.3.56.84.47.559.7.885.89.353.775.6.33.58.87.4.56.76.889.94.36.787.8.35.6.89.43.565.79.893.99.366.8.3.38.63.9.46.568.7.896.3.37.8.3.4.66.95.48.57.75.9.8.379.85.34.43.68.97.43.574.79.94..385.839.36.45.7.3.434.577.73.98.7.39.853.38.48.73.3.437.58.735.9..399.867.4.5.76.35.439.58.739.95.6.45.88.4.53.79.38.44.585.74.99.3.4.896.44.55.8.3.445.588.745.93.36.49.9.46.58.84.33.448.59.749.97.4.46.97.48.6.86.36.45.594.75.93.46.433.944.5.63.89.38.453.597.755.935.5.44.96.5.65.9.3.456.6.759.938.55.447.978.54.68.94.33.459.63.76.94.6.454.996.56.7.96.36.46.66.765.946.65.46.5.58.73.99.39.464.69.769.95.7.469.34.6.75..33.467.6.77.954.75.476.54.6.78.4.334.47.65.775.958.8.484.75.64.8.7.337.473.69.779.96.85.49.97.66.83.9.339.476.6.78.966.9.499..68.85..34.478.65.786.97.95.57.45.7.88.4.345.48.68.789.974..54.7.7.9.7.347.484.63.79.978.6.5.98.74.93.9.35.487.634.796.98..53.7.76.95..353.49.637.799.986.6.539.58.78.98.5.355.493.64.83.99..547.9.8..7.358.495.643.86.994.7.555.37.8.3.3.36.498.646.8.999.3.564.366.84.5.3.363.5.649.83.3.37.57.49.86.8.35.366.54.65.87.7.43.58.458.88..37.369.57.655.8..48.59.53.9.3.4.37.5.659.84.5.54.599.576.9.5.43.374.53.66.87.9.59.68.65.94.8.45.377.55.665.83.4.65.67.748.96..48.379.58.668.834.8.7.66.879.98.3.5.38.5.67.838.3.76.636 3.9 77

Table 8: Percetage poits of the t-distributio α.6.7.8.9.95.975.99.995.999.9995.35.77.376 3.78 6.34.7 3.8 63.66 38.3 636.6.89.67.6.886.9 4.33 6.965 9.95.33 3.6 3.77.584.978.638.353 3.8 4.54 5.84..9 4.7.569.94.533.3.776 3.747 4.64 7.73 8.6 5.67.559.9.476.5.57 3.365 4.3 5.893 6.869 6.65.553.96.44.943.447 3.43 3.77 5.8 5.959 7.63.549.896.45.895.365.998 3.499 4.785 5.48 8.6.546.889.397.86.36.896 3.355 4.5 5.4 9.6.543.883.383.833.6.8 3.5 4.97 4.78.6.54.879.37.8.8.764 3.69 4.44 4.587.6.54.876.363.796..78 3.6 4.5 4.437.59.539.873.356.78.79.68 3.55 3.93 4.38 3.59.538.87.35.77.6.65 3. 3.85 4. 4.58.537.868.345.76.45.64.977 3.787 4.4 5.58.536.866.34.753.3.6.947 3.733 4.73 6.58.535.865.337.746..583.9 3.686 4.5 7.57.534.863.333.74..567.898 3.646 3.965 8.57.534.86.33.734..55.878 3.6 3.9 9.57.533.86.38.79.93.539.86 3.579 3.883.57.533.86.35.75.86.58.845 3.55 3.85.57.53.859.33.7.8.58.83 3.57 3.89.56.53.858.3.77.74.58.89 3.55 3.79 3.56.53.858.39.74.69.5.87 3.485 3.768 4.56.53.857.38.7.64.49.797 3.467 3.745 5.56.53.856.36.78.6.485.787 3.45 3.75 6.56.53.856.35.76.56.479.779 3.435 3.77 7.56.53.855.34.73.5.473.77 3.4 3.69 8.56.53.855.33.7.48.467.763 3.48 3.674 9.56.53.854.3.699.45.46.756 3.396 3.659 3.56.53.854.3.697.4.457.75 3.385 3.646 4.55.59.85.33.684..43.74 3.37 3.55 5.55.58.849.99.676.9.43.678 3.6 3.496 6.54.57.848.96.67..39.66 3.3 3.46 7.54.57.847.94.667.994.38.648 3. 3.435 8.54.56.846.9.664.99.374.639 3.95 3.46 9.54.56.846.9.66.987.368.63 3.83 3.4.54.56.845.9.66.984.364.66 3.74 3.39.54.56.845.89.659.98.36.6 3.66 3.38.54.56.845.89.658.98.358.67 3.6 3.373.53.54.84.8.645.96.36.576 3.9 3.9 78

Table 9: Expressios for the Beta fuctio B(m, ) for iteger ad half-iteger argumets m π 3 5 7 9 3 4 5 3 π 3 8 π 4 3 4 5 6 5 3 8 π 5 6 π 4 35 3 8 π 3 6 5 3 6 5 6 35 3 7 5 6 π 7 5 8 π 4 63 3 56 π 6 693 5 4 π 4 3 35 4 3 35 3 55 6 3 33 4 9 35 8 π 9 7 56 π 4 99 7 4 π 6 87 5 48 π 3 6435 35 3768 π 5 56 35 5 56 3465 3 56 55 5 56 4545 8 56 9395 63 63 56 π 4 π 4 43 9 48 π 6 45 45 3768 π 3 55 35 65536 π 56 3945 6 5 693 6 5 99 4 5 4545 68 5 5353 54 5 457 6 3 3 4 π 3 33 48 π 4 95 99 3768 π 6 335 55 65536 π 3 995 77 644 π 56 44895 7 48 33 7 48 4545 56 48 5555 5 48 969969 84 48 9997 3 5 49 48 π 5 49 3768 π 4 55 43 65536 6 4845 43 644 π 3 3395 9 5488 π 56 7845 m 3 5 6 7 63 644 π 6 5 969969 77 3 63 5488 π 5 87 3 49434 π 7 48 743649 5544 48 69975 5 73 49434 π 5 395 3 838868 π 48 355 49 3355443 π 79

Mathematical Costats Itroductio It is hady to have available the values of differet mathematical costats appearig i may expressios i statistical calculatios. I this sectio we list, with high precisio, may of those which may be eeded. I some cases we give, after the tables, basic expressios which may be ice to recall. Note, however, that this is ot full explaatios ad cosult the mai text or other sources for details. Some Basic Costats exact approx. π 3.459 6535 89793 3846 e.788 884 5945 3536 γ.577 56649 53 866 π.7745 3859 556 73 / π.39894 84 43 67794 e = = γ = lim (! = lim + ) ( ) k= k l Gamma Fuctio exact approx. Γ( ) π.7745 3859 556 73 Γ( 3 ) π.886 6954 5758 365 Γ( 5 ) 3 4 π.3934 388 7937 47 Γ( 7 ) 5 8 π 3.3335 974 4784 558 Γ( 9 ) 5 6 π.637 83965 67448 994 Γ(z) = t z e t dt! = Γ( + ) = Γ() Γ ( ) + ( )!! = Γ ( ) = = ()! π! See further sectio 4. o page 54 ad referece [4] for more details. Beta Fuctio For exact expressios for the Beta fuctio for halfiteger ad iteger values see table 9 o page 79. B(a, b) = Γ(a)Γ(b) Γ(a + b) = = = x a ( x) b dx = x a dx ( + x) a+b See further sectio 4.6 o page 6. Digamma Fuctio exact approx. ψ( ) γ l.9635 6 43 47944 ψ( 3 ) ψ( ) +.3648 99739 78576 556 ψ( 5 ) ψ( ) + 8 3.735 6646 4543 873 ψ( 7 ) ψ( ) + 46 5.35 6646 4543 873 ψ( 9 ) ψ( ) + 35 5.38887 963 5958 95 ψ( ) ψ( 9 ) + 9.69 3485 875 373 ψ( 3 ) ψ( ) +.799 333 9993 949 ψ( 5 ) ψ( 3 ) + 3.94675 7484 4686 7887 ψ( 7 ) ψ( 5 ) + 5.89 875 94 4 ψ( 9 ) ψ( 7 ) + 7.9773 78764 949 5337 ψ() γ.577 56649 53 866 ψ() γ.478 4335 98467 3939 3 ψ(3) γ.978 4335 98467 3939 ψ(4) 6 γ.56 76684 38 4773 5 ψ(5) γ.56 76684 38 4773 37 ψ(6) 6 γ.76 76684 38 4773 49 ψ(7) γ.8778 4335 98467 3939 363 ψ(8) 4 γ.564 4779 5569 99654 γ.464 4779 5569 99654 γ.575 589 667 765 ψ(9) 76 8 ψ() 79 5 ψ(z) = d dz ψ(z + ) = ψ(z) + z ψ() = γ + dγ(z) l Γ(z) = Γ(z) dz m= m ψ( + ) = γ l + See further sectio 4.3 o page 56. m= m 8

Polygamma Fuctio exact approx. ψ () ( ) π / 4.9348 5 44679 394 ψ () ( ) 4ζ 3 6.8879 6644 3439 9956 ψ (3) ( ) π4 97.499 34 437 3644 ψ (4) ( ) 744ζ 5 77.4744 9866 675 954 ψ (5) ( ) 8π6 769.354 864 35496 476 ψ () () ζ.64493 4668 486 43647 ψ () () ζ 3.44 3863 988 578 ψ (3) () 6ζ 4 6.49393 94 6689 49 ψ (4) () 4ζ 5 4.8866 634 4878 395 ψ (5) () ζ 6.86 7438 33896 76574 ψ () (z) = d d+ ψ(z) = l Γ(z) dz dz+ ψ () (z) = ( ) +! (z + k) + k= ψ () () = ( ) +!ζ + ψ () ( ) = (+ )ψ () () [ ψ (m) ( + ) = ( ) m m! ζ m+ + + ] + m+ +... + m+ = = ψ (m) () + ( ) m m! See further sectio 4.4 o page 58. Beroulli Numbers m+ See also page 59 ad for details referece [3]. Sum of Powers I may calculatios ivolvig discrete distributios sums of powers are eeded. A geeral formula for this is give by k i = k= i ( ) i ( ) j i j+ B j j i j + j= where B j deotes the Beroulli umbers (see page 74). More specifically k = ( + )/ k= k = ( + )( + )/6 k= ( k 3 = ( + ) /4 = k k= k= ) k 4 = ( + )( + )(3 + 3 )/3 k= k 5 = ( + ) ( + )/ k= k 6 = ( + )( + )(3 4 + 6 3 3 + )/4 k= See table 4 o page 74. Riema s Zeta-fuctio exact approx. ζ - ζ ζ π /6.64493 4668 486 43647 ζ 3.5 693 59594 854 ζ 4 π 4 /9.83 3337 38 95 ζ 5.369 7755 43369 9633 ζ 6 π 6 /945.734 369 84449 397 ζ 7.834 9773 89 8684 ζ 8 π 8 /945.47 7356 97944 33938 ζ 9. 8398 68 44 ζ π /93555.99 4575 788 8534 ζ = k= k ζ = π B ()! 8

ERRATA et ADDENDA Errors i this report are corrected as they are foud but for those who have prited a early versio of the had-book we list here errata. These are thus already obsolete i this copy. Mior errors i laguage etc are ot listed. Note, however, that a few additios (subsectios ad tables) have bee made to the origial report (see below). Cotets part ow havig roma page umbers thus shiftig arabic page umbers for the mai text. A ew sectio 6. o coditioal probability desity for biormal distributio has bee added after the first editio Sectio 4.6, formula, lie, ν chaged ito λ givig f(x; µ, λ) = λ e λ x µ Sectio.3, formula, lie 4 has bee corrected φ x (t) = E(e ıtx ) = e ıtµ E(e ıt(x µ) ) = e ıtµ φ x µ (t) = e ıtµ λ Sectio 4.4, formula, lie chaged to φ(t) = E(e ıtx ) = α e (ıt α )x dx = ıtα λ + t Sectio 8., figure 4 was erroeous i early editios ad should look as is ow show i figure 73. Sectio 7., lie : chage νr i to νp i. Sectio 7.6 o sigificace levels for the multiomial distributio has bee added after the first editio. Sectio 7.7 o the case with equal group probabilities for a multiomial distributio has bee added after the first editio. A small paragraph added to sectio 8. itroducig the multiormal distributio. A ew sectio 8. o coditioal probability desity for the multiormal distributio has bee added after the first editio. Sectio 36.4, first formula, lie 5, should read: r µ k e µ P (r) = k! k= = P (r +, µ) 8

Sectio 36.4, secod formula, lie 9, should read: r µ k e µ P (r) = k! k= = µ f(x; ν = r + )dx ad i the ext lie it should read f(x; ν = r + ). Sectio 4.5., formula 3, lie 6, should read: Γ(z) = α z t z e αt dt for Re(z) >, Re(α) > Sectio 4.6, lie 6: a referece to table 9 has bee added (cf below). Table 9 o page 79, o the Beta fuctio B(m, ) for iteger ad half-iteger argumets, has bee added after the first versio of the paper. These were, mostly mior, chages up to the 8th of March 998 i order of apperace. I October 998 the first somewhat larger revisio was made: Some text cocerig the coefficiet of kurtosis added i sectio.. Figure 6 for the chi-square distributio corrected for a ormalizatio error for the = curve. Added figure 8 for the chi distributio o page 44. Added sectio for the doubly o-cetral F -distributio ad sectio for the doubly o-cetral t-distributio. Added figure for the F -distributio o page 6. Added sectio 3 o the o-cetral Beta-distributio o page 8. For the o-cetral chi-square distributio we have added figure ad subsectios 3.4 ad 3.6 for the cumulative distributio ad radom umber geeratio, respectively. For the o-cetral F -distributio figure 3 has bee added o page 3. Errors i the formulæ for f(f ; m,, λ) i the itroductio ad z i the sectio o approximatios have bee corrected. Subsectios 3. o momets, 3.3 for the cumulative distributio, ad 3.5 for radom umber geeratio have bee added. For the o-cetral t-distributio figure 4 has bee added o page 6, some text altered i the first subsectio, ad a error corrected i the deomiator of the approximatio formula i subsectio 33.5. Subsectios 33. o the derivatio of the distributio, 33.3 o its momets, 33.4 o the cumulative distributio, ad 33.6 o radom umber geeratio have bee added. 83

A ew subsectio 34.8.9 has bee added o yet aother method, usig a ratio betwee two uiform deviates, to achieve stadard ormal radom umbers. With this chage three ew refereces [38,39,4] were itroduced. A compariso of the efficiecy for differet algorithms to obtai stadard ormal radom umbers have bee itroduced as subsectio 34.8.. Added a commet o factorial momets ad cumulats for a Poisso distributio i sectio 36.. This list of Errata et Addeda for past versios of the had-book has bee added o page 8 ad owards. Table o page 7 ad table 3 o page 73 for extreme sigificace levels of the chi-square distributio have bee added thus shiftig the umbers of several other tables. This also slightly affected the text i sectio 8.. The Beroulli umbers used i sectio 5.4 ow follow the same covetio used e.g. i sectio 4.3. This chage also affected the formula for κ i sectio 3.4. Table 4 o page 74 o Beroulli umbers was itroduced at the same time shiftig the umbers of several other tables. A list of some mathematical costats which are useful i statistical calculatios have bee itroduced o page 8. Mior chages afterwards iclude: Added a proof for the formula for algebraic momets of the log-ormal distributio i sectio 4. ad added a sectio for the cumulative distributio as sectio 4.3. Added formula also for c < for F (x) of a Geeralized Gamma distributio i sectio 8.. Corrected bug i first formula i sectio 6.6. Replaced table for multiormal cofidece levels o page with a more precise oe based o a aalytical formula. New sectio o sums of powers o page 8. The illustratio for the log-ormal distributio i Figure 6 i sectio 4 was wrog ad has bee replaced. 84

Refereces [] The Advaced Theory of Statistics by M. G. Kedall ad A. Stuart, Vol., Charles Griffi & Compay Limited, Lodo 958. [] The Advaced Theory of Statistics by M. G. Kedall ad A. Stuart, Vol., Charles Griffi & Compay Limited, Lodo 96. [3] A Itroductio to Mathematical Statistics ad Its Applicatios by Richard J. Larse ad Morris L. Marx, Pretice-Hall Iteratioal, Ic. (986). [4] Statistical Methods i Experimetal Physics by W. T. Eadie, D. Drijard, F. E. James, M. Roos ad B. Sadoulet, North-Hollad Publishig Compay, Amsterdam-Lodo (97). [5] Probability ad Statistics i Particle Physics by A. G. Frodese, O. Skjeggestad ad H. Tøfte, Uiversitetsforlaget, Berge-Oslo-Tromsø (979). [6] Statistics for Nuclear ad Particle Physics by Louis Lyos, Cambridge Uiversity Press (986). [7] Statistics A Guide to the Use of Statistical Methods i the Physical Scieces by Roger J. Barlow, Joh Wiley & Sos Ltd., 989. [8] Statistical Data Aalysis by Gle Cowa, Oxford Uiversity Press, 998. [9] Numerical Recipes (The Art of Scietific Computig) by William H. Press, Bria P. Flaery, Saul A. Teukolsky ad William T. Vetterlig, Cambridge Uiversity Press, 986. [] Numerical Recipes i Fortra (The Art of Scietific Computig), secod editio by William H. Press, Saul A. Teukolsky, William T. Vetterlig, ad Bria P. Flaery, Cambridge Uiversity Press, 99. [] Hadbook of Mathematical Fuctios with Formulas, Graphs, ad Mathematical Tables, edited by Milto Abramowitz ad Iree A. Stegu, Dover Publicatios, Ic., New York, 965. [] Statistical Distributios by N. A. J. Hastigs ad J. B. Peacock, Butterworth & Co (Publishers) Ltd, 975. [3] A Mote Carlo Sampler by C. J. Everett ad E. D. Cashwell, LA-56-MS Iformal Report, October 97, Los Alamos Scietific Laboratory of the Uiversity of Califoria, New Mexico. [4] Radom Number Geeratio by Christia Walck, USIP Report 87-5, Stockholm Uiversity, December 987. [5] A Review of Pseudoradom Number Geerators by F. James, Computer Physics Commuicatios 6 (99) 39 344. [6] Toward a Uiversal Radom Number Geerator by George Marsaglia, Arif Zama ad Wai Wa Tsag, Statistics & Probability Letters 9 (99) 35 39. [7] Implemetatio of a New Uiform Radom Number Geerator (icludig bechmark tests) by Christia Walck, Iteral Note SUF PFY/89, Particle Physics Group, Fysikum, Stockholm Uiversity, December 989. 85

[8] A Radom Number Geerator for PC s by George Marsaglia, B. Narasimha ad Arif Zama, Computer Physics Commuicatios 6 (99) 345 349. [9] A Portable High-Quality Radom Number Geerator for Lattice Field Theory Simulatios by Marti Lüscher, Computer Physics Commuicatios 79 (994). [] Implemetatio of Yet Aother Uiform Radom Number Geerator by Christia Walck, Iteral Note SUF PFY/94, Particle Physics Group, Fysikum, Stockholm Uiversity, 7 February 994. [] Radom umber geeratio by Birger Jasso, Victor Pettersos Bokidustri AB, Stockholm, 966. [] J. W. Butler, Symp. o Mote Carlo Methods, Whiley, New York (956) 49 64. [3] J. C. Butcher, Comp. J. 3 (96) 5 53. [4] Ars Cojectadi by Jacques Beroulli, published posthumously i 73. [5] G. E. P. Box ad M. E. Muller i Aals of Math. Stat. 9 (958) 6 6. [6] Probability Fuctios by M. Zele ad N. C. Severo i Hadbook of Mathematical Fuctios, ed. M. Abramowitz ad I. A. Stegu, Dover Publicatios, Ic., New York, 965, 95. [7] Error Fuctio ad Fresel Itegrals by Walter Gautschi i Hadbook of Mathematical Fuctios, ed. M. Abramowitz ad I. A. Stegu, Dover Publicatios, Ic., New York, 965, 95. [8] Computer Methods for Samplig From the Expoetial ad Normal Distributios by J. H. Ahres ad U. Dieter, Commuicatios of the ACM 5 (97) 873. [9] J. vo Neuma, Nat. Bureau Stadards, AMS (95) 36. [3] G. Marsaglia, A. Math. Stat. 3 (96) 899. [3] Beroulli ad Euler Polyomials Riema Zeta Fuctio by Emilie V. Haysworth ad Karl Goldberg i Hadbook of Mathematical Fuctios, ed. M. Abramowitz ad I. A. Stegu, Dover Publicatios, Ic., New York, 965, 83. [3] Irratioalité de ζ() et ζ(3) by R. Apéry, Astérisque 6 (979) 3. [33] Multiplicity Distributios i Strog Iteractios: A Geeralized Negative Biomial Model by S. Hegyi, Phys. Lett. B387 (996) 64. [34] Probability, Radom Variables ad Stochastic Processes by Athaasios Papoulis, McGraw- Hill book compay (965). [35] Theory of Ioizatio Fluctuatio by J. E. Moyal, Phil. Mag. 46 (955) 63. [36] A New Empirical Regularity for Multiplicity Distributios i Place of KNO Scalig by the UA5 Collaboratio: G. J. Aler et al., Phys. Lett. B6 (985) 99. [37] G. Marsaglia, M. D. MacLare ad T. A. Bray. Comm. ACM 7 (964) 4. [38] Computer Geeratio of Radom Variables Usig the Ratio of Uiform Deviates by A. J. Kiderma ad Joh F. Moaha, ACM Trasactios o Mathematical Software 3 (977) 57 6. 86

[39] A Fast Normal Radom Number Geerator by Joseph L. Leva, ACM Trasactios o Mathematical Software 8 (99) 449 453. [4] Algorithm 7: A Normal Radom Number Geerator by Joseph L. Leva, ACM Trasactios o Mathematical Software 8 (99) 454 455. [4] Mathematical Methods for Physicists by George Arfke, Academic Press, 97. [4] Gamma Fuctio ad Related Fuctios by Philip J. Davis i Hadbook of Mathematical Fuctios, ed. M. Abramowitz ad I. A. Stegu, Dover Publicatios, Ic., New York, 965, 53. [43] Tables of Itegrals, Series, ad Products by I. S. Gradshtey ad I. M. Ryzhik, Fourth Editio, Academic Press, New York ad Lodo, 965. [44] The Special Fuctios ad their Approximatios by Yudell L. Luke, Volume, Academic Press, New York ad Lodo, 969. [45] Programs for Calculatig the Logarithm of the Gamma Fuctio, ad the Digamma Fuctio, for Complex Argumet by K. S. Kölbig, Computer Physics Commuicatios 4 (97) 6. [46] Hypergeometric Fuctios by Fritz Oberhettiger i Hadbook of Mathematical Fuctios, ed. M. Abramowitz ad I. A. Stegu, Dover Publicatios, Ic., New York, 965, 555. [47] Cofluet Hypergeometric Fuctios by Lucy Joa Slater i Hadbook of Mathematical Fuctios, ed. M. Abramowitz ad I. A. Stegu, Dover Publicatios, Ic., New York, 965, 53. 87

Idex A Absolute momets..................... Accept-reject techique...................7 Additio theorem (Normal)............ Additio theorem (Poisso)............ 35 Algebraic momets....................... 3 Aalysis of variace..................... 65 Apéry s umber......................... 6 B Beroulli, Jacques....................... Beroulli distributio................... Beroulli umbers...................59,58 Table.............................. 74 Beta distributio........................ 3 No-cetral........................ 8 Beta fuctio.......................... 6 Table.............................. 79 Biomial distributio................... 6 Biormal distributio................... Bivariate momets....................... 9 Box-Muller trasformatio.............. Brachig process.................... 6,5 Breit, Gregory.......................... 8 Breit-Wiger distributio............... 8 C Cauchy, Augusti Louis................. 6 Cauchy distributio..................... 6 Cetral momets......................... 3 Characteristic fuctio................... 4 Chi distributio......................... 43 Chi-square distributio.................. 36 Extreme cofidece levels...... 7,73 No-cetral........................ Percetage poits.................. 7 Cholesky decompositio................ Compoud Poisso distributio......... 45 Cofluet hypergeometric fuctio..... 68 Costats.............................. 8 Correlatio coefficiet................... Covariace............................... 9 Covariace form.........................99 Cumulat geeratig fuctio............ 6 Cumulats............................... 6 Logistic distributio.................84 Moyal distributio.................. 9 No-cetral chi-square distributio. Poisso distributio................35 Cumulative fuctio......................3 D Digamma fuctio..................... 56 Distributio fuctio..................... 3 Double-Expoetial distributio.........47 Doubly o-cetral F -distributio....... 49 Doubly o-cetral t-distributio........5 E Erlagia distributio................... 69 Error fuctio...........................53 Euler gamma-fuctio...................8 Euler s costat......................... 59 Euler-Mascheroi costat.............. 59 Excess, coefficiet of......................3 Expectatio value........................ 3 Expoetial distributio................ 54 Extreme value distributio.............. 57 F Factorial............................... 54 Factorial momets........................6 Poisso distributio................35 F -distributio...........................6 Doubly o-cetral................. 49 No-cetral........................ 3 Percetage poits.................. 75 F -ratio..................................63 Fisher, Sir Roald Aylmer............39,6 Fisher-Tippett distributio.............. 57 Fisher z-distributio.................... 6 G Gamma distributio.....................69 Gamma fuctio....................... 54 Gauss distributio..................... 9 Gauss s differetial equatio............67 Geeralized Gamma distributio........ 73 Geometric distributio.................. 75 Gosset, William Sealy.................. 4 Guiess brewery, Dubli.............. 4 Gumbel, E. J............................ 57 88

Gumbel distributio.....................57 H Hilferty, M. M........................ 39,7 Hit-miss techique....................... 7 Hyperexpoetial distributio........... 77 Hypergeometric distributio.............79 Hypergeometric fuctio............... 67 I Icomplete Beta fuctio.............. 6 Icomplete Gamma distributio........ 59 Idepedece............................ 5 Idepedece theorem (ormal)........ J Johk s algorithm.................... 7,9 K Kedall, Maurice G....................... Kietic theory.......................... 89 Kietic theory, -dim...................39 Kurtosis, Coefficiet of................... 3 Kummer s equatio.................... 68 Kummer trasformatios...............69 L Laczos formula....................... 55 Laplace, Pierre Simo de................ 47 Laplace distributio..................... 47 Leptokurtic.............................. 3 Logarithmic distributio................ 8 Logistic distributio.....................83 Log-ormal distributio................. 86 Log-Weibull distributio................ 57 M Mathematical costats................ 8 Maxwell, James Clerk................... 88 Maxwell distributio.................... 88 Kietic theory...................... 89 Media...............................7,3 Mesokurtic............................... 3 Mode................................... 7 Moyal distributio...................... 9 Multiomial distributio................ 95 Multiormal distributio................ 99 Multivariate momets.................... 9 N Negative biomial distributio......... No-cetral Beta-distributio.......... 8 No-cetral chi-square distributio..... No-cetral F -distributio............. 3 No-cetral t-distributio.............. 6 Normal distributio.................... 9 Additio theorem.................. Idepedece theorem............. Tables......................... 76,77 P Pareto. Vilfredo........................33 Pareto distributio..................... 33 Pascal, Blaise.......................... Pascal distributio..................... Platykurtic............................... 3 Pochhammer symbol................... 67 Poisso, Siméo Deis................. 34 Poisso distributio.................... 34 Additio theorem.................. 35 Polygamma fuctio................... 58 Probability geeratig fuctio........... 5 Brachig process................. 5 Psi fuctio............................56 Q Quatile................................ 3 Quartile.................................3 R Rayleigh, Lord......................... 38 Rayleigh distributio...................38 Kietic theory, -dim.............. 39 Riema s zeta-fuctio................. 59 S Semi-faculty........................... 56 Semi-iterquartile rage................. 3 Skewess, Coefficiet of.................. 3 Stadard ormal distributio.......... 9 Stirlig s formula...................... 55 Stuart, Ala.............................. Studet s t-distributio................ 4 T t-distributio.......................... 4 89

Doubly o-cetral................. 5 No-cetral........................ 6 Percetage poits.................. 78 t-ratio................................. 44 Trapezoidal distributio................5 Triagular distributio................. 5 U Uiform distributio................... 5 V Variace................................. 3 Variace-ratio distributio.............. 6 W Weibull, Waloddi...................... 5 Weibull distributio.................... 5 Wiger, Eugee......................... 8 Wilso, E. B......................... 39,7 Z Zeta-fuctio, Riema s................ 59 9