MAXIMUM LIKELIHOODESTIMATION OF DISCRETELY SAMPLED DIFFUSIONS: A CLOSEDFORM APPROXIMATION APPROACH. By Yacine AïtSahalia 1


 Simon Harper
 2 years ago
 Views:
Transcription
1 Ecoometrica, Vol. 7, No. 1 (Jauary, 22), MAXIMUM LIKELIHOODESTIMATION OF DISCRETEL SAMPLED DIFFUSIONS: A CLOSEDFORM APPROXIMATION APPROACH By acie AïtSahalia 1 Whe a cotiuoustime diffusio is observed oly at discrete dates, i most cases the trasitio distributio ad hece the likelihood fuctio of the observatios is ot explicitly computable. Usig Hermite polyomials, I costruct a explicit sequece of closedform fuctios ad show that it coverges to the true (but ukow) likelihood fuctio. I documet that the approximatio is very accurate ad prove that maximizig the sequece results i a estimator that coverges to the true maximum likelihood estimator ad shares its asymptotic properties. Mote Carlo evidece reveals that this method outperforms other approximatio schemes i situatios relevat for fiacial models. Keywords: Maximumlikelihood estimatio, cotiuoustime diffusio, discrete samplig, trasitio desity, Hermite expasio. 1 itroductio Cosider a cotiuoustime parametric diffusio (1.1) dx t = X t dt + X t dw t where X t is the state variable, W t a stadard Browia motio, ad are kow fuctios, ad a ukow parameter vector i a ope bouded set R K. Diffusio processes are widely used i fiacial models, for istace to represet the stochastic dyamics of asset returs, exchage rates, iterest rates, macroecoomic factors, etc. While the model is writte i cotiuous time, the available data are always sampled discretely i time. Igorig the differece ca result i icosistet estimators (see, e.g., Merto (198) ad Melio (1994)). A umber of ecoometric methods have bee recetly developed to estimate the parameters of (1.1), without requirig that a cotiuous record of observatios be available. Some of these methods are based o simulatios (Gouriéroux, Mofort, ad Reault (1993), Gallat ad Tauche (1996)), others o the geeralized method of momets (Hase ad Scheikma (1995), Duffie ad Gly (1997), Kessler ad Sorese 1 I am grateful to David Bates, Reé Carmoa, Freddy Delbae, Ro Gallat, Lars Hase, Bjarke Jese, Per Myklad, Peter C. B. Phillips, Rolf Poulse, Peter Robiso, Chris Rogers, Agel Serrat, Chris Sims, George Tauche, ad i particular a coeditor ad three aoymous referees for very helpful commets ad suggestios. Robert Kimmel ad Erst Schaumburg provided excellet research assistace. This research was supported by a Alfred P. Sloa Research Fellowship ad by the NSF uder Grat SBR Mathematica code to calculate the closedform desity sequece ca be foud at yacie. 223
2 224 yacie aïtsahalia (1999)), oparametric desitymatchig (AïtSahalia (1996a, 1996b)), oparametric regressio for approximate momets (Stato (1997)), or are Bayesia (Eraker (1997) ad Joes (1997)). As i most cotexts, provided oe trusts the parametric specificatio (1.1), maximumlikelihood is the method of choice. The major caveat i the preset cotext is that the likelihood fuctio for discrete observatios geerated by (1.1) caot be determied explicitly for most models. Let p X x x deote the coditioal desity of X t+ = x give X t = x iduced by the model (1.1), also called the trasitio fuctio. Assume that we observe the process at dates t = i i =, where > is fixed. 2 Bayes rule combied with the Markovia ature of (1.1), which the discrete data iherit, imply that the loglikelihood fuctio has the simple form (1.2) l l { p X X i X i 1 i=1 For some of the rare exceptios where p X is available i closedform, see Wog (1964); i fiace, the models of Black ad Scholes (1973), Vasicek (1977), Cox, Igersoll, ad Ross (1985), ad Cox (1975) all rely o the kow closedform expressios. If samplig of the process were cotiuous, the situatio would be simpler. First, the likelihood fuctio for a cotiuous record ca be obtaied by meas of a classical absolutely cotiuous chage of measure (see, e.g., Basawa ad Prakasa Rao (198)). 3 Secod, whe the samplig iterval goes to zero, expasios for the trasitio fuctio i small time are available i the statistical literature (see, e.g., Azecott (1981)). DacuhaCastelle ad FloresZmirou (1986) calculate expressios for the trasitio fuctio i terms of fuctioals of a Browia Bridge. With discretetime samplig, the available methods to compute the likelihood fuctio ivolve either solvig umerically the Fokker PlackKolmogorov partial differetial equatio (see, e.g., Lo (1988)) or simulatig a large umber of sample paths alog which the process is sampled very fiely (see Pederse (1995) ad SataClara (1995)). Neither method produces a closedform expressio to be maximized over : the criterio fuctio takes either the form of a implicit solutio to a partial differetial equatio, or a sum over the outcome of the simulatios. By cotrast, I costruct a closedform sequece p J X of approximatios to the trasitio desity, hece from (1.2) a sequece l J of approximatios to the loglikelihood fuctio l. I also provide empirical evidece that J = 2 or 3 is amply adequate for models that are relevat i fiace. 4 Sice the expressio 2 See Sectio 3.1 for extesios to the cases where the samplig iterval is timevaryig ad eve possibly radom. 3 Note that the cotiuousobservatio likelihood is oly defied if the diffusio fuctio is kow. 4 I additio, Jese ad Poulse (1999) have recetly completed a compariso of the method of this paper agaist four alteratives: a discrete Euler approximatio of the cotiuoustime model
3 maximum likelihood estimatio 225 Notes: This figure reports the average uiform absolute error of various desity approximatio techiques applied to the Vasicek, CoxIgersollRoss ad BlackScholes models. Euler refers to the discretetime, cotiuousstate, firstorder Gaussia approximatio scheme for the trasitio desity give i equatio (5.4); Biomial Tree refers to the discretetime, discretestate (two) approximatio; Simulatios refers to a implemetatio of Pederse (1995) s simulatedlikelihood method; PDE refers to the umerical solutio of the FokkerPlackKolmogorov partial differetial equatio satisfied by the trasitio desity, usig the Crak Nicolso algorithm. For implemetatio details o the differet methods cosidered, see Jese ad Poulse (1999). Figure 1. Accuracy ad speed of differet approximatio methods for p X. to be maximized is explicit, the effort ivolved is miimal, idetical to a stadard maximumlikelihood problem with a kow likelihood fuctio. Examples are cotaied i a compaio paper (AïtSahalia (1999)), which provides, for differet models, the correspodig expressio of p J X. Besides makig maximumlikelihood estimatio feasible, these closedform approximatios have other applicatios i fiacial ecoometrics. For istace, they could be used for derivative pricig, for idirect iferece (see Gouriéroux, Mofort, ad Reault (1993)), which i its simplest versio uses a Euler approximatio as istrumetal model, or for Bayesia iferece basically wheever a expressio for the trasitio desity is required. The paper is orgaized as follows. Sectio 2 describes the sequece of desity approximatios ad proves its covergece. Sectio 3 studies the properties of the resultig maximumlikelihood estimator. I Sectio 4, I show how to calculate i closedform the coefficiets of the approximatio ad readers primarily iterested i applyig these results to a specific model ca go there directly. l J (1.1), a biomial tree approximatio, the umerical solutio of the PDE, ad simulatiobased methods, all i the cotext of various specificatios ad parameter values that are relevat for iterest rate ad stock retur models. To give a idea of the relative accuracy ad speed of these approximatios, Figure 1 summarizes their mai results. As is clear from the figure, the approximatio of the trasitio fuctio derived here provides a degree of accuracy ad speed that is umatched by ay of the other methods.
4 226 yacie aïtsahalia Sectio 5 gives the results of Mote Carlo simulatios. Sectio 6 cocludes. All proofs are i the Appedix. 2 a sequece of expasios of the trasitio fuctio To uderstad the costructio of the sequece of approximatios to p X, the followig aalogy may be helpful. Cosider a stadardized sum of radom variables to which the Cetral Limit Theorem (CLT) apply. Ofte, oe is willig to approximate the actual sample size by ifiity ad use the N 1 limitig distributio for the properly stadardized trasformatio of the data. If ot, higher order terms of the limitig distributio (for example the classical Edgeworth expasio based o Hermite polyomials) ca be calculated to improve the small sample performace of the approximatio. The basic idea of this paper is to create a aalogy betwee this situatio ad that of approximatig the trasitio desity of a diffusio. Thik of the samplig iterval as playig the role of the sample size i the CLT. If we properly stadardize the data, the we ca fid out the limitig distributio of the stadardized data as teds to (by aalogy with what happes i the CLT whe teds to ifiity). Properly stadardizig the data i the CLT meas summig them ad dividig by 1/2 ; here it will ivolve trasformig the origial diffusio X ito aother oe, which I call Z below. I both cases, the appropriate stadardizatio makes N 1 the leadig term. I will the refie this N 1 approximatio by correctig for the fact that is ot (just as i practical applicatios of the CLT is ot ifiity), i.e., by computig the higher order terms. As i the CLT case, it is atural to cosider higher order terms based o Hermite polyomials, which are orthogoal with respect to the leadig N 1 term. But i what sese does such a expasio coverge? I the CLT case, the covergece is uderstood to mea that the series with a fixed umber of corrective terms (i.e., fixed J ) coverges whe the sample size goes to ifiity. I fact, for a fixed, the Edgeworth expasio will typically diverge as more ad more corrective terms are added, uless the desity of each of these radom variables was close to a Normal desity to start with. I will make this statemet precise later, usig the criterio of Cramér (1925): the desity p z to be expaded aroud a N 1 must have tails sufficietly thi for exp z 2 /2 p z 2 to be itegrable. The poit however is that the desity p X caot i geeral be approximated for fixed aroud a Normal desity, because the distributio of the diffusio X is i geeral too far from that of a Normal. For istace, if X follows a geometric Browia motio, the right tail of the correspodig logormal desity p X is too large for its Hermite expasio to coverge. Ideed, that tail is of order x 1 exp l 2 x as x teds to +. Similarly, the expasio of ay N v desity aroud a N 1 diverges if v>2, ad hece the class of trasitio desities p X for which straight Hermite expasios coverge i the sese of addig more terms (J icreases with fixed) is quite limited.
5 maximum likelihood estimatio 227 To obtai evertheless a expasio that coverges as more correctio terms are added while remais fixed, I will show that the trasformatio of the diffusio process X ito Z i fact guaratees (ulike the CLT situatio) that the resultig variable Z has a desity p Z that belogs to the class of desities for which the Hermite series coverges as more polyomial terms are added. I will the costruct a coverget Hermite series for p Z. Sice Z is a kow trasformatio of X, I will be able to revert the trasformatio from X to Z ad by the Jacobia formula obtai a expasio for the desity of X. As a result of trasformig Z back ito X, which i geeral is a oliear trasformatio (uless x is idepedet of the state variable x), the leadig term of the expasio for the desity p X will be a deformed, or stretched, Normal desity rather tha the N 1 leadig term of the expasio for p Z. The rest of this sectio makes this basic ituitio rigorous. I particular, Theorem 1 will prove that such a expasio coverges uiformly to the ukow p X. 2 1 Assumptios ad First Trasformatio X I start by makig fairly geeral assumptios o the fuctios ad. I particular, I do ot assume that ad satisfy the typical growth coditios at ifiity, or do I restrict attetio to statioary diffusios oly. Let D X = x x deote the domai of the diffusio X. I will cosider the two cases where D X = + ad D X = +. The latter case is ofte relevat i fiace, whe cosiderig models for asset prices or omial iterest rates. I additio, the fuctio is ofte specified i fiacial models i such a way that lim x + x = ad ad/or violate the liear growth coditios ear the boudaries. For these reasos, I will devise a set of assumptios where growth coditios (without costrait o the sig of the drift fuctio ear the boudaries) are replaced by assumptios o the sig of the drift ear the boudaries (without restrictio o the growth of the coefficiets). The assumptios are: Assumptio 1 (Smoothess of the Coefficiets): The fuctios x ad x are ifiitely differetiable i x, ad three times cotiuously differetiable i, for all x D X ad. Assumptio 2 (NoDegeeracy of the Diffusio): 1. If D X = +, there exists a costat c such that x >c> for all x D X ad. 2. If D X = + lim x + x = is possible, but the there exist costats > > such that x x for all <x ad. Whether or ot lim x + x =, is a odegeerate o +, that is: for each >, there exists a costat c such that x c > for all x + ad. The first step I employ towards costructig the sequece of approximatios to p X cosists i stadardizig the diffusio fuctio of X, i.e., trasformig X
6 228 yacie aïtsahalia ito defied as 5 (2.1) X X = du/ u where ay primitive of the fuctio 1/ may be selected, i.e., the costat of itegratio is irrelevat. Because >od X, the fuctio is icreasig ad ivertible for all. It maps D X ito D = y ȳ, the domai of, where y lim x x + x ad ȳ lim x x x. For example, if D X = + ad x = x, the = 1 X 1 if < <1 (so D = +, = l X if = 1 (so D = + ad = 1 X 1 if >1 (so D =. For a give model uder cosideratio, assume that the parameter space is restricted i such a way that D is idepedet of i. This restrictio o is iessetial, but it helps keep the otatio simple. By applyig Itô s Lemma, has uit diffusio, that is (2.2) d t = t dt + dw t where y = 1 y 1 y 1 2 x 1 y Assumptio 3 (Boudary Behavior): For all y ad its derivatives with respect to y ad have at most polyomial growth 6 ear the boudaries ad lim y y + or y y < + where is the potetial, i.e., y 2 y + y / y /2. 1. Left Boudary: If y =, there exist costats such that for all < y ad y y where either >1 ad >, or = 1 ad 1. Ify =, there exist costats E > ad K> such that for all y E ad y Ky. 2. Right Boudary: If ȳ =+, there exist costats E > ad K> such that for all y E ad y Ky. Ifȳ =, there exist costats such that for all >y ad y y where either >1 ad > or = 1 ad 1/2. Note that is ot restricted from goig to ear the boudaries. Assumptio 3 is formulated i terms of the fuctio for reasos of coveiece, but the restrictio it imposes o the origial fuctios ad follows from (2.1). Assumptio 3 oly restricts how large ca grow if it has the wrog sig, meaig that is positive ear y ad egative ear y: the liear growth is the maximum possible rate. But if has the right sig, the process is beig pulled 5 The same trasformatio, sometimes referred to as the Lamperti trasform, has bee used, for istace, by Flores (1999). 6 Defie a ifiitely differetiable fuctio f as havig at most polyomial growth if there exists a iteger p such that y p f y is bouded above i a eighborhood of ifiity. If p = 1 f is said to have at most liear growth, ad if p = 2 at most quadratic growth. Near, polyomial growth meas that y +p f y is bouded.
7 maximum likelihood estimatio 229 back away from the boudaries ad I do ot restrict how fast meareversio occurs (up to a arbitrary large polyomial rate for techical reasos). The costraits o the behavior of the fuctio are essetially the best possible for the followig reasos. If has the wrog sig ear a ifiity boudary, ad grows faster tha liearly, the explodes (i.e., ca reach the ifiity boudary) i fiite time. Near a zero boudary, say y =, if there exist > ad <1 such that y ky i a eighborhood of +, the becomes attaiable. The behavior of the diffusio implied by the assumptios made is fully characterized by the followig propositio, where T if t t D = y ȳ deotes the exit time from D : Propositio 1: Uder Assumptios 1 3, (2.2) admits a weak solutio t t, uique i probability law, for every distributio of its iitial value. 7 The boudaries of D are uattaiable, i the sese that Prob T = = 1. Fially, if + is a right boudary, the it is atural if, ear + y Ky ad etrace if y Ky for some >1. If is a left boudary, the it is atural if, ear y K y ad etrace if y K y for some >1. If is a boudary (either right or left), the it is etrace. 8 Note also that Assumptio 3 either requires or implies that the process is statioary. Whe both boudaries of the domai D are etrace boudaries, the the process is ecessarily statioary with commo ucoditioal (margial) desity for all t { y / ȳ { v (2.3) y exp 2 u du exp 2 u du dv y provided that the iitial radom variable is itself distributed with desity (2.3) (see, e.g., Karli ad Taylor (1981)). Whe at least oe of the boudaries is atural, statioarity is either precluded or implied i that the (oly) possible cadidate for statioary desity,, may or may ot be itegrable ear 7 A weak solutio to (2.2) i the iterval D is a pair W, a probability space ad a filtratio, such that W satisfies the stochastic itegral equatio that uderlies the stochastic differetial equatio (2.2). For a formal defiitio, see, e.g., Karatzas ad Shreve (1991, Defiitio 5.5.2). Uiqueess i law meas that two solutios would have idetical fiitedimesioal distributios, i.e., i particular the same observable implicatios for ay discretetime data. From the perspective of statistical iferece from discrete observatios, this is therefore the appropriate cocept of uiqueess. 8 Natural boudaries ca either be reached i fiite time, or ca the diffusio be started or escape from there. Etrace boudaries, such as, caot be reached startig from a iterior poit i D = +, but it is possible for to begi there. I that case, the process moves quickly away from ad ever returs there. Typically, ecoomic cosideratios require the boudaries to be uattaiable; however, they say little about how the process would behave if it were to start at the boudary, or whether that is eve possible, ad hece it is sesible to allow both types of boudary behavior.
8 23 yacie aïtsahalia the boudaries. 9 Next, I show that the diffusio admits a smooth trasitio desity: Propositio 2: Uder Assumptios 1 3, admits a trasitio desity p y y that is cotiuously differetiable i >, ifiitely differetiable i y D ad y D, ad three times cotiuously differetiable i. Furthermore, there exists > such that for every, there exist positive costats C i i= 4, ad D such that for every ad y y D 2 : <p y y C 1/2 e 3 y y 2 /8 e C 1 y y y +C 2 y y +C 3 y +C 4 y 2 (2.4) p y y / y (2.5) D 1/2 e 3 y y 2 /8 P y y e C 1 y y y +C 2 y y +C 3 y +C 4 y 2 where P is a polyomial of fiite order i y y, with coefficiets uiformly bouded i. Fially, if ear the right boudary + ad ear the left boudary (either or ), the =+. The ext result shows that these properties essetially exted to the diffusio X of origial iterest. Corollary 1: Uder Assumptios 1 3, (1.1) admits a weak solutio X t t, uique i probability law, for every distributio of its iitial value X. The boudaries of D X are uattaiable, i the sese that Prob T X = = 1 where T X if t X t D X. I additio, X admits a trasitio desity p X x x which is cotiuously differetiable i >, ifiitely differetiable i x D X ad x D X, ad three times cotiuously differetiable i. 2 2 Secod Trasformatio Z The boud (2.4) implies that the tails of p have a Gaussialike upper boud. I light of the discussio at the begiig of Sectio 2 about the requiremets for covergece of a Hermite series, this is a big step forward. However, while, thaks to its uit diffusio = 1, is closer to a Normal variable tha X is, it is ot practical to expad p. This is due to the fact that p gets peaked aroud the coditioal value y whe gets small. Ad a Dirac mass is ot a particularly appealig leadig term for a expasio. For that reaso, I perform a further trasformatio. For give >, ad y R, defie the pseudoormalized icremet of as (2.6) Z 1/2 y 9 For istace, both a OrsteiUhlebeck process, where y = y, ad a Browia motio, where y =, satisfy the assumptios made, ad both have atural boudaries at ad +. et the former process is statioary, due to meareversio, while the latter (ull recurret) is ot.
9 maximum likelihood estimatio 231 Of course, sice I do ot require that, I make o claim regardig the degree of accuracy of this stadardizatio device, hece the term pseudo. However, I will show that for fixed Z defied i (2.6) happes to be close eough to a N 1 variable to make it possible to create a coverget series of expasios for its desity p Z aroud a N 1. I other words, Z turs out to be the appropriate trasformatio of X if we are goig to start a expasio with a N 1 term. Expasios startig with a differet leadig term could be cosidered (with matchig orthogoal polyomials) but, should i fact be small, they would have the drawback of startig with a iadequate leadig term ad therefore requirig additioal correctio. 1 Let p y y deote the coditioal desity of t+ t, ad defie the desity fuctio of Z (2.7) p Z z y 1/2 p 1/2 z + y y Oce I have obtaied a sequece of approximatios to the fuctio z y p Z z y, I will backtrack ad ifer a sequece of approximatios to the fuctio y y p y y by ivertig (2.7): (2.8) p y y 1/2 p Z 1/2 y y y ad the back to the object of iterest x x p X x x, by applyig agai the Jacobia formula for the chage of desity: (2.9) p X x x = x 1 p x x 2 3 Approximatio of the Trasitio Fuctio of the Trasformed Data So this leaves us with the eed to approximate the desity fuctio p Z.For that purpose, I costruct a Hermite series expasio for the coditioal desity of the variable Z t, which has bee costructed precisely so that it is close eough to a N 1 variable for a expasio aroud a N 1 desity to coverge. The classical Hermite polyomials are H j z e z2 /2 dj [ e z2 /2 (2.1) ] j dz j ad let z e z2 /2 / 2 deote the N 1 desity fuctio. Also, defie J (2.11) p J Z z y z j= j Z y H j z as the Hermite expasio of the desity fuctio z p Z z y (for fixed y, ad ). 11 By orthoormality of the Hermite polyomials, divided by j! 1 This is because the limitig form of the desity for a diffusio, which is drive by a Browia motio, is Gaussia. However a differet leadig term would be appropriate for processes of a differet kid (for example drive by a obrowia Lévy process). 11 Hece the boudary behavior of the trasitio desity approximatio is desiged to match that of the true desity as the forward variable (ot the backward variable) ears the boudaries of the support: uder the assumptios made, p Z ear the boudaries.
10 232 yacie aïtsahalia with respect to the L 2 scalar product weighted by the Normal desity, the coefficiets j Z are give by (2.12) j Z y 1/j! H j z p Z z y dz Sectio 4 will idicate how to approximate these coefficiets i closedform, yieldig a fully explicit sequece of approximatios to p Z. By aalogy with (2.8), I ca the form the sequece of approximatios to p as (2.13) p J y y 1/2 p J 1/2 y y y Z ad fially approximate p X by mimickig (2.9), i.e., (2.14) p J X x x x 1 p J x x The followig theorem proves that the expasio (2.14) coverges uiformly as more terms are added, ad that the limit is ideed the true (but ukow) desity fuctio p X. Theorem 1: Uder Assumptios 1 3, there exists > (give i Propositio 1) such that for every, ad x x D 2 X : (2.15) p J X x x p X x x J I additio, the covergece is uiform i over ad i x over compact subsets of D X.If x >c> o D X, the the covergece is further uiform i x over the etire domai D X.IfD X = + ad lim x + x =, the the covergece is uiform i x i each iterval of the form + >. 3 a sequece of approximatios to the maximumlikelihood estimator I ow study the properties of the sequece of maximumlikelihood estimators ˆ J derived from maximizig over i the approximate likelihood fuctio computed from p J X, i.e., (1.2) with p X replaced by p J X.12 I will show that ˆ J coverges as J to the true (but ucomputable i practice) maximumlikelihood estimator ˆ. I further prove that whe the sample size gets larger ( ), oe ca fid J such that ˆ J coverges to the true parameter value The roots of the Hermite polyomials are such that p J X > o a iterval c J c J with c J as J.Leta J be a positive sequece covergig to as J. Defie J as a (smooth) versio of the trimmig idex takig value 1 if p J X >a J ad a J otherwise. Before takig the logarithm, replace p J X by J p J X. It is show i the Appedix that such trimmig is asymptotically irrelevat. 13 This setup is differet from either the psuedomaximum likelihood oe (see White (1982) ad Gouriéroux, Mofort, ad Trogo (1984)), or the semioparametric case (Gallat ad Nychka
11 maximum likelihood estimatio Likelihood Fuctio: Iitial Observatio ad Radom Samplig Extesio Whe defiig the loglikelihood fuctio i (1.2), I igored the ucoditioal desity of the first observatio, l X, because it is domiated by the sum of the coditioal desity terms l p X X i X i 1 as. The sample cotais oly oe observatio o the ucoditioal desity ad o the trasitio fuctio, so that the iformatio o cotaied i the sample does ot icrease with. All the distributioal properties below will be asymptotic, so the defiitio (1.2) is appropriate for the loglikelihood fuctio (see Billigsley (1961)). I ay case, reitroducig the term l X back ito the loglikelihood poses o difficulty. Note also that I have assumed for coveiece that is idetical across pairs of successive observatios. If istead varies determiistically, say i is the time iterval betwee the i 1 th ad ith observatios, the it is clear from (1.2) that it suffices to replace by its actual value i whe evaluatig the trasitio desity for the ith pair of observatios. If the samplig iterval is radom, the oe ca write dow the joit likelihood fuctio of the pair of observatios ad i ad utilize Bayes Rule to express it as the product of the coditioal desity of the ith observatio X i give the i 1 th ad i, times the margial desity q of i : that is p X i X i X i 1 q j where is a parameter vector parameterizig the samplig desity. 14 If the samplig process is idepedet of X ad, the the margial desity is irrelevat for the likelihood maximizatio ad the coditioal desity is the same fuctio p X as before, evaluated at the realizatio i. Hece for the purpose of estimatig, the criterio fuctio (1.2) is uchaged ad as i the determiistic case it suffices to replace by the realizatio i correspodig to the time iterval betwee the i 1 th ad ith observatios. By cotrast, whe the samplig iterval is radom ad iformative about the parameters of the uderlyig process (for example, if more rapid arrivals of trade sigal a icrease of price volatility), the the joit desity caot be itegrated out as simply. I ow retur to the base case of fixed samplig at iterval. 3 2 Properties of the MaximumLikelihood Estimator To aalyze the properties of the estimators ˆ ad ˆ J, I itroduce the followig otatio. Defie the K K idetity matrix as Id ad L i (1987)). We are i a somewhat atypical situatio i the sese that the psuedolikelihood does approximate the true likelihood fuctio, ad we wish to exploit this fact. I particular, the choice of J is idepedet of ad J ca always be chose sufficietly large to make the resultig estimator arbitrarily close to the true MLE. This paper is ot cocered with the potetial misspecificatio of the true likelihood fuctio, i.e., it accepts (1.1) as the data geeratig process, but the does ot require that the desities belog to specific classes such as the liear expoetial family. 14 To isure that Theorem 1 remais applicable whe is ot costat, assume that the distributio of has a support cotaied i a iterval of the form where < < <. I this case, the covergece i Theorem 1 is uiform i.
12 234 yacie aïtsahalia l p X X i X i 1. L i (ad additioal dots) deotes differetiatio with respect to, ad T deotes traspositio. The score vector V L i=1 i is a martigale. Corollary 1 proved that p X admits three cotiuous derivatives with respect to i ; the same holds for p J X by direct ispectio of its expressio give i Sectio 2.3. Next defie (3.1) i E L i L i T H L i i=1... I diag i T L i i=1 i=1 The fiiteess of i for every is proved as part of Propositio 3 below. Note that if the process is ot statioary. E L i L i T varies with the time idex i because it depeds o the joit distributio of X i X i 1. The square root of the diagoal elemet i i will determie the appropriate speeds of covergece for the correspodig compoet of ˆ, ad I defie the local I 1/2 eighborhoods of the true parameter as N / I 1/2, where deotes the Euclidea orm o K. Recall that E H = i. 15 To idetify the parameters, we make the followig assumptio. Assumptio 4 (Idetificatio): The true parameter vector I is ivertible, belogs to (3.2) I 1 a s as uiformly i ad R I 1/2 T I 1/2 is uiformly bouded i probability for all i a I 1/2 eighborhood of. If X is a statioary diffusio, a sufficiet coditio that guaratees (3.2) is that for all k = 1 K, ad x D X, (3.3) < I kk = < + x x x x l p X x x / k 2 p X x x dxdx uiformly i (where p X x x = p X x x x deotes the joit desity of observatios sampled uits of time apart) sice i that case I 1 = 1 I 1 a s. For the upper boud, it is sufficiet that l p X x x / k remai bouded as x varies i D X, but ot ecessary. For the lower boud, it is sufficiet that p X x x / k ot be zero i a 15 The order of differetiatio with respect to ad itegratio with respect to the coditioal desity p X (i.e., computatio of coditioal expectatios) ca be iterchaged due to the smoothess of the loglikelihood resultig from Corollary 1.
13 maximum likelihood estimatio 235 regio x x where the joit desity has positive mass, i.e., the trasitio fuctio p X must ot be uiformly flat i the directio of ay oe of the parameters k. Otherwise p X x x / k for all x x ad the parameter vector caot be idetified. Furthermore, a sufficiet coditio for (3.3) is that x = x ad x = x for almost all x imply =. I show i the proof of Propositio 3 that the boudedess coditio o R i Assumptio 4 is automatically satisfied i the statioary case. A ostatioary example is provided i Sectio 5. The strategy I employ to study the asymptotic properties of ˆ J is to first determie those of ˆ (see Propositio 3) ad the show that ˆ J ad ˆ share the same asymptotic properties provided oe lets J go to ifiity with (Theorem 2). I Propositio 3, I show that geeral results pertaiig to time series asymptotics (see, e.g., Basawa ad Scott (1983) ad Jegaatha (1995)) ca be applied to the preset cotext. These properties follow from first establishig that the likelihood ratio has the locally asymptotically quadratic (LAQ) structure, i.e., (3.4) l + I 1/2 h l = h S h T G h /2 + o p 1 for every bouded sequece h such that + I 1/2 h, where S I 1/2 V ad G I 1/2 H I 1/2. The, depedig upo the joit distributio of S G, differet cases arise: Propositio 3: Uder Assumptios 1 4, ad for, the likelihood ratio satisfies the LAQ structure (3.4), the MLE ˆ is cosistet ad has the followig properties: i. (Locally Asymptotically Mixed Normal Structure): If (3.5) d S G G 1/2 Z G where Z is a N Id variable idepedet of the possibly radom but almost surely fiite ad positive defiite matrix G, the d (3.6) I 1/2 ˆ G 1/2 N Id Suppose that is a alterative estimator such that for ay h R K ad, (3.7) I 1/2 I 1/2 h d F uder P 1/2 +I h where F is a proper law, ot ecessarily Normal. The ˆ has maximum cocetratio i that class, i.e., is closer to tha is, i the sese that for ay > ) (3.8) C lim Prob ( I 1/2 ) ˆ C lim ( Prob I 1/2 where C + K. Further, if has the distributio I 1/2 d G 1/2 N Ṽ uder P, the Ṽ Id is oegative defiite.
14 236 yacie aïtsahalia ii. (Locally Asymptotically Normal Structure): If X is a statioary diffusio, the a special case of the LAMN structure arises where (3.3) is a sufficiet coditio for Assumptio 4, i E L 1 L 1 T is Fisher s iformatio matrix, i = i I diag i I = I G = I 1/2 i I 1/2 is a oradom matrix ad (3.6) reduces to (3.9) d 1/2 ˆ N i ( 1) The efficiecy result simplifies to the FisherRao form: i 1 is the smallest possible asymptotic variace amog that of all cosistet ad asymptotically Normal estimators of. iii. (Locally Asymptotically Browia Fuctioal structure): If ( S G d 1 1 ) (3.1) M dw M M T d where M W is a Gaussia process such that W is a stadard Browia motio, the (3.11) ( I 1/2 ˆ d 1 ) 1 M M T d 1 M dw If M ad W are idepedet, the LABF is a special case of LAMN, but ot otherwise. If oe had ormed the differece ˆ by the stochastic factor diag H 1/2 rather tha by the determiistic factor I 1/2, the the asymptotic distributio of the estimator would have bee N Id rather tha G 1/2 N Id (see the example i Sectio 5). I other words, the stochastic ormig, while itrisically more complicated, may be useful if the distributio of G is itractable, sice i that case, the distributio of I 1/2 eed ot be asymptotically Normal (ad depeds o ) whereas that of the stochastically ormed differece would simply be N Id. Noe of these difficulties are preset i the statioary case, where G is oradom. 16 Sufficiet coditios ca be give that isure that the LAMN structure holds: p for istace, if G G uiformly i over compact subsets of the (3.5) ecessarily holds by applyig Theorem 1 i Basawa ad Scott (1983, page 34). Note also that whe the parameter vector is multidimesioal, the K diagoal terms of i 1/2 do ot ecessarily go to ifiity at the same rate, ulike the commo rate 1/2 i the statioary case (see agai the example i Sectio 5). Propositio 3 is ot a ed i itself sice i our cotext ˆ caot be computed explicitly. It becomes useful however whe oe proves that the approximate maximumlikelihood estimator ˆ J is a good substitute for ˆ, i the sese 16 I the termiology of Basawa ad Scott (1983), whe G is determiistic (resp. radom), the model is called ergodic (resp. oergodic). But the LAMN situatio where G is radom is oly oe particularly tractable form of oergodicity.
15 maximum likelihood estimatio 237 that the asymptotic properties of ˆ idetified i Propositio 3 carry over to ˆ J For techical reasos, a mior refiemet of Assumptio 2 is eeded:. Assumptio 5 (Stregtheig of Assumptio 2 i the limitig case where = 1 ad the diffusio is degeerate at ): Recall the costat i Assumptio 2(2), ad the costats ad i Assumptio 3(1). If = 1, the either 1 with o restrictio o, or 2 / 1 if <1. If >1, o restrictio is required. The followig theorem shows that ˆ J (ucomputable) true MLE ˆ : iherit the asymptotic properties of the Theorem 2: Uder Assumptios 1 5, ad for : i. Fix the sample size. The as J ˆ J p ˆ uder P. ii. As, a sequece J ca be chose sufficietly large to deliver ay rate of covergece of ˆ J to ˆ. I particular, there exists a sequece J such that ˆ J ˆ = o p I 1/2 uder P which the makes ˆ J ad ˆ share the same asymptotic distributio described i Propositio 3. 4 explicit expressios for the desity expasio I ow tur to the explicit computatio of the terms i the desity expasio. Theorem 1 showed that (4.1) p Z z y = z j Z y H j z j= Recall that p J Z z y deotes the partial sum i (4.1) up to j = J. From (2.12), we have (4.2) j Z y = 1/j! = 1/j! H j z p Z z y dz H j z 1/2 p ( 1/2 z + y y ) dz ( = 1/j! H j 1/2 y y ) p y y dy = 1/j! E [ ( H j 1/2 t+ y ) t = y ] so that the coefficiets j Z are specific coditioal momets of the process.as such, they ca be computed i a umber of ways, icludig for istace Mote Carlo itegratio. A particularly attractive alterative is to calculate explicitly a Taylor series expasio i for the coefficiets j Z.Letf y y be a polyomial.
16 238 yacie aïtsahalia Taylor s Theorem applied to the fuctio s E f t+s y t = y yields (4.3) E [ f t+ y t = y ] = K k= A k f y y k k! + E [ A K+1 f t+ y t = y ] K+1 K + 1! where A is the ifiitesimal geerator of the diffusio, defied as the operator A f f/ y+ 1/2 2 f/ y 2. The followig propositio provides sufficiet coditios uder which the series (4.3) is coverget: Propositio 4: Uder Assumptios 1 3, suppose that for the relevat boudaries of D = y ȳ, ear ȳ =+ y Ky for some >1; ear y = y K y for some >1; ear y = y y for some >1 ad >; ad ear ȳ = y y for some >1 ad >. The the diffusio is statioary with ucoditioal desity ad the series (4.3) coverges i L 2 for fixed >. J K Now let pz deote the Taylor series up to order K i of p J Z. The series for the first seve Hermite coefficiets j = 6 are give by Z = 1, ad to order K = 3 by: (4.4) (4.5) (4.6) (4.7) (4.8) (4.9) 1 3 Z 2 3 Z 3 3 Z 4 3 Z 5 3 Z 6 3 Z = 1/2 ( 2 1 ( = ( ( ) / ( = ( ) / 2 3/ = ( ( = ( = ( ) / 3/2 6 ( ) / 4 5/2 24 ) / 2 12 ) 3 / ) / 5/2 48 ) 2 / ) / 5/ ) 3 / ) /
17 maximum likelihood estimatio 239 where I have used the more compact otatio k m for k y / y k m. Differet ways of gatherig the terms are available (as i the CLT, where for example both the Edgeworth ad GramCharlier expasios are based o a Hermite expasio). Here, if we gather all the terms accordig to icreasig powers of istead of icreasig order of the Hermite polyomials, ad let p K Z p K Z (ad similarly for, so that p K y y = 1/2 p K ( Z 1/2 y y y ), ad the for X), we obtai a explicit represetatio of p K, give by (4.1) p K y y = 1/2 ( y y 1/2 where c y y = 1 ad for all j 1: ) ( y exp w dw y ) K k= c k y y k k! (4.11) y c j y y =j y y j w y j 1 y { w c j 1 w y + ( 2 c j 1 w y / )/ w 2 2 dw Fially, ote that i geeral, coditioal momets of the process eed ot be aalytic i time, 17 i which case (4.3) ad (4.1) must be iterpreted strictly as Taylor series. Eve whe that is the case, their relevace for empirical work lies i the fact that icludig a small umber of terms (oe, two, or three) makes the approximatio very accurate for the values these variables typically take i fiacial ecoometrics, as we shall ow see accuracy of the approximatios ad mote carlo evidece While Figure 1 shows that the approximatio of p X was extremely accurate as a fuctio of the state variables, it does ot ecessarily imply that the resultig parameter estimates would i practice ecessarily be close to the true MLE, as was proved theoretically i Theorem 2. To aswer that questio, I perform Mote Carlo experimets. Cosider first the OrsteiUhlebeck specificatio, dx t = X t dt + dw t, where 2 ad D X = +. The process X has a Gaussia trasitio desity with mea x e ad variace 1 e 2 2/ 2. I this case, = X = 1 X y =y, ad the additioal terms i 17 Note however that as a result of Theorem 1 the trasitio fuctio is aalytic i the forward state variable. The expasio is desiged to deliver a approximatio of the desity fuctio y p y y for a fixed value of the backward (coditioig) variable y. Therefore, except i the limit where becomes ifiitely small, it is ot desiged to reproduce the limitig behavior of p i the limit where y teds to the boudaries. The expasio delivers the correct behavior for y tedig to the boudaries, except i the limitig situatio of y y 1 i Assumptio 3.1 where it is oly appropriate if becomes ifiitesimally small. 18 See also the compaio paper (AïtSahalia (1999)) for examples ad a applicatio to the estimatio of iterest rate models.
18 24 yacie aïtsahalia the approximatio p J Z eed oly correct for the iadequacy of the coditioal momets i the leadig term p Z, ot for ay ogaussiaity. I other words, i the trasformatio from X to beig liear, there is o deformatio or stretchig of the Gaussia leadig term whe goig from the approximatio of p to that of p X. By specializig Propositio 3 to this model, oe obtais the followig asymptotic distributios for the MLE: 19 Corollary 2: (Asymptotic Distributio of the MLE for the Orstei Uhlebeck Model): i. If > (LAN, statioary case): (( ) ( )) ˆ (5.1) ˆ 2 2 ( d N ) e e ( 2 e e e e e 2 1 ii. If < (LAMN, explosive case), assume X = ; the ) e +1 (5.2) e 2 1 ˆ d G 1/2 N 1 ad ˆ 2 2 d N 2 4 where G has a 2 [1] distributio idepedet of the N 1. G 1/2 N 1 is a Cauchy distributio. iii. If = (LAQ, uit root case), assume X = ; the / ( d 1 ) ˆ 1 W W 2 t dt ad ˆ 2 (5.3) 2 d N 2 4 where W t deotes a stadard Browia motio. I Mote Carlo experimets, I study the behavior of the true MLE of (which is computable i this example sice the trasitio fuctio is kow i closedform), the Euler estimator, ad the estimators of this paper correspodig to oe ad two orders i respectively. The Euler approximatio correspods to a simple discretizatio of the cotiuoustime stochastic differetial equatio, where the differetial equatio (1.1) is replaced by the differece equatio X t+ X t = X t + X t t+ with t+ N 1, so that (5.4) p Euler X x x = 2 2 x 1/2 exp { ( x x x ) 2/ 2 2 x 19 Sice there is o cofusio possible i what follows, the subscript is omitted whe deotig the true parameter values.
19 maximum likelihood estimatio 241 I set the true value of 2 at 1., ad examie the behavior of the various estimators of ad 2 for the differet cases of Corollary 2, by settig = 1, 5, ad 1 (statioary root LAN), = (uit root LABF), ad = 1 (explosive root LAMN). For each value of the parameters, I perform M = 5 Mote Carlo simulatios of the sample paths geerated by the model, each cotaiig = 1 observatios. These Mote Carlo experimets aswer four separate questios. Firstly, how accurate are the various asymptotic distributios i Corollary 2? This questio is aswered i Figures 2 ad 3, where I plot the fiite sample distributios of the estimators (histograms) ad the correspodig asymptotic distributio (solid lie). The asymptotic distributio of ˆ reported i Paels A C of Figure 2 is from (5.1). Not surprisigly, as the drift parameter makes the process closer ad closer to a uit root ( decreasig from 1 to 1), the quality of the asymptotic approximatio (5.1) deteriorates ad the small sample distributio starts to resemble (5.3), which is strogly skewed. This oly affects the drift parameter; the estimator of 2 behaves i small samples as predicted by the asymptotic distributio which is compatible with the fact that the distributio for estimatig 2 is cotiuous whe goig through the = boudary. Pael A of Figure 3 reports results for the uit root case, with the asymptotic distributio give i (5.3). I the explosive case <, Pael B of Figure 3 is based o the Cauchy distributio (5.2), while Pael C exploits the possibility of radom ormig to obtai a Gaussia asymptotic distributio of the drift coefficiet (see (A.69) i the Appedix). The diffusio estimator is idetical i both Paels B ad C, ad is therefore ot repeated i Pael C. Sice the rate of covergece i ostatioary cases varies, both Paels B ad C report the distributio of the drift estimator scaled by the relevat rate of covergece, rather tha the raw distributio of ˆ as i all other paels. The simulatios show that i both ostatioary cases, ad i the statioary case whe sufficietly far away from a uit root, the asymptotic distributio of the drift estimator is a accurate guide to its small sample distributio. The secod questio these experimets address is: what is the dispersio of the MLE aroud the true value? Tables I ad II report the first four momets of the fiite sample ad asymptotic distributios. For each of the parameter values ad the M samples, I also report i these tables the first two momets of the differeces betwee the true MLE estimators of ad 2, their Euler versios ad the estimators from usig the method of this paper with oe ad two terms. This makes it possible, thirdly, to compare the MLE dispersio, or samplig oise, to the distace betwee the MLE ad the various approximatios uder cosideratio. I particular, whe selectig the order of approximatio, it is uecessary to select a value larger tha what is required to make the distace betwee ˆ J ad ˆ a order of magitude smaller tha the distace betwee ˆ ad the true value (as measured by the exact MLE samplig distributio). These simulatios show that the parameter estimates obtaied with oe ad eve more so two terms are several orders of magitude closer to the exact MLE tha
20 242 yacie aïtsahalia Figure 2. Small sample ad asymptotic distributios of the MLE for the OrsteiUhlebeck process: statioary processes. the MLE is to the parameter, so that the approximate estimates ca be used i place of the exact MLE i practice. Fially, these Mote Carlo experimets make it possible to compare the relative accuracy of the three estimators based o the Euler discretizatio approximatio, ad those of this paper. The results of the bottom part of Tables I ad II,
Properties of MLE: consistency, asymptotic normality. Fisher information.
Lecture 3 Properties of MLE: cosistecy, asymptotic ormality. Fisher iformatio. I this sectio we will try to uderstad why MLEs are good. Let us recall two facts from probability that we be used ofte throughout
More informationSequences and Series
CHAPTER 9 Sequeces ad Series 9.. Covergece: Defiitio ad Examples Sequeces The purpose of this chapter is to itroduce a particular way of geeratig algorithms for fidig the values of fuctios defied by their
More informationChapter 7 Methods of Finding Estimators
Chapter 7 for BST 695: Special Topics i Statistical Theory. Kui Zhag, 011 Chapter 7 Methods of Fidig Estimators Sectio 7.1 Itroductio Defiitio 7.1.1 A poit estimator is ay fuctio W( X) W( X1, X,, X ) of
More informationIn nite Sequences. Dr. Philippe B. Laval Kennesaw State University. October 9, 2008
I ite Sequeces Dr. Philippe B. Laval Keesaw State Uiversity October 9, 2008 Abstract This had out is a itroductio to i ite sequeces. mai de itios ad presets some elemetary results. It gives the I ite Sequeces
More informationAsymptotic Growth of Functions
CMPS Itroductio to Aalysis of Algorithms Fall 3 Asymptotic Growth of Fuctios We itroduce several types of asymptotic otatio which are used to compare the performace ad efficiecy of algorithms As we ll
More informationORDERS OF GROWTH KEITH CONRAD
ORDERS OF GROWTH KEITH CONRAD Itroductio Gaiig a ituitive feel for the relative growth of fuctios is importat if you really wat to uderstad their behavior It also helps you better grasp topics i calculus
More informationI. Chisquared Distributions
1 M 358K Supplemet to Chapter 23: CHISQUARED DISTRIBUTIONS, TDISTRIBUTIONS, AND DEGREES OF FREEDOM To uderstad tdistributios, we first eed to look at aother family of distributios, the chisquared distributios.
More informationLecture 4: Cauchy sequences, BolzanoWeierstrass, and the Squeeze theorem
Lecture 4: Cauchy sequeces, BolzaoWeierstrass, ad the Squeeze theorem The purpose of this lecture is more modest tha the previous oes. It is to state certai coditios uder which we are guarateed that limits
More informationSection IV.5: Recurrence Relations from Algorithms
Sectio IV.5: Recurrece Relatios from Algorithms Give a recursive algorithm with iput size, we wish to fid a Θ (best big O) estimate for its ru time T() either by obtaiig a explicit formula for T() or by
More informationDepartment of Computer Science, University of Otago
Departmet of Computer Sciece, Uiversity of Otago Techical Report OUCS200609 Permutatios Cotaiig May Patters Authors: M.H. Albert Departmet of Computer Sciece, Uiversity of Otago Micah Colema, Rya Fly
More informationModule 4: Mathematical Induction
Module 4: Mathematical Iductio Theme 1: Priciple of Mathematical Iductio Mathematical iductio is used to prove statemets about atural umbers. As studets may remember, we ca write such a statemet as a predicate
More information1 Introduction to reducing variance in Monte Carlo simulations
Copyright c 007 by Karl Sigma 1 Itroductio to reducig variace i Mote Carlo simulatios 11 Review of cofidece itervals for estimatig a mea I statistics, we estimate a uow mea µ = E(X) of a distributio by
More informationA probabilistic proof of a binomial identity
A probabilistic proof of a biomial idetity Joatho Peterso Abstract We give a elemetary probabilistic proof of a biomial idetity. The proof is obtaied by computig the probability of a certai evet i two
More informationClass Meeting # 16: The Fourier Transform on R n
MATH 18.152 COUSE NOTES  CLASS MEETING # 16 18.152 Itroductio to PDEs, Fall 2011 Professor: Jared Speck Class Meetig # 16: The Fourier Trasform o 1. Itroductio to the Fourier Trasform Earlier i the course,
More informationSAMPLE QUESTIONS FOR FINAL EXAM. (1) (2) (3) (4) Find the following using the definition of the Riemann integral: (2x + 1)dx
SAMPLE QUESTIONS FOR FINAL EXAM REAL ANALYSIS I FALL 006 3 4 Fid the followig usig the defiitio of the Riema itegral: a 0 x + dx 3 Cosider the partitio P x 0 3, x 3 +, x 3 +,......, x 3 3 + 3 of the iterval
More information3. Covariance and Correlation
Virtual Laboratories > 3. Expected Value > 1 2 3 4 5 6 3. Covariace ad Correlatio Recall that by takig the expected value of various trasformatios of a radom variable, we ca measure may iterestig characteristics
More informationSequences II. Chapter 3. 3.1 Convergent Sequences
Chapter 3 Sequeces II 3. Coverget Sequeces Plot a graph of the sequece a ) = 2, 3 2, 4 3, 5 + 4,...,,... To what limit do you thik this sequece teds? What ca you say about the sequece a )? For ǫ = 0.,
More informationB1. Fourier Analysis of Discrete Time Signals
B. Fourier Aalysis of Discrete Time Sigals Objectives Itroduce discrete time periodic sigals Defie the Discrete Fourier Series (DFS) expasio of periodic sigals Defie the Discrete Fourier Trasform (DFT)
More information1 The Binomial Theorem: Another Approach
The Biomial Theorem: Aother Approach Pascal s Triagle I class (ad i our text we saw that, for iteger, the biomial theorem ca be stated (a + b = c a + c a b + c a b + + c ab + c b, where the coefficiets
More informationINFINITE SERIES KEITH CONRAD
INFINITE SERIES KEITH CONRAD. Itroductio The two basic cocepts of calculus, differetiatio ad itegratio, are defied i terms of limits (Newto quotiets ad Riema sums). I additio to these is a third fudametal
More informationSoving Recurrence Relations
Sovig Recurrece Relatios Part 1. Homogeeous liear 2d degree relatios with costat coefficiets. Cosider the recurrece relatio ( ) T () + at ( 1) + bt ( 2) = 0 This is called a homogeeous liear 2d degree
More informationChapter 6: Variance, the law of large numbers and the MonteCarlo method
Chapter 6: Variace, the law of large umbers ad the MoteCarlo method Expected value, variace, ad Chebyshev iequality. If X is a radom variable recall that the expected value of X, E[X] is the average value
More informationTHE ABRACADABRA PROBLEM
THE ABRACADABRA PROBLEM FRANCESCO CARAVENNA Abstract. We preset a detailed solutio of Exercise E0.6 i [Wil9]: i a radom sequece of letters, draw idepedetly ad uiformly from the Eglish alphabet, the expected
More informationKey Ideas Section 81: Overview hypothesis testing Hypothesis Hypothesis Test Section 82: Basics of Hypothesis Testing Null Hypothesis
Chapter 8 Key Ideas Hypothesis (Null ad Alterative), Hypothesis Test, Test Statistic, Pvalue Type I Error, Type II Error, Sigificace Level, Power Sectio 81: Overview Cofidece Itervals (Chapter 7) are
More informationOutput Analysis (2, Chapters 10 &11 Law)
B. Maddah ENMG 6 Simulatio 05/0/07 Output Aalysis (, Chapters 10 &11 Law) Comparig alterative system cofiguratio Sice the output of a simulatio is radom, the comparig differet systems via simulatio should
More informationLesson 12. Sequences and Series
Retur to List of Lessos Lesso. Sequeces ad Series A ifiite sequece { a, a, a,... a,...} ca be thought of as a list of umbers writte i defiite order ad certai patter. It is usually deoted by { a } =, or
More informationAn example of nonquenched convergence in the conditional central limit theorem for partial sums of a linear process
A example of oqueched covergece i the coditioal cetral limit theorem for partial sums of a liear process Dalibor Volý ad Michael Woodroofe Abstract A causal liear processes X,X 0,X is costructed for which
More informationThe second difference is the sequence of differences of the first difference sequence, 2
Differece Equatios I differetial equatios, you look for a fuctio that satisfies ad equatio ivolvig derivatives. I differece equatios, istead of a fuctio of a cotiuous variable (such as time), we look for
More information7. Sample Covariance and Correlation
1 of 8 7/16/2009 6:06 AM Virtual Laboratories > 6. Radom Samples > 1 2 3 4 5 6 7 7. Sample Covariace ad Correlatio The Bivariate Model Suppose agai that we have a basic radom experimet, ad that X ad Y
More informationNPTEL STRUCTURAL RELIABILITY
NPTEL Course O STRUCTURAL RELIABILITY Module # 0 Lecture 1 Course Format: Web Istructor: Dr. Aruasis Chakraborty Departmet of Civil Egieerig Idia Istitute of Techology Guwahati 1. Lecture 01: Basic Statistics
More informationTAYLOR SERIES, POWER SERIES
TAYLOR SERIES, POWER SERIES The followig represets a (icomplete) collectio of thigs that we covered o the subject of Taylor series ad power series. Warig. Be prepared to prove ay of these thigs durig the
More informationDiscrete Mathematics and Probability Theory Spring 2014 Anant Sahai Note 13
EECS 70 Discrete Mathematics ad Probability Theory Sprig 2014 Aat Sahai Note 13 Itroductio At this poit, we have see eough examples that it is worth just takig stock of our model of probability ad may
More informationConvexity, Inequalities, and Norms
Covexity, Iequalities, ad Norms Covex Fuctios You are probably familiar with the otio of cocavity of fuctios. Give a twicedifferetiable fuctio ϕ: R R, We say that ϕ is covex (or cocave up) if ϕ (x) 0 for
More informationMARTINGALES AND A BASIC APPLICATION
MARTINGALES AND A BASIC APPLICATION TURNER SMITH Abstract. This paper will develop the measuretheoretic approach to probability i order to preset the defiitio of martigales. From there we will apply this
More informationChapter 7  Sampling Distributions. 1 Introduction. What is statistics? It consist of three major areas:
Chapter 7  Samplig Distributios 1 Itroductio What is statistics? It cosist of three major areas: Data Collectio: samplig plas ad experimetal desigs Descriptive Statistics: umerical ad graphical summaries
More informationDefinition. Definition. 72 Estimating a Population Proportion. Definition. Definition
7 stimatig a Populatio Proportio I this sectio we preset methods for usig a sample proportio to estimate the value of a populatio proportio. The sample proportio is the best poit estimate of the populatio
More informationRecursion and Recurrences
Chapter 5 Recursio ad Recurreces 5.1 Growth Rates of Solutios to Recurreces Divide ad Coquer Algorithms Oe of the most basic ad powerful algorithmic techiques is divide ad coquer. Cosider, for example,
More informationarxiv:1506.03481v1 [stat.me] 10 Jun 2015
BEHAVIOUR OF ABC FOR BIG DATA By Wetao Li ad Paul Fearhead Lacaster Uiversity arxiv:1506.03481v1 [stat.me] 10 Ju 2015 May statistical applicatios ivolve models that it is difficult to evaluate the likelihood,
More informationEconomics 140A Confidence Intervals and Hypothesis Testing
Ecoomics 140A Cofidece Itervals ad Hypothesis Testig Obtaiig a estimate of a parameter is ot the al purpose of statistical iferece because it is highly ulikely that the populatio value of a parameter is
More information5 Boolean Decision Trees (February 11)
5 Boolea Decisio Trees (February 11) 5.1 Graph Coectivity Suppose we are give a udirected graph G, represeted as a boolea adjacecy matrix = (a ij ), where a ij = 1 if ad oly if vertices i ad j are coected
More informationReview for College Algebra Final Exam
Review for College Algebra Fial Exam (Please remember that half of the fial exam will cover chapters 14. This review sheet covers oly the ew material, from chapters 5 ad 7.) 5.1 Systems of equatios i
More informationResearch Article Sign Data Derivative Recovery
Iteratioal Scholarly Research Network ISRN Applied Mathematics Volume 0, Article ID 63070, 7 pages doi:0.540/0/63070 Research Article Sig Data Derivative Recovery L. M. Housto, G. A. Glass, ad A. D. Dymikov
More informationHypothesis testing. Null and alternative hypotheses
Hypothesis testig Aother importat use of samplig distributios is to test hypotheses about populatio parameters, e.g. mea, proportio, regressio coefficiets, etc. For example, it is possible to stipulate
More informationIncremental calculation of weighted mean and variance
Icremetal calculatio of weighted mea ad variace Toy Fich faf@cam.ac.uk dot@dotat.at Uiversity of Cambridge Computig Service February 009 Abstract I these otes I eplai how to derive formulae for umerically
More informationTheorems About Power Series
Physics 6A Witer 20 Theorems About Power Series Cosider a power series, f(x) = a x, () where the a are real coefficiets ad x is a real variable. There exists a real oegative umber R, called the radius
More informationHypothesis Tests Applied to Means
The Samplig Distributio of the Mea Hypothesis Tests Applied to Meas Recall that the samplig distributio of the mea is the distributio of sample meas that would be obtaied from a particular populatio (with
More informationLecture Notes CMSC 251
We have this messy summatio to solve though First observe that the value remais costat throughout the sum, ad so we ca pull it out frot Also ote that we ca write 3 i / i ad (3/) i T () = log 3 (log ) 1
More informationTHE HEIGHT OF qbinary SEARCH TREES
THE HEIGHT OF qbinary SEARCH TREES MICHAEL DRMOTA AND HELMUT PRODINGER Abstract. q biary search trees are obtaied from words, equipped with the geometric distributio istead of permutatios. The average
More informationConfidence Intervals for One Mean with Tolerance Probability
Chapter 421 Cofidece Itervals for Oe Mea with Tolerace Probability Itroductio This procedure calculates the sample size ecessary to achieve a specified distace from the mea to the cofidece limit(s) with
More informationSection 9.2 Series and Convergence
Sectio 9. Series ad Covergece Goals of Chapter 9 Approximate Pi Prove ifiite series are aother importat applicatio of limits, derivatives, approximatio, slope, ad cocavity of fuctios. Fid challegig atiderivatives
More informationChapter 5: Inner Product Spaces
Chapter 5: Ier Product Spaces Chapter 5: Ier Product Spaces SECION A Itroductio to Ier Product Spaces By the ed of this sectio you will be able to uderstad what is meat by a ier product space give examples
More informationWeek 3 Conditional probabilities, Bayes formula, WEEK 3 page 1 Expected value of a random variable
Week 3 Coditioal probabilities, Bayes formula, WEEK 3 page 1 Expected value of a radom variable We recall our discussio of 5 card poker hads. Example 13 : a) What is the probability of evet A that a 5
More informationThe Limit of a Sequence
3 The Limit of a Sequece 3. Defiitio of limit. I Chapter we discussed the limit of sequeces that were mootoe; this restrictio allowed some shortcuts ad gave a quick itroductio to the cocept. But may importat
More information2.3. GEOMETRIC SERIES
6 CHAPTER INFINITE SERIES GEOMETRIC SERIES Oe of the most importat types of ifiite series are geometric series A geometric series is simply the sum of a geometric sequece, Fortuately, geometric series
More informationwhen n = 1, 2, 3, 4, 5, 6, This list represents the amount of dollars you have after n days. Note: The use of is read as and so on.
Geometric eries Before we defie what is meat by a series, we eed to itroduce a related topic, that of sequeces. Formally, a sequece is a fuctio that computes a ordered list. uppose that o day 1, you have
More informationInfinite Sequences and Series
CHAPTER 4 Ifiite Sequeces ad Series 4.1. Sequeces A sequece is a ifiite ordered list of umbers, for example the sequece of odd positive itegers: 1, 3, 5, 7, 9, 11, 13, 15, 17, 19, 21, 23, 25, 27, 29...
More informationLecture 7: Borel Sets and Lebesgue Measure
EE50: Probability Foudatios for Electrical Egieers JulyNovember 205 Lecture 7: Borel Sets ad Lebesgue Measure Lecturer: Dr. Krisha Jagaatha Scribes: Ravi Kolla, Aseem Sharma, Vishakh Hegde I this lecture,
More informationHere are a couple of warnings to my students who may be here to get a copy of what happened on a day that you missed.
This documet was writte ad copyrighted by Paul Dawkis. Use of this documet ad its olie versio is govered by the Terms ad Coditios of Use located at http://tutorial.math.lamar.edu/terms.asp. The olie versio
More information3. Continuous Random Variables
Statistics ad probability: 31 3. Cotiuous Radom Variables A cotiuous radom variable is a radom variable which ca take values measured o a cotiuous scale e.g. weights, stregths, times or legths. For ay
More informationNumerical Solution of Equations
School of Mechaical Aerospace ad Civil Egieerig Numerical Solutio of Equatios T J Craft George Begg Buildig, C4 TPFE MSc CFD Readig: J Ferziger, M Peric, Computatioal Methods for Fluid Dyamics HK Versteeg,
More informationConfidence Intervals for the Mean of Nonnormal Data Class 23, 18.05, Spring 2014 Jeremy Orloff and Jonathan Bloom
Cofidece Itervals for the Mea of Noormal Data Class 23, 8.05, Sprig 204 Jeremy Orloff ad Joatha Bloom Learig Goals. Be able to derive the formula for coservative ormal cofidece itervals for the proportio
More informationSwaps: Constant maturity swaps (CMS) and constant maturity. Treasury (CMT) swaps
Swaps: Costat maturity swaps (CMS) ad costat maturity reasury (CM) swaps A Costat Maturity Swap (CMS) swap is a swap where oe of the legs pays (respectively receives) a swap rate of a fixed maturity, while
More informationCHAPTER 3 DIGITAL CODING OF SIGNALS
CHAPTER 3 DIGITAL CODING OF SIGNALS Computers are ofte used to automate the recordig of measuremets. The trasducers ad sigal coditioig circuits produce a voltage sigal that is proportioal to a quatity
More information8.5 Alternating infinite series
65 8.5 Alteratig ifiite series I the previous two sectios we cosidered oly series with positive terms. I this sectio we cosider series with both positive ad egative terms which alterate: positive, egative,
More information5: Introduction to Estimation
5: Itroductio to Estimatio Cotets Acroyms ad symbols... 1 Statistical iferece... Estimatig µ with cofidece... 3 Samplig distributio of the mea... 3 Cofidece Iterval for μ whe σ is kow before had... 4 Sample
More informationCase Study. Normal and t Distributions. Density Plot. Normal Distributions
Case Study Normal ad t Distributios Bret Halo ad Bret Larget Departmet of Statistics Uiversity of Wiscosi Madiso October 11 13, 2011 Case Study Body temperature varies withi idividuals over time (it ca
More informationA Gentle Introduction to Algorithms: Part II
A Getle Itroductio to Algorithms: Part II Cotets of Part I:. Merge: (to merge two sorted lists ito a sigle sorted list.) 2. Bubble Sort 3. Merge Sort: 4. The BigO, BigΘ, BigΩ otatios: asymptotic bouds
More informationSECTION 1.5 : SUMMATION NOTATION + WORK WITH SEQUENCES
SECTION 1.5 : SUMMATION NOTATION + WORK WITH SEQUENCES Read Sectio 1.5 (pages 5 9) Overview I Sectio 1.5 we lear to work with summatio otatio ad formulas. We will also itroduce a brief overview of sequeces,
More information13 Fast Fourier Transform (FFT)
13 Fast Fourier Trasform FFT) The fast Fourier trasform FFT) is a algorithm for the efficiet implemetatio of the discrete Fourier trasform. We begi our discussio oce more with the cotiuous Fourier trasform.
More informationMaximum Likelihood Estimators.
Lecture 2 Maximum Likelihood Estimators. Matlab example. As a motivatio, let us look at oe Matlab example. Let us geerate a radom sample of size 00 from beta distributio Beta(5, 2). We will lear the defiitio
More informationThe Euler Totient, the Möbius and the Divisor Functions
The Euler Totiet, the Möbius ad the Divisor Fuctios Rosica Dieva July 29, 2005 Mout Holyoke College South Hadley, MA 01075 1 Ackowledgemets This work was supported by the Mout Holyoke College fellowship
More information1. C. The formula for the confidence interval for a population mean is: x t, which was
s 1. C. The formula for the cofidece iterval for a populatio mea is: x t, which was based o the sample Mea. So, x is guarateed to be i the iterval you form.. D. Use the rule : pvalue
More information9.8: THE POWER OF A TEST
9.8: The Power of a Test CD91 9.8: THE POWER OF A TEST I the iitial discussio of statistical hypothesis testig, the two types of risks that are take whe decisios are made about populatio parameters based
More informationFourier Series and the Wave Equation Part 2
Fourier Series ad the Wave Equatio Part There are two big ideas i our work this week. The first is the use of liearity to break complicated problems ito simple pieces. The secod is the use of the symmetries
More informationIrreducible polynomials with consecutive zero coefficients
Irreducible polyomials with cosecutive zero coefficiets Theodoulos Garefalakis Departmet of Mathematics, Uiversity of Crete, 71409 Heraklio, Greece Abstract Let q be a prime power. We cosider the problem
More informationLecture 13. Lecturer: Jonathan Kelner Scribe: Jonathan Pines (2009)
18.409 A Algorithmist s Toolkit October 27, 2009 Lecture 13 Lecturer: Joatha Keler Scribe: Joatha Pies (2009) 1 Outlie Last time, we proved the BruMikowski iequality for boxes. Today we ll go over the
More information4 n. n 1. You shold think of the Ratio Test as a generalization of the Geometric Series Test. For example, if a n ar n is a geometric sequence then
SECTION 2.6 THE RATIO TEST 79 2.6. THE RATIO TEST We ow kow how to hadle series which we ca itegrate (the Itegral Test), ad series which are similar to geometric or pseries (the Compariso Test), but of
More informationSection 73 Estimating a Population. Requirements
Sectio 73 Estimatig a Populatio Mea: σ Kow Key Cocept This sectio presets methods for usig sample data to fid a poit estimate ad cofidece iterval estimate of a populatio mea. A key requiremet i this sectio
More information1 n. n > dt. t < n 1 + n=1
Math 05 otes C. Pomerace The harmoic sum The harmoic sum is the sum of recirocals of the ositive itegers. We kow from calculus that it diverges, this is usually doe by the itegral test. There s a more
More informationSection 11.3: The Integral Test
Sectio.3: The Itegral Test Most of the series we have looked at have either diverged or have coverged ad we have bee able to fid what they coverge to. I geeral however, the problem is much more difficult
More informationThe shaded region above represents the region in which z lies.
GCE A Level H Maths Solutio Paper SECTION A (PURE MATHEMATICS) (i) Im 3 Note: Uless required i the questio, it would be sufficiet to just idicate the cetre ad radius of the circle i such a locus drawig.
More informationGregory Carey, 1998 Linear Transformations & Composites  1. Linear Transformations and Linear Composites
Gregory Carey, 1998 Liear Trasformatios & Composites  1 Liear Trasformatios ad Liear Composites I Liear Trasformatios of Variables Meas ad Stadard Deviatios of Liear Trasformatios A liear trasformatio
More information4.3. The Integral and Comparison Tests
4.3. THE INTEGRAL AND COMPARISON TESTS 9 4.3. The Itegral ad Compariso Tests 4.3.. The Itegral Test. Suppose f is a cotiuous, positive, decreasig fuctio o [, ), ad let a = f(). The the covergece or divergece
More informationStandard Errors and Confidence Intervals
Stadard Errors ad Cofidece Itervals Itroductio I the documet Data Descriptio, Populatios ad the Normal Distributio a sample had bee obtaied from the populatio of heights of 5yearold boys. If we assume
More informationMethods of Evaluating Estimators
Math 541: Statistical Theory II Istructor: Sogfeg Zheg Methods of Evaluatig Estimators Let X 1, X 2,, X be i.i.d. radom variables, i.e., a radom sample from f(x θ), where θ is ukow. A estimator of θ is
More informationDivide and Conquer. Maximum/minimum. Integer Multiplication. CS125 Lecture 4 Fall 2015
CS125 Lecture 4 Fall 2015 Divide ad Coquer We have see oe geeral paradigm for fidig algorithms: the greedy approach. We ow cosider aother geeral paradigm, kow as divide ad coquer. We have already see a
More informationUnit 20 Hypotheses Testing
Uit 2 Hypotheses Testig Objectives: To uderstad how to formulate a ull hypothesis ad a alterative hypothesis about a populatio proportio, ad how to choose a sigificace level To uderstad how to collect
More informationOverview of some probability distributions.
Lecture Overview of some probability distributios. I this lecture we will review several commo distributios that will be used ofte throughtout the class. Each distributio is usually described by its probability
More information4.1 Sigma Notation and Riemann Sums
0 the itegral. Sigma Notatio ad Riema Sums Oe strategy for calculatig the area of a regio is to cut the regio ito simple shapes, calculate the area of each simple shape, ad the add these smaller areas
More informationOur aim is to show that under reasonable assumptions a given 2πperiodic function f can be represented as convergent series
8 Fourier Series Our aim is to show that uder reasoable assumptios a give periodic fuctio f ca be represeted as coverget series f(x) = a + (a cos x + b si x). (8.) By defiitio, the covergece of the series
More information8.1 Arithmetic Sequences
MCR3U Uit 8: Sequeces & Series Page 1 of 1 8.1 Arithmetic Sequeces Defiitio: A sequece is a comma separated list of ordered terms that follow a patter. Examples: 1, 2, 3, 4, 5 : a sequece of the first
More informationChapter Gaussian Elimination
Chapter 04.06 Gaussia Elimiatio After readig this chapter, you should be able to:. solve a set of simultaeous liear equatios usig Naïve Gauss elimiatio,. lear the pitfalls of the Naïve Gauss elimiatio
More informationTHE REGRESSION MODEL IN MATRIX FORM. For simple linear regression, meaning one predictor, the model is. for i = 1, 2, 3,, n
We will cosider the liear regressio model i matrix form. For simple liear regressio, meaig oe predictor, the model is i = + x i + ε i for i =,,,, This model icludes the assumptio that the ε i s are a sample
More informationThe following example will help us understand The Sampling Distribution of the Mean. C1 C2 C3 C4 C5 50 miles 84 miles 38 miles 120 miles 48 miles
The followig eample will help us uderstad The Samplig Distributio of the Mea Review: The populatio is the etire collectio of all idividuals or objects of iterest The sample is the portio of the populatio
More information0.7 0.6 0.2 0 0 96 96.5 97 97.5 98 98.5 99 99.5 100 100.5 96.5 97 97.5 98 98.5 99 99.5 100 100.5
Sectio 13 KolmogorovSmirov test. Suppose that we have a i.i.d. sample X 1,..., X with some ukow distributio P ad we would like to test the hypothesis that P is equal to a particular distributio P 0, i.e.
More informationUC Berkeley Department of Electrical Engineering and Computer Science. EE 126: Probablity and Random Processes. Solutions 9 Spring 2006
Exam format UC Bereley Departmet of Electrical Egieerig ad Computer Sciece EE 6: Probablity ad Radom Processes Solutios 9 Sprig 006 The secod midterm will be held o Wedesday May 7; CHECK the fial exam
More informationDivide and Conquer, Solving Recurrences, Integer Multiplication Scribe: Juliana Cook (2015), V. Williams Date: April 6, 2016
CS 6, Lecture 3 Divide ad Coquer, Solvig Recurreces, Iteger Multiplicatio Scribe: Juliaa Cook (05, V Williams Date: April 6, 06 Itroductio Today we will cotiue to talk about divide ad coquer, ad go ito
More informationNormal Distribution.
Normal Distributio www.icrf.l Normal distributio I probability theory, the ormal or Gaussia distributio, is a cotiuous probability distributio that is ofte used as a first approimatio to describe realvalued
More informationCHAPTER 3 THE TIME VALUE OF MONEY
CHAPTER 3 THE TIME VALUE OF MONEY OVERVIEW A dollar i the had today is worth more tha a dollar to be received i the future because, if you had it ow, you could ivest that dollar ad ear iterest. Of all
More informationAQA STATISTICS 1 REVISION NOTES
AQA STATISTICS 1 REVISION NOTES AVERAGES AND MEASURES OF SPREAD www.mathsbox.org.uk Mode : the most commo or most popular data value the oly average that ca be used for qualitative data ot suitable if
More information