SEMIMARTINGALE STOCHASTIC APPROXIMATION PROCEDURE AND RECURSIVE ESTIMATION N. LAZRIEVA, 2, T. SHARIA 3, 2 AND T. TORONJADZE Georgian American Universiy, Business School, 3, Alleyway II, Chavchavadze Ave. 7 a, Tbilisi, Georgia, E-mail: oronj333@yahoo.com 2 A. Razmadze Mahemaical Insiue,, M. Aleksidze S., Tbilisi, Georgia 3 Deparmen of Mahemaics, Royal Holloway, Universiy of London, Egham, Surrey TW2EX, E-mail:.sharia@shul.ac.uk Absrac. The semimaringale sochasic approximaion procedure, namely, he Robbins Monro ype SDE is inroduced which naurally includes boh generalized sochasic approximaion algorihms wih maringale noises and recursive parameer esimaion procedures for saisical models associaed wih semimaringales. General resuls concerning he asympoic behaviour of he soluion are presened. In paricular, he condiions ensuring he convergence, rae of convergence and asympoic expansion are esablished. The resuls concerning he Polyak weighed averaging procedure are also presened. Conens. Inroducion...................................................... 2. Convergence...................................................... 6.. The semimaringales convergence ses........................ 6.2. Main heorem................................................ 9.3. Some simple sufficien condiions for I and II............. 2.4. Examples................................................... 3. Recursive parameer esimaion procedures for saisical models associaed wih semimaringale...................... 3 2. Discree ime................................................ 2 3. RM Algorihm wih Deerminisic Regression Funcion...... 23 2. Rae of Convergence and Asympoic Expansion................. 26 2.. Noaion and preliminaries.................................. 26 2.2. Rae of convergence......................................... 28 2.3. Asympoic expansion....................................... 4 3. The Polyak Weighed Averaging Procedure...................... 46 2 Mahemaics Subjec Classificaion. 62L2, 6H, 6H3. Key words and phrases. Sochasic approximaion, Robbins Monro ype SDE, semimaringale convergence ses, sandard and nonsandard represenaions, recursive esimaion, Polyak s weighed averaging procedures.
2 N. LAZRIEVA, T. SHARIA AND T. TORONJADZE 3.. Preliminaries................................................ 48 3.2. Asympoic properies of z. Linear Case.................. 54 3.3. Asympoic properies of z. General case.................... 58 References....................................................... 6. Inroducion In 95 in he famous paper of H. Robbins and S. Monro Sochasic approximaion mehod [36] a mehod was creaed o address he problem of locaion of roos of funcions, which can only be observed wih random errors. In fac, hey carried in he classical Newon s mehod a sochasic componen. This mehod is known in probabiliy heory as he Robbins Monro RM sochasic approximaion algorihm procedure. Since hen, a considerable amoun of works has been done o relax assumpions on he regression funcions, on he srucure of he measuremen errors as well see, e.g., [7], [23], [26], [27], [28], [29], [3], [4], [42]. In paricular, in [28] by A. V. Melnikov he generalized sochasic approximaion algorihms wih deerminisic regression funcions and maringale noises do no depending on he phase variable as he srong soluions of semimaringale SDEs were inroduced. Beginning from he paper [] of A. Alber and L. Gardner a link beween RM sochasic approximaion algorihm and recursive parameer esimaion procedures was inensively exploied. Laer on recursive parameer esimaion procedures for various special models e.g., i.i.d models, non i.i.d. models in discree ime, diffusion models ec. have been sudied by a number of auhors using mehods of sochasic approximaion see, e.g., [7], [7], [23], [26], [27], [38], [39], [4]. I would be menioned he fundamenal book [32] by M. B. Nevelson and R.Z. Khas minski 972 beween hem. In 987 by N. Lazrieva and T. Toronjadze an heurisic algorihm of a consrucion of he recursive parameer esimaion procedures for saisical models associaed wih semimaringales including boh discree and coninuous ime semimaringale saisical models was proposed [8]. These procedures could no be covered by he generalized sochasic approximaion algorihm proposed by Melnikov, while in i.i.d. case he classical RM algorihm conains recursive esimaion procedures. To recover he link beween he sochasic approximaion and recursive parameer esimaion in [9], [2], [2] by Lazrieva, Sharia and Toronjadze he semimaringale sochasic differenial equaion was inroduced, which naurally includes boh generalized RM sochasic approximaion algorihms wih maringale noises and recursive parameer esimaion procedures for semimaringale saisical models. Le on he sochasic basis Ω, F, F = F, P saisfying he usual condiions he following objecs be given:
SEMIMARTINGALE STOCHASTIC APPROXIMATION PROCEDURE 3 a he random field H = {H u,, u R } = {H ω, u,, ω Ω, u R } such ha for each u R he process Hu = H u P i.e. is predicable; b he random field M = {M, u,, u R } = {Mω,, u, ω Ω,, u R } such ha for each u R he process Mu = M, u M 2 loc P ; c he predicable increasing process K = K i.e. K V + P. In he sequel we resric ourselves o he consideraion of he following paricular cases:. Mu m M 2 loc P ; 2. for each u R Mu = fu m + gu n, where m M c loc P, n M d,2 loc P, he processes fu = f, u and gu = g, u are predicable, he corresponding sochasic inegrals are well-defined and Mu M 2 loc P ; 3. for each u R Mu = ϕu m+w u µ ν, where m M c loc P, µ is an ineger-valued random measure on R E, BR + ε, ν is is P - compensaor, E, ε is he Blackwell space, W u = W, x, u,, x E P ε. Here we also mean ha all sochasic inegrals are well-defined. Laer on by he symbol Mds, u s, where u = u is some predicable process, we denoe he following sochasic line inegrals: fs, u s dm s + gs, u s dn s in case 2 or ϕs, u s dm s + W s, x, u s µ νds, dx in case 3 E provided he laers are well-defined. Consider he following semimaringale sochasic differenial equaion z = z + H s z s dk s + Mds, z s, z F.. We call SDE. he Robbins Monro PM ype SDE if he drif coefficien H u,, u R saisfies he following condiions: for all [, P -a.s. A H =, H uu < for all u. The quesion of srong solvabiliy of SDE. is well-invesigaed see, e.g., [8], [9], [3].
4 N. LAZRIEVA, T. SHARIA AND T. TORONJADZE We assume ha here exiss an unique srong soluion z = z of equaion. on he whole ime inerval [, and such ha M M 2 loc P, where M = Mds, z s. Some sufficien condiions for he laer can be found in [8], [9], [3]. The unique soluion z = z of RM ype SDE. can be viewed as a semimaringale sochasic approximaion procedure. In he presen work we are concerning wih he asympoic behaviour of he process z and also of he averized procedure z = ε z ε see Secion 3 for he definiion of z as. The work is organized as follows: In Secion we sudy he problem of convergence z as P -a.s..2 Our approach o his problem is based, a firs, on he descripion of he nonnegaive semimaringale convergence ses given in subsecion. [9] see also [9] for oher references and, a he second, on wo represenaions sandard and nonsandard of he predicable process A = A in he canonical decomposiion of he semimaringale z 2, z 2 = A + mar, in he form of difference of wo predicable increasing processes A and A 2. According o hese represenaions wo groups of condiions I and II ensuring he convergence.2 are inroduced. in subsecion.2 he main heorem concerning.2 is formulaed. Also he relaionship beween groups I and II are invesigaed. In subsecion.3 some simple condiions for I and II are given. In subsecion.4 he series of examples illusraing he efficience of all aspecs of our approach are given. In paricular, we inroduced in Example he recursive parameer esimaion procedure for semimaringale saisical models and showed how can i be reduced o he SDE.. In Example 2 we show ha he recursive parameer esimaion procedure for discree ime general saisical models can also be embedded in sochasic approximaion procedure given by.. This procedure was sudied in [39] in a full capaciy. In Example 3 we demonsrae ha he generalized sochasic approximaion algorihm proposed in [28] is covered by SDE.. In Secion 2 we esablish he rae of convergence see subsecion 2.2 and also show ha under very mild condiions he process z = z admis an asympoic represenaion where he main erm is a normed locally square inegrable maringale. In he conex of he parameric saisical esimaion his implies he local asympoic lineariy of he corresponding recursive esimaor. This resul enables one o sudy he asympoic behaviour of process z = z using a suiable form of he Cenral limi heorem for maringales see Refs. [], [2], [4], [25], [35].
SEMIMARTINGALE STOCHASTIC APPROXIMATION PROCEDURE 5 In subsecion 2. we inroduce some noaions and presen he normed process χ 2 z 2 in form χ 2 z 2 = L + R L /2,.3 where L = L M 2 loc P and L is he shifed square characerisic of L, i.e. L := + L F,P. See also subsecion 2. for he definiion of all objecs presened in.3. In subsecion 2.2 assuming z as P -a.s., we give various sufficien condiions o ensure he convergence γ δ z 2 as P -a.s..4 for all δ, < δ < δ, where γ = γ is a predicable increasing process and δ, < δ, is some consan. In his subsecion we also give series if examples illusraing hese resuls. In subsecion 2.3 assuming ha Eq..4 holds wih he process asympoically equivalen o χ 2, we sudy sufficien condiions o ensure he convergence R P as.5 which implies he local asympoic lineariy of recursive procedure z = z. As an example illusraing he efficience of inroduced condiions we consider RM sochasic approximaion procedure wih slowly varying gains see [3]. An imporan approach o sochasic approximaion problems has been proposed by Polyak in 99 [33] and Rupper in 988 [38]. The main idea of his approach is he use of averaging ieraes obained from primary schemes. Since hen he averaging procedures were sudied by a number of auhors for various schemes of sochasic approximaion [], [2], [3], [4], [5], [6], [7], [3], [34]. The mos imporan resuls of hese sudies is ha he averaging procedures lead o he asympoically opimal esimaes, and in some cases, hey converges o he limi faser han he iniial algorihms. In Secion 3 he Polyak weighed averaging procedures of he iniial process z = z are considered. They are defined as z = ε g K where g = g is a predicable process, g, z s dε s g K,.6 g s dk s <, g dk = and ε X as usual is he Dolean exponenial. Here he condiions are saed which guaranee he asympoic normally of process z = z in case of coninuous process under consideraion. The main resul of his secion is presened in Theorem 3.3., where assuming ha Eq..4 holds rue wih some increasing process γ = γ asympoically equivalen o he process Γ 2 L he condiions are given
6 N. LAZRIEVA, T. SHARIA AND T. TORONJADZE ha ensure he convergence where ε = + ε /2 z d 2 ξ, ξ N,,.7 Γ 2 s L s β s dk s. As special cases we have obained he resuls concerning averaging procedures for sandard RM sochasic approximaion algorihms and hose wih slowly varying gains. All noaions and fac concerning he maringale heory used in he presened work can be found in [2], [4], [25].. Convergence.. The semimaringales convergence ses. Le Ω, F, F = F, P be a sochasic basis saisfying he usual condiions, and le X = X be an F -adaped process wih rajecories in Skorokhod space D noaion X = F D. Le X = lim X and le {X } denoe he se, where X exiss and is a finie random variable r.v.. In his secion we sudy he srucure of he se {X } for nonnegaive special semimaringale X. Our approach is based on he muliplicaive decomposiion of he posiive semimaringales. Denoe V + V he se of processes A = A, A =, A F D wih nondecreasing bounded variaion on each inerval [, [ rajecories. We wrie X P if X is a predicable process. Denoe S P he class of special semimaringales, i.e. X S p if X F D and X = X + A + M, where A V P, M M loc. Le X S P. Denoe εx he soluion of he Dolean equaion where Y X := Y s dx s. Y = + Y X, If Γ, Γ 2 F, hen Γ = Γ 2 P -a.s. or Γ Γ 2 P -a.s. means P Γ Γ 2 = or P Γ Ω \ Γ 2 = respecively, where is he sign of he symmeric difference of ses. Le X S P. Pu A = A A 2, where A, A 2 V + P. Denoe  = + X + A 2 A 2 := + X s + A 2 s da s. Theorem... Le X S P, X. Then { < } {X } {A 2 < } P -a.s..
SEMIMARTINGALE STOCHASTIC APPROXIMATION PROCEDURE 7 Proof. Consider he process Y = + X + A 2. Then Y = Y + A + M, Y = + X, Y, Y A. Thus he processes  = Y A and M = Y + A M are well-defined and besides  V+ P, M Mloc. Then, using Theorem, 5, Ch. 2 from [25] we ge he following muliplicaive decomposiion Y = Y εâε M, where εâ V+ P, ε M M loc. Noe ha M >. Indeed, M = Y + A M. Bu M = Y A = Y Y + A > Y + A. Therefore ε M > and {ε M } = Ω P -a.s.. On he oher hand see, e.g., [3], Lemma 2.5 Hence ε   as. { < } {Y } = {X } {A 2 < }, since A 2 < Y and A 2 V +. Theorem is proved. Corollary... {A < } = { + X A < } = { < } P -a.s.. Proof. I is eviden ha I remains o noe ha Corollary is proved. Corollary..2. {A < } { + X A < } { < } {X } {A 2 < } P -a.s.. {A < } {X } {A 2 < } = { < } {X } {A 2 < } P -a.s.. { < } {ε M > } = {X } {A 2 < } {ε M > } P -a.s., as i easily follows from he proof of Theorem... Remark... The relaion {A < } {X } {A 2 < } P -a.s. has been proved in [25], Ch. 2, 6, Th. 7. Under he following addiional assumpions:. EX < ; 2. one of he following condiions α or β are saisfied: α here exiss ε > such ha A +ε F for all >,
8 N. LAZRIEVA, T. SHARIA AND T. TORONJADZE β for any predicable Markov momen σ E A σi {σ< } <. Le A, B F D. We wrie A B if B A V +. Corollary..3. Le X S P, X, A A A 2 and A A, where A, A 2 V + P. Then {A < } = { + X A < } {X } {A 2 < } P -a.s.. Proof. Rewrie X in he form X = X + A à 2 + M, where à 2 = A A V P. Then he desirable follows from Theorem.., Corollary.. and rivial inclusion {Ã2 < } {A 2 < }. The corollary is proved. Corollary..4. Le X S P, X and X = X + X B + A + M wih B V + P, A V P and M M loc. Suppose ha for A, A 2 V + P Then A A A 2 and A A. {A < } {B < } {X } {A 2 < } P -a.s.. The proof is quie similar o he prof of Corollary..3 if we consider he process Xε B. Remark..2. Consider he discree ime case. Le F, F,... be a non-decreasing sequence of σ-algebras and X n, β n, ξ n, ζ n F n, n, are nonnegaive r.v. and n X n = X + X i β i + A n + M n i= we mean ha X = X, F = F and β = ξ = ζ =, where A n F n wih A = and M M loc. Noe ha X n can always be represened in his form aking A n = n EX i F i X i n X i β i. Denoe i= A n = n ξ i and A 2 n = i= I is clear ha in his case i= n ζ i. i= A A A n ξ n A n := A n A n, n. So, in his case Corollary..4 can be formulaed in he following way:
and SEMIMARTINGALE STOCHASTIC APPROXIMATION PROCEDURE 9 Le for each n Then { } { ξ i < i= i= A n n ξ i ζ i i= A n ξ n. } { β i < {X } i= } ζ i < P -a.s.. From his corollary follows he resul by Robbins and Siegmund see Robbins, Siegmund [37]. Really, he above inclusion holds if in paricular A n ξ n ζ n, n, i.e. when EX n F n X n + β n + ξ n ζ n, n. In our erms he previous inequaliy means A A A 2..2. Main heorem. Consider he sochasic equaion RM procedure z = z + H s z s dk s + or in he differenial form Mds, z s,, z F,.2. dz = H z dk + Md, z, z F. Assume ha here exiss an unique srong soluion z = z of.2. on he whole ime inerval [,, M M 2 loc, where M := Mds, z s. We sudy he problem of P -a.s. convergence z, as. For his purpose apply Theorem.. o he semimaringale X = z 2,. Using he Io formula we ge for he process z 2 where wih dz 2 = da + dn,.2.2 da = V z dk + V + z dk d + d M, dn = 2z d M + H z K d M d + d[ M] M, V u := 2H uu, V + u := H 2 u K. Noe ha A = A V P, N M loc.
N. LAZRIEVA, T. SHARIA AND T. TORONJADZE Represen he process A in he form A = A A 2.2.3 wih { da = V + z dk d + d M, da 2 = V z dk, or { da = [V z I { K } + V + z ] + dk d + d M, 2 da 2 = {V z I { K =} [V z I { K } + V + z ] }dk, where [a] + = max, a, [a] = min, a. As i follows from condiion A α z for all and so, he represenaion.2.3 direcly corresponds o he usual in sochasic approximaion procedures sandard form of process A in.2.2 A = A A 2 wih A, A 2 from.2.3. Therefore we call represenaion.2.3 sandard, while he represenaion.2.32 is called nonsandard. Inroduce he following group of condiions: For all u R and [, A For all [, P -a.s. H =, H u < for all u ; B i Mu K, ii h u B + u 2, B, B = B P, B K <, I where h u = d Mu dk ; i i I { K } H u C + u, C, C = C P, C K <, i 2 C 2 K K d <, ii for each ε > inf V u K = ; ε u /ε II i [V ui { K } + V + u] + D + u 2, D, D = D P, D K d <, ii for each ε > inf { V u I { K =} + [V ui { K } + V + u] } K =. ε u /ε Remark.2.. When Mu m M 2 loc, we do no require he condiion m K and replace he condiion B by B m <. Remark.2.2. Everywhere we assume ha all condiions are saisfied P -a.s.
SEMIMARTINGALE STOCHASTIC APPROXIMATION PROCEDURE Remark.2.3. I is eviden ha I ii= C K =. Theorem.2.. Le condiions A, B, I or A, B, II be saisfied. Then z P -a.s. as. Proof. Assume, for example, ha he condiions A, B and I are saisfied. Then by virue of Corollary.. and.2.2 wih sandard represenaion.2.3 of process A we ge { + z 2 A < } {z 2 } {A 2 < }..2.4 Bu from condiions B and I i we have and so { + z 2 A < } = Ω P -a.s. {z 2 } {A 2 < } = Ω P -a.s...2.5 Denoe z 2 = lim z 2, N = {z 2 > } and assume ha P N >. In his case from I ii by simple argumens we ge P V z K = >, which conradics wih.2.4. Hence P N =. The proof of he second case is quie similar. The heorem is proved. In he following proposiions he relaionship beween condiions I and II are given. Proposiion.2.. I II. Proof. From I i we have [V ui { K } + V + u] + V + u C 2 K + u 2 and if ake D = C 2 K, hen II i follows from I i 2. Furher, from I i we have for each ε > and u wih ε u /ε V u I { K =} + [V u + V + u] I { K } V u V + u V u C 2 K +. ε 2 Now II ii follows from I i 2 and I ii. The proposiion is proved. Proposiion.2.2. Under I i we have I ii II ii. Proof immediaely follows from previous proposiion and rivial implicaion II ii I ii.
2 N. LAZRIEVA, T. SHARIA AND T. TORONJADZE.3. Some simple sufficien condiions for I and II. Inroduce he following group of condiions: for each u R and [, S. S.2 i i G u H u G u, G, G = G, G = G P, G K <, i 2 G 2 K K d < ; ii G K = ;.3. i G[ 2 + G K] + K d < ;.3.2 ii G{2I { K=} + [ 2 + G K] I { K } K =..3.3 Proposiion.3.. S. I, S.i, S.2 II. Proof. The firs implicaion is eviden. For he second, noe ha So V ui { K } + V + u = 2 H u u I { K } + H 2 u K H u u [ 2I { K } + G K ]..3.4 [V ui { K } + V + u] + H u u [ 2I { K } + G K ] + and II i follows from.3.2 if we ake Furher, from.3.4 we have G [ 2I { K } + G K ] + u 2 D = G [ 2 + G K ] + I { K }. V u I { K =} + [V ui { K } + V + u] and II ii follows from.2.3. Proposiion is proved. u 2 G {2I { K=} + [ 2I { K } + G K ] } Remark.3.. a S. S.2, b under S. i we have S. ii S.2 ii, c S.2 ii S. ii. Summarizing he above we come o he following conclusions: a if he condiion S. ii is no saisfied, hen S.2 ii is no saisfied also; b if S. i and S. ii are saisfied, bu S. i 2 is violaed, hen neverheless he condiions S.2 i and S.2 ii can be saisfied.
SEMIMARTINGALE STOCHASTIC APPROXIMATION PROCEDURE 3 In his case he nonsandard represenaions.2.32 is useful. Remark.3.2. Denoe G K = 2 + δ, δ 2 for all [,. I is obvious ha if δ for all [,, hen [ 2 + G K ] + =. So S.2 i is rivially saisfied and S.2 ii akes he form G{2I { K=} + δ I { K } K =..3.5 Noe ha if G min2, δ K =, hen.3.5 holds, and he simples sufficien condiion.3.5 is: for all G K =, δ cons >. Remark.3.3. Le he condiions A, B and I be saisfied. Since we apply Theorem.. and is Corollaries on he semimaringales convergence ses given in subsecion., we ge rid of many of usual resricions: momen resricions, boundedness of regression funcion, ec..4. Examples..4.. Recursive parameer esimaion procedures for saisical models associaed wih semimaringale.. Basic model and regulariy. Our objec of consideraion is a parameric filered saisical model ε = Ω, F, F = F, {P θ ; θ R} associaed wih one-dimensional F-adaped RCLL process X = X in he following way: for each θ R P θ is an unique measure on Ω, F such ha under his measure X is a semimaringale wih predicable characerisics Bθ, Cθ, ν θ w.r.. sandard runcaion funcion hx = xi { x }. Assume for simpliciy ha all P θ coincide on F. Suppose ha for each pair θ, θ loc P θ P θ. Fix θ = and denoe P = P, B = B, C = C, ν = ν. Le ρθ = ρ θ be a local densiy process likelihood raio process ρ θ = dp θ, dp, where for each θ P θ, := P θ F, P := P F are resricions of measures P θ and P on F, respecively. As i is well-known see, e.g., [4], Ch. III, 3d, Th. 3.24 for each θ here exiss a P-measurable posiive funcion Y θ = {Y ω,, x; θ, and a predicable process βθ = β θ wih ω,, x Ω R + R}, hy θ ν A + loc P, β2 θ C A + loc P,
4 N. LAZRIEVA, T. SHARIA AND T. TORONJADZE and such ha Bθ = B + βθ C + hy θ ν, 2 Cθ = C, 3 ν θ = Y θ ν. In addiion he funcion Y θ can be chosen in such a way ha a := ν{}, R = a θ := ν θ {}, R = Y, x; θν{}dx =..4. We assume ha he model is regular in he Jacod sense see [5], 3, Df. 3.2 a each poin θ, ha is he process ρ θ /ρ θ /2 is locally differeniable w.r. θ a θ wih he derivaive process Lθ = L θ M 2 locp θ. In his case he Fisher informaion process is defined as Î θ = Lθ, Lθ..4.2 In [5] see 2-c, Th. 2.28 was proved ha he regulariy of he model a poin θ is equivalen o he differeniabiliy of characerisics βθ, Y θ, aθ in he following sense: here exis a predicable process βθ and P-measurable funcion W θ wih β 2 θ C <, W 2 θ ν θ, < for all R + and such ha for all R + we have as θ θ where βθ βθ βθθ θ 2 C /θ θ 2 P θ, Y θ /2 2 2 / Y θ 2 W θθ θ ν θ, θ θ 2 P θ, [ 3 a s θ /2 a s θ /2 s a s θ< + 2 ] Ŵs θ θ 2/ a s θ /2 θ θ θ θ 2 P θ, Ŵ θ θ = W, x; θν θ {}, dx..4.3 and In his case a s θ = Ŵ s θ θ = and he process Lθ can be wrien as Lθ = βθ X c βθ C + Ŵ θ θ + Ŵ θ θ µ ν θ,.4.4 aθ Îθ = β 2 θ C + Ŵ θ θ 2 ν θ + s Ŵ θ s θ 2 a s θ..4.5
Denoe SEMIMARTINGALE STOCHASTIC APPROXIMATION PROCEDURE 5 Φθ = W θ + Ŵ θ θ aθ. One can consider he anoher alernaive definiion of he regulariy of he model see, e.g., [35] based on he following represenaion of he process ρθ: ρθ = εmθ, where Mθ = βθ X c + Y θ + Ŷ θ a a I {<a<} µ ν M loc P..4.6 Here X c is a coninuous maringale par of X under measure P see, e.g., [6], [28]. We say ha he model is regular if for almos all ω,, x he funcions β : θ β ω; θ and Y : θ Y ω,, x; θ are differeniable noaion βθ := βθ, Ẏ θ := Y θ and differeniabiliy under inegral sign is possible. θ θ Then θ ln ρθ = LṀθ, Mθ := Lθ M loc P θ, where Lm, M is he Girsanov ransformaion defined as follows: if m, M M loc P and Q P wih dq = εm, hen dp Lm, M := m + M [m, M] M loc Q. I is no hard o verify ha where Lθ = βθ X c βθ C + Φθ µ ν θ,.4.7 Φθ = Ẏ θ Y θ + ȧθ aθ wih I {aθ=} ȧθ =. If we assume ha for each θ R Lθ M 2 loc P θ, hen he Fisher informaion process is Î θ = Lθ, Lθ. I should be noiced ha from he regulariy of he model in he Jacod sense i follows ha Lθ M 2 loc P θ, while under he laer regulariy condiions Lθ M 2 loc P θ is an assumpion, in general. In he sequel we assume ha he model is regular in boh above given senses. Then W θ = Ẏ θ Y θ, Ŵ θ θ = ȧθ, Lθ = Lθ.
6 N. LAZRIEVA, T. SHARIA AND T. TORONJADZE 2. Recursive esimaion procedure for MLE. In [8] an heurisic algorihm was proposed for he consrucion of recursive esimaors of unknown parameer θ asympoically equivalen o he maximum likelihood esimaor MLE. This algorihm was derived using he following reasons: Consider he MLE θ = θ, where θ is a soluion of esimaional equaion L θ =. Assume ha for each θ R he process Îθ /2 θ θ is P θ -sochasically bounded and, in addiion, he process θ is a P θ -semimaringale; 2 for each pair θ, θ he process Lθ M 2 loc P θ and is a P θ-special semimaringale; 3 he family Lθ, θ R is such ha he Io Venzel formula is applicable o he process L, θ w.r.. P θ for each θ R ; 4 for each θ R here exiss a posiive increasing predicable process γ θ asympoically equivalen o Î θ, i.e. γ θîθ P θ as. Under hese assumpions using he Io Venzel formula for he process L, θ we ge an implici sochasic equaion for θ = θ. Analyzing he orders of infiniesimaliy of erms of his equaion and rejecing he high order erms we ge he following SDE recursive procedure dθ = γ θ Ld, θ,.4.8 where Ld, u is a sochasic line inegral w.r.. he family {L, u, u R, R + } of P θ -special semimaringales along he predicable curve u = u. To give an explici form o he SDE.4.8 for he saisical model associaed wih he semimaringale X assume for a momen ha for each u, θ including he case u = θ Φu µ A + loc P θ..4.9 Then for each pair u, θ we have Φu µ ν u = Φu µ ν θ + Φu Y u Y θ ν θ. Based on his equaliy one can obain he canonical decomposiion of P θ - special semimaringale Lu w.r.. measure P θ : Lu = βu X c βθ C + Φu µ ν θ + βuβθ βu C + Φu Y u ν θ..4. Y θ Now, using.4. he meaning of Ld, u is
SEMIMARTINGALE STOCHASTIC APPROXIMATION PROCEDURE 7 Lds, u s = β s u s dx c βθ C s + Φs, x, u s µ ν θ ds, dx + β s u s β s θ β s u s dc s + Φs, x, u s Finally, he recursive SDE.4.8 akes he form θ = θ + γ s θ s β s θ s dx c βθ C s Y s, x, u s Y s, x, θ ν θ ds, dx. + γ s θ s Φs, x, θ s µ ν θ ds, dx + + γ s θ β s θ s β s θ β s θ s dc s γ s θ s Φs, x, θ s Y s, x, θ s ν θ ds, dx..4. Y s, x, θ Remark.4.. One can give more accurae han.4.9 sufficien condiions see, e.g., [2], [4], [25] o ensure he validiy of decomposiion.4.. Assume ha here exiss an unique srong soluion θ of he SDE.4.. To invesigae he asympoic properies of recursive esimaors θ as, namely, a srong consisency, rae of convergence and asympoic expansion we reduce he SDE.4. o he Robbins Monro ype SDE. For his aim denoe z = θ θ. Then.4. can be rewrien as z = z + + + γ s θ + z s βθ + z s β s θ β s θ + z s dc s γ s θ + z s Φs, x, θ + z s Y s, x, θ + z s ν θ ds, dx Y s, x, θ γ s θ + z s β s θ + z s dx c βθ C s
8 N. LAZRIEVA, T. SHARIA AND T. TORONJADZE + γ s θ + z s Φs, x, θ + z s µ ν θ ds, dx..4.2 For he definiion of he objecs K θ, {H θ u, u R } and {M θ u, u R } we consider such a version of characerisics C, ν θ ha C = C θ A θ, ν θ ω, d, dx = da θ B θ ω,dx, where A θ = A θ A + loc P θ, C θ = C θ is a nonnegaive predicable process, and B θ ω,dx is a ransiion kernel from Ω R +, P in R, BR wih B θ ω,{} = and A θ B θ ω,r see [4], Ch. 2, 2, Prop. 2.9. Pu K θ = A θ, { H θ u = γ θ + u β θ + uβ θ β θ + uc θ } Y, x, θ + u + φ, x, θ + u B θ Y, x, θ ω,dx,.4.3 M θ, u = γ s θ + u β s θ + udx c βθ C s + γ s θ + uφs, x, θ + uµ ν θ ds, dx..4.4 Assume ha for each u M θ u = M θ, u M 2 loc P θ. Then M θ u = + + γ s θ + u β s θ + u 2 C θ s da θ s γsθ 2 + u { γsθ 2 + ubω,r θ Φ 2 s, x, θ + ubω,sdx θ da θ,c s Φ 2 s, x, θ + uq θ ω,sdx a s θ 2 } Φs, x, θ + uqω,sdx θ da θ,d s, where a s θ = A θ sb θ ω,sr, q θ ω,sdxi {asθ>} = Bθ ω,sdx B θ ω,sr I {a sθ>}.
SEMIMARTINGALE STOCHASTIC APPROXIMATION PROCEDURE 9 Now we give a more deailed descripion of Φθ, Îθ, Hθ u and M θ u. Denoe dνθ c qω,dx θ := F θ, dνc q ω, dx := f ω,x, θ := f θ. Then and Therefore Y θ = F θi {a=} + aθ a ȧθ Ẏ θ = F θi {a=} + a Φθ = F θ F θ I {a=} + fθ wih I {aθ>} fθ qθ dx =. Denoe βθ = l c F θ θ, { fθ fθ + fθi {a>} fθ + aθ a fθ I {a>}. } ȧθ I {a>}.4.5 aθ aθ fθ := F θ lπ θ, := fθ lδ ȧθ θ, := aθ aθ lb θ. Indices i = c, π, δ, b carry he following loads: c corresponds o he coninuous par, π o he Poisson ype par, δ o he predicable momens of jumps including a main special case he discree ime case, b o he binomial ype par of he likelihood score lθ = l c θ, l π θ, l δ θ, l b θ. In hese noaions we have for he Fisher informaion process: Î θ = + + l c sθ 2 dc s + [ Bω,sR θ l π s x; θ 2 B θ ω,sdxda θ,c s ] l δ sx; θ 2 qω,sdx θ da θ,d s l b sθ 2 a s θda θ,d s..4.6 For he random field H θ u we have: { H θ u = γ θ + u l c θ + uβ θ β θ + uc θ + l π x; θ + u F x; θ + u B θ F x; θ ω,dxi { A θ =} { + l δ x; θ + uqω,dx θ + l b θ + u a θ a θ + u a θ } B θ ω,ri { A θ >}..4.7
2 N. LAZRIEVA, T. SHARIA AND T. TORONJADZE Finally, we have for M θ u : M θ u = γθ + ul c θ + u 2 C θ A θ + γsθ 2 + u l π s x; θ + u 2 Bω,dxdA θ θ,c s + { γsθ 2 + ubω,sr θ l δ sx; θ + u + l b sθ + u 2 q θ ω,sdx a s θ 2 } l δ sx; θ + u + l b sθ + uqω,sdx θ da θ,d s..4.8 Thus, we reduced SDE.4.2 o he Robbins Monro ype SDE wih K θ = A θ, and H θ u and M θ u defined by.4.7 and.4.4, respecively. As i follows from.4.7 H θ = for all, P θ -a.s. As for condiion A o be saisfied i ie enough o require ha for all, u P θ -a.s. β θ + uβ θ β θ + u <, F, x, θ + u F, x; θ + u B θ F, x, θ + u F, x; θ ω,dx I { A θ =}u <, f, x; θ + u f, x; θ + u qθ dx I { A θ >}u <, ȧ θ + ua θ a θ + uu <, and he simples sufficien condiions for he laer ones is he monooniciy P -a.s. of funcions βθ, F θ, fθ and aθ w.r. θ. Remark.4.2. In he case when he model is regular in he Jacod sense only we save he same form of all above-given objecs namely of Φθ using he formal definiions: F θ F θ I {aθ=} := W θi {aθ=},.4.2. Discree ime. ȧθ := Ŵ θ, fθ fθ := W θi {aθ>} Ŵ θ θ aθ I {aθ>}. a Recursive MLE in parameer saisical models. Le X, X,..., X n,... be observaions aking values in some measurable space X, BX such ha he regular condiional densiies of disribuions w.r.. some measure µ
SEMIMARTINGALE STOCHASTIC APPROXIMATION PROCEDURE 2 f i x i, θ x i,..., x, i n, n exis, f x, θ f x, θ R is he parameer o be esimaed. Denoe P θ corresponding disribuion on Ω, F := X, BX. Idenify he process X = X i i wih coordinae process and denoe F = σx, F n = σ X i, i n. If ψ = ψx i, X i,..., X is a r.v., hen under E θ ψ F i we mean he following version of condiional expecaion E θ ψ F i := ψz, X i,..., X f i z, θ X i,..., X µdz, if he las inegral exiss. Assume ha he usual regulariy condiions are saisfied and denoe θ f ix i, θ x i,..., x := f i x i, θ x i,..., x, he maximum likelihood scores f l i θ := i X i, θ X i,..., X f i and he empirical Fisher informaion n I n θ := E θ li 2 θ F i. Denoe also i= b n θ, u := E θ l n θ + u F n and indicae ha for each θ R, n Consider he following recursive procedure b n θ, = P θ -a.s...4.9 θ n = θ n + I n θ n l n θ n, θ F. Fix θ, denoe z n = θ n θ and rewrie he las equaion in he form z n = z n + In θ + z n b n θ, z n + In θ + z n m n, z = θ θ,.4.2 where m n = mn, z n wih mn, u = l n θ + u E θ l n θ + u F n. Noe ha he algorihm.4.2 is embedded in sochasic approximaion scheme.2. wih H n u = In θ + ub n θ, u F n, K n =, Mn, u = In θ + u mn, u. This example clearly shows he necessiy of consideraion of random fields H n u and Mn, u. In Sharia [39] he convergence z n P -a.s. as n was proved under condiions equivalen o A, B and I conneced wih sandard represenaion.2.2.
22 N. LAZRIEVA, T. SHARIA AND T. TORONJADZE Remark.4.3. Le θ Θ R where θ is open proper subse of R. I may be possible ha he objecs l n θ and I n θ are defined only on he se Θ, bu for each fixed θ Θ he objecs H n u and Mn, u are well-defined funcions of variable u on whole R. Then under condiions of Theorem.2. θ n θ P θ -a.s. as n saring from arbirary θ. The example given below illusraes his siuaion. The same example illusraes also efficiency of he represenaion.2.32. b Galon Wason Branching Process wih Immigraion. Le he observable process be X i = X i j= Y i,j +, i =, 2,..., n; X =, Y i,j are i.i.d. random variables having he Poisson disribuion wih parameer θ, θ >, o be esimaed. If F i = σx j, j i, hen P θ X i = m F i = θx i m e θx i, i =, 2,... ; m. m! From his we have l i θ = X i θx i θ The recursive procedure has he form θ n = θ n + X n θ n X n n i= X i and if, as usual z n = θ n θ, hen n, I n θ = θ X i. i=, θ F,.4.2 z n = z n z n X n ε n i= X + n n i i= X,.4.22 i where ε n = X n θx n is a P θ -square inegrable maringale-difference. In fac, E θ ε n F n =, E θ ε 2 n F n = θx n. In his case H n u = ux n / n X i, Mn, u = m n = ε n / n X i, K = and so are i= well-defined on whole R. Indicae now ha he soluion of Eq..4.2 coincides wih MLE n i= θ n = X i n i= X i i= and i is easy o see ha θ n n is srongly consisen for all θ >. Indeed, n i= θ n θ = ε i n i= X i
SEMIMARTINGALE STOCHASTIC APPROXIMATION PROCEDURE 23 and desirable follows from srong law of large numbers for maringales and well-known fac see, e.g., [] ha for all θ > X i = P θ -a.s...4.23 i= Derive his resul as he corollary of Theorem.2.. Noe a firs ha for each θ > he condiions A and B are saisfied. Indeed, A H n uu = u2 X n n i= X i for all u X i >, i ; < B X n m = θ n n= i= X i <, 2 hanks o.4.23. Now o illusrae he efficiency of group of condiions II le us consider wo cases: < θ and 2 θ is arbirary, i.e. θ >. / n In case condiions I are saisfied. In fac, H n u = X n X i u and / n 2 Xn 2 X i <, Pθ -a.s. Bu if θ > he las series diverges, n= i= so he condiion I i is no saisfied. On he oher hand, he proving of desirable convergence by checking he condiions II is almos rivial. Really, use Remark.3.2 and ake G n = = X n / n X i. Then G n = P θ -a.s., for all θ >. Besides G n i= δ n = 2 + G n <, δ n. n=.4.3. RM Algorihm wih Deerminisic Regression Funcion. Consider he paricular case of algorihm.2. when H ω, u = γ ωru, where he process γ = γ P, γ > for all, dm, u = γ dm, m M 2 loc, i.e. dz = γ Rz dk + γ dm, z F. a Le he following condiions be saisfied: A R =, Ruu < for all u, B γ 2 m <, Ru C + u, C > is consan, 2 for each ε >, inf ε u ε Ru >, 3 γ K <,, γ K =, 4 γ 2 K K d <. Then z P -a.s., as. Indeed, i is easy o see ha A, B, 4 A, B and I of Theorem.2.. i=
24 N. LAZRIEVA, T. SHARIA AND T. TORONJADZE In Melnikov [28] his resul has been proved on he basis of he heorem on he semimaringale convergence ses noed in Remark... In he case when K d his auomaically leads o he momen resricions and he addiional assumpion Ru cons. b Le, as in case a, condiions A and B be saisfied. Besides assume ha for each u R and [, : V u + V + u, 2 for all ε > I ε := inf { V u + V + u} K =. ε u ε Then z P -a.s., as. Indeed, i is no hard o verify ha, 2 II. The following quesion arises: is i possible and 2 o be saisfied? Suppose in addiion ha C u Ru C 2 u, C, C 2 are consans,.4.24 3 2 C 2 γ K, 4 γ2 C 2 γ K K =. Then 3 and 4 2. Indeed, V u + V + u C γ u 2 [ 2 + C 2 γ K ], Remark.4.4. 4 γ K =. I ε C ε 2 {γ2 C 2 γ K K } =. In [3] he convergence z P -a.s., as was proved under he following condiions: A R =, Ruu < for all u ; M here exiss a non-negaive predicable process r = r inegrable w.r. process K = K on any finie inerval [, ] wih properies: a r K =, b A = γ 2 ε r K m <, c all jumps of process A are bounded, d r u 2 + γ 2 K R 2 u 2γ Ruu, for all u R and [,. Show ha M B, and 2. I is eviden ha b B. Furher, d, Finally, 2 follows from a and d hanks o he relaion I ε := inf V u + V + u K ε 2 r K =. ε u ε The implicaion is proved. In paricular case when.4.24 holds and for all γ K q, q > is a consan and C and C 2 in.4.24 are chosen such ha 2C qc2 2 >, if
SEMIMARTINGALE STOCHASTIC APPROXIMATION PROCEDURE 25 we ake r = bγ, b >, wih b < 2C qc 2 2, hen a and d are saisfied if γ K =. Bu hese condiions imply 3 and 4. In fac, on he one hand, < 2C qc 2 2 C 2 qc 2 and so 3 follows, since 2 C 2 γ K 2 qc 2 >. On he oher hand, 4 follows from γ2 C 2 γ K K 2 qc 2 γ K =. From he above we may conclude ha if he condiions A, B,.4.24, γ K q, q >, 2 qc 2 > and γ K = are saisfied, hen he desirable convergence z P -a.s. akes place and so, he choosing of process r = r wih properies M is unnecessary cf. [3], Remark.2.3 and Subsecion.3. c Linear Model see, e.g., [28]. Consider he linear RM procedure where b B,, m M 2 loc. Assume ha dz = bγ z dk + γ dm, z F, γ 2 m <,.4.25 γ K =,.4.26 γ 2 K K d <. Then for each b B he condiions A, B and I are saisfied. Hence z P -a.s., as..4.27 Now le.4.25 and.4.26 be saisfied, bu P γ 2 K K d = >. A he same ime assume ha B = [b, b 2 ], < b b 2 < and for all > γ K < b. Then for each b B.4.27 holds. Indeed, [V ui { K } + V + u] + = b γ u 2 [ 2 + b γ K I { K }] + and herefore II i is saisfied. On he oher hand, I { K } b γ u 2 [ 2 + b γ K ] + = inf u 2 {2γ b I { K } + bγ[2 b γ K]I { K } } K ε u ε ε 2 b γ[2 b γ K] K ε 2 b γ K =. So II ii is saisfied also.
26 N. LAZRIEVA, T. SHARIA AND T. TORONJADZE 2. Rae of Convergence and Asympoic Expansion 2.. Noaion and preliminaries. We consider he RM ype sochasic differenial equaion SDE z = z + H s z s dk s + Mds, z s. 2.. As usual, we assume ha here exiss a unique srong soluion z = z of Eq. 2.. on he whole ime inerval [, [ and M = M M 2 loc P, where M = Mds, z s see [8], [9], [3]. Le us denoe H u β = lim u u assuming ha his limi exiss and is finie for each and define he random field { Hu if u, β u = u β if u =. I follows from A ha for all and u R, Furher, rewrie Eq. 2.. as z = z + β and β u P -a.s.. β s z s I {βs K s }dk s + β s β s z s z s dk s + we suppose ha M,. Denoe R 2 = In his noaion, β = β I {β K }, R Mds, s = s β s β s z s z s dk s, R 3 = z = z β s z s dk s + z s I {βs K s=} Mds, z s Mds, z s I {βs K s=}, Mds, z s Mds,. Mds, + R, where R = R + R 2 + R 3.
SEMIMARTINGALE STOCHASTIC APPROXIMATION PROCEDURE 27 Solving his equaion w.r. z yields where Here, α K = z = Γ z + Γ s Mds, + Γ = ε β K. α s dk s and ε A is he Dolean exponen. Γ s dr s, 2..2 The process Γ = Γ is predicable bu no posiive in general and herefore, he process L = L defined by L = Γ s Mds, belongs o he class M 2 loc P. I follows from Eq. 2..2 ha where χ z = R = z L /2 L + R L /2, χ = Γ L /2, + L /2 Γ s dr s and L is he shifed square characerisic of L, i.e. L := + L F,P. This secion is organized as follows. In subsecion 2.2 assuming z as P -a.s., we give various sufficien condiions o ensure he convergence γ δ z 2 as P -a.s. 2..3 for all δ δ, where γ = γ is a predicable increasing process and δ, δ, is some consan. There we also give series of examples illusraing hese resuls. In subsecion 2.3 assuming ha Eq. 2..3 holds wih γ asympoically equivalen o χ 2 see he definiion in subsecion 2.2, we sudy sufficien condiions o ensure he convergence R P as, which implies he local asympoic lineariy of he soluion. We say ha he process ξ = ξ has some propery evenually if for every ω in a se Ω of P probabiliy, he rajecory ξ ω of he process has his propery on he se [ ω, for some ω <. Everywhere in his secion we assume ha z as P -a.s..
28 N. LAZRIEVA, T. SHARIA AND T. TORONJADZE 2.2. Rae of convergence. Throughou subsecion 2.2 we assume ha γ = γ is a predicable increasing process such ha P -a.s. γ =, γ =. Suppose also ha for each u R he processes Mu and γ are locally absoluely coninuous w.r.. he process K and denoe h u, v = d Mu, Mv dk and g = dγ dk assuming for simpliciy ha g > and hence, I { K } = I { γ } P -a.s. for all >. In his subsecion, we sudy he problem of he convergence γ δ z as P -a.s. for all δ, < δ < δ /2, < δ. I should be sressed ha he consideraion of he wo conrol parameers δ and δ subsanially simplifies applicaion of he resuls and also clarifies heir relaion wih he classical ones see Examples and 6. We shall consider wo approaches o his problem. The firs approach is based on he resuls on he convergence ses of non-negaive semimaringales and on he so-called non-sandard represenaions. The second approach presened explois he sochasic version of he Kronecker Lemma. This approach is employed in [39] for he discree ime case under he assumpion 2.2.23. The comparison of he resuls obained in his secion wih hose obained before is also presened. Noe also ha he wo approaches give differen ses of condiions in general. This fac is illusraed by he various examples. Le us formulae some auxiliary resuls based on he convergence ses. Suppose ha r = r is a non-negaive predicable process such ha r K <, r K < P -a.s. for each > and r K = P -a.s.. Denoe by ε = ε r K he Dolean exponenial, i.e. ε = e r sdks c r s K s. Then, as i is well known see [25], [28], he process ε is he soluion of he linear SDE and ε ε as P -a.s.. s = ε r dk, ε = = {ε r K}
SEMIMARTINGALE STOCHASTIC APPROXIMATION PROCEDURE 29 Proposiion 2.2.. Suppose ha ε ε [r 2β z + β 2 z K ] + dk < P -a.s. 2.2. and ε h z, z dk < P -a.s., 2.2.2 where [x] + denoes he posiive par of x. Then ε z 2 P -a.s. he noaion X means ha X = X has a finie limi as. Proof. Using he Io formula, where dε z 2 = z dε 2 + ε dz 2 = ε z r 2 2β z + β 2 z K dk + ε h z, z dk + dmar = ε z db 2 + da da 2 + dmar, db = ε ε [ r 2β z + β 2 z K ] + dk, da = ε h z, z dk, da 2 = ε ε [ r 2β z + β 2 z K ] dk. Now, applying Corollary..4 o he non-negaive semimaringale ε z 2, we obain {B < } {A < } {ε z 2 } {A 2 < } and he resul follows from Eqs. 2.2. and 2.2.2. The following lemma is an immediae consequence of he Io formula applying o he process γ δ, < δ <. Lemma 2.2.. Suppose ha < δ <. Then where γ δ = ε r δ K, r δ = r δ g /γ and r δ = δi { γ =} + γ /γ δ γ /γ I { γ }. The following heorem is he main resul based on he firs approach.
3 N. LAZRIEVA, T. SHARIA AND T. TORONJADZE Theorem 2.2.. Suppose ha for each δ, < δ < δ, < δ, and γ γ δ [r δ 2β z + β 2 z K ] + dk < P -a.s. 2.2.3 γ δ h z, z dk < P -a.s.. 2.2.4 Then γ δ z 2 as P -a.s. for each δ, < δ < δ, < δ. Proof. I follows from Proposiion 2.2., Lemma 2.2. and he condiions 2.2.3 and 2.2.4 ha P {γ δ z 2 } = for all δ, < δ < δ, < δ. Now he resul follows since {γ δ z 2 for all δ, < δ < δ } {γ δ z 2 for all δ, < δ < δ }. Remark 2.2.. Noe ha if Eq. 2.2.3 holds for δ = δ, han i holds for all δ δ. Some simple condiions ensuring Eq. 2.2.3 are given in he following corollaries. Corollary 2.2.. Suppose ha he process γ γ is evenually bounded. 2.2.5 Then for each δ, < δ < δ, < δ, {[ δi { γ=} + I { γ } g {[ δ + δ γ { γ γ ] + γ 2βz + β 2 z K K < } ] + g γ γ 2βz + β 2 z K K < } δ [r δ 2βz + β 2 z K] + K < }. Proof. The proof immediaely follows from he following simple inequaliies x δ δx + δx 2 x if < x < and < δ <, which aking x = γ /γ gives r δ δ + δ γ δi { γ =} + I { γ }. γ I remains only o apply he condiion 2.2.5.
SEMIMARTINGALE STOCHASTIC APPROXIMATION PROCEDURE 3 In he nex corollary we will need he following group of condiions: For δ, < δ < δ /2, [ δ g γ βz ] + K c < P -a.s., 2.2.6 [ β z K [ β z K Corollary 2.2.2. Suppose ha he process γ ] δ + I {βz K } < P -a.s., 2.2.7 γ γ ] δ + I {βz K } < P -a.s.. 2.2.8 γ β z K is evenually bounded. 2.2.9 Then if Eq. 2.2.5 holds, {2.2.6, 2.2.7, 2.2.8 for all δ, < δ < δ /2} {2.2.3 for all δ, < δ < δ }; 2 if, in addiion, he process ξ = ξ, wih ξ = sup s γ s /γ s is evenually <, hen he reverse implicaion holds in ; 3 {2.2.6, 2.2.7, 2.2.8 for δ = δ /2} {2.2.6, 2.2.7, 2.2.8 for all δ, < δ < δ /2} here δ is some fixed consan wih < δ. Proof. By he simple calculaions, for all δ, < δ < δ, < δ, [ δ δi { γ=} + γ /γ δ γ g γ 2β z + β 2 z K ] + dk = γ γ /γ I { γ } γ [ δ g ] + 2β z dk c γ δ β z K γ /γ δ/2 + γ [ β z K + γ /γ δ/2] + I{β z K } δ γ β z K + γ /γ δ/2 + γ [ β z K γ /γ δ/2] + I{β z K }. 2.2. Now for he validiy if implicaions and 2 i is enough o show ha under condiions 2.2.5 and 2.2.9, he processes βz K + γ/γ δ/2 I {βz K } and βz K + γ/γ δ/2 I {βz K }
32 N. LAZRIEVA, T. SHARIA AND T. TORONJADZE are evenually bounded and, moreover, if ξ < evenually, hese processes are bounded from below by a sricly posiive random consan. Indeed, for each < δ < and, if β z K, sup s and, if β z K, sup s γ s γ s β z K + γ /γ δ/2 2 2.2. γ s γ s β z K + γ /γ δ/2 β z K. 2.2.2 The implicaion 3 simply follows from he inequaliy x δ x /2 if < x < and < δ < /2. The following resul is an immediae consequence of Corollary 2.2.2. Corollary 2.2.3. Suppose ha I {β z K } < and and 2 γ < P -a.s.. 2.2.3 Then Eq. 2.2.7 is equivalen o [ δ γ ] + β z dγ d < P -a.s. 2.2.4 γ γ {2.2.6, 2.2.4 for all δ, δ δ /2} {2.2.3 for all δ, < δ < δ }. Proof. The condiions 2.2.8 and 2.2.9 are auomaically saisfied and also ξ < evenually ξ = ξ is he process wih ξ = sup γ s /γ s. So i s follows from Corollary 2.2.2 2 ha {2.2.6, 2.2.7 for all δ, < δ < δ /2} {2.2.3 for all δ, < δ < δ }. I remains o prove ha Eq. 2.2.7 is equivalen o Eq. 2.2.4. This immediaely follows from he inequaliies [a + b] + [a] + + [b] +, δx x δ δx + δx 2, γ < x <, < δ <, applying o he x = γ s /γ s and o he expression [ β z K + γ /γ δ] +, and from he condiion γ /γ 2 < P -a.s.. Remark 2.2.2. The condiion 2.2.4 can be wrien as [ δ γ ] + β z K < P -a.s.. γ
SEMIMARTINGALE STOCHASTIC APPROXIMATION PROCEDURE 33 Below using he sochasic version of Kronecker Lemma, we give an alernaive group of condiions o ensure he convergence γ δ z as P -a.s. for all < δ < δ /2, < δ. Rewrie Eq. 2.. in he following form z = z + z s db s + G, where and db = β z dk, β u = β ui {βu K } G = z s I {β z K =} + Mds, z s. 2.2.5 s Since B = β z K we can represen z as z = ε B z + ε s B dg s and muliplying his equaion by γ δ yields γ δ z = sign ε BΓ δ z + sign ε s B{Γ δ s } γs δ dg s, 2.2.6 where Γ δ = γ δ ε B. Definiion 2.2.. We say ha predicable processes ξ = ξ and η = η are equivalen as and wrie ξ η if here exiss a process ζ = ζ such ha ξ = ζ η, and < ζ < ζ < ζ 2 < evenually, for some random consans ζ and ζ 2. The proof of he following resul is based on he sochasic version of he Kronecker Lemma. Proposiion 2.2.2. Suppose ha for all δ, < δ < δ /2, < δ, here exiss a posiive and decreasing predicable process Γ δ = Γ δ such ha and Γ δ = P -a.s., P { lim Γ δ = } =, Γ δ Γ δ
34 N. LAZRIEVA, T. SHARIA AND T. TORONJADZE 2 I {β z K =} < P -a.s., 2.2.7 γ 2δ h z, z dk < P -a.s.. 2.2.8 Then γ δ z as P -a.s. for all < δ < δ /2, < δ. Proof. Recall he sochasic version of Kronecker Lemma see, e.g., [25], Ch. 2, Secion 6: Kronecker Lemma. Suppose ha X = X is s semimaringale and L = L is s predicable increasing process. Then { } X {L = } {Y } L P -a.s., where Y = + L X. Pu + L = Γ δ and X = Γ δ s sign ε s BγsdG δ s. Then i follows from he condiion ha L is an increasing process wih L = P -a.s. and { } A = {Γ δ = } Γ δ s Γ δ s sign ε s BγsdG δ s { Γ δ Γ δ } Γ s δ sign ε s BγsdG δ s {γ δ z }, where he laer inequaliy follows from he relaion Γ δ Γ δ and Eq. 2.2.6. A he same ime, from Eq. 2.2.5 and from he well-known fac ha if M M 2 loc, hen { M < } {M } see, e.g., [25], we have } { } {Γ δ = } { I {β z K =} < The resul now follows from Eqs. 2.2.7 and 2.2.8. γ 2δ h z, z dk < A. Now we esablish some simple resuls which are useful for verifying he condiion of Proposiion 2.2.2. By he definiion of ε B, ε B = e Bc + B s s
and since we obain SEMIMARTINGALE STOCHASTIC APPROXIMATION PROCEDURE 35 Γ δ where D = /γ and γ δ = exp = exp δ B c + δ = exp C δ = dγ c s γ s s dγ c s γ s D s dc δ s {βs z s γ s Using he formula of inegraion by pars and he relaion we ge from Eq. 2.2.9 ha Therefore, where Γ δ = exp Γ δ g s log γ δ s γ s + log + B s s γ s δ γ s, 2.2.9 } δ I { γs =} γ s log + B s γ s γ s δ I { γs } dγ s. 2.2.2 γ s dd C = D dc + C dd d = dγ γ γ γ = exp Cδ C δ dγ s s. γ γ s γ s [ C δ s γ s ] + dγ s γ s Γ δ = ζ Γ δ, 2.2.2, ζ = exp Cδ + γ [ C δ s γ s ] + dγ s The following proposiion is an immediae consequence of Eq. 2.2.2. Proposiion 2.2.3. Suppose ha for each δ, < δ < δ /2, < δ, he following condiions hold: γ s.
36 N. LAZRIEVA, T. SHARIA AND T. TORONJADZE a There exis random consans Cδ and Cδ such ha < Cδ < Cδ γ evenually, where C δ /γ = C δ /γ. [ δ ] C dγ b < P -a.s.. γ γ c [ δ ] + C dγ γ γ = P -a.s.. Then Γ δ Γ δ for each δ, < δ < δ /2. Corollary 2.2.4. Suppose ha < Cδ /2 γ < C γ < Cδ < < C < evenually, where C is some random consan and he processes C δ /2 and C are defined in Eq. 2.2.2 for δ = δ /2 and δ =, respecively. Then Γ δ Γ δ for each δ, < δ < δ /2, < δ. This resul follows since, as i is easy o check, C δ /2 < C δ < C and C δ C δ /2 for each δ, < δ < δ /2, which gives δ 2 δ < Cδ γ < C δ 2 δ γ and [ ] C δ + > δ [ ] C δ γ 2 δ and = γ evenually. We shall now formulae he main resul of his approach which is an immediae consequence of Proposiions 2.2.2 and 2.2.3. Theorem 2.2.2. Suppose ha he condiions 2.2.7, 2.2.8 and he condiions of Proposiion 2.2.3 hold for all δ, < δ < δ /2, < δ. Then P -a.s., γ δ z as for all δ, < δ < δ /2, < δ. Consider in more deail wo cases: all he processes under he consideraion are coninuous; 2 he discree ime case. In addiion assume ha M, u = M for all u R,.
SEMIMARTINGALE STOCHASTIC APPROXIMATION PROCEDURE 37 In he case of coninuous processes condiions 2.2.7 and 2.2.8 are saisfied rivially, he condiion 2.2.6 akes he form [ δ γ ] + β z dγ < P -a.s. 2.2.22 g γ and also {2.2.22 for δ = δ /2} {2.2.22 for all δ, < δ < δ /2}. Furher, since C δ γ = γ β s z s γ s g s dγ s δ δ, he condiions a c of Proposiion 2.2.3 can be simplified o: a The process β s z s γ s dγ s γ g s is evenually bounded. [ b β s z s γ s g s c bc γ [ γ β s z s γ s g s ] dγ dγ s δ γ ] + dγ dγ s δ γ Also, if a holds and C δ /2 β s z s γ s = dγ s δ γ g s 2 γ < P -a.s.. = P -a.s.. >, evenually, hen b and c hold for each δ, < δ < δ /2. In he discree ime case we assume addiionally ha 2 γ < and β z 2 < P -a.s.. 2.2.23 γ Then he condiions of Corollary 2.2.3 are rivially saisfied. Hence, he condiions 2.2.3 and 2.2.4 are equivalen and can be wrien as [ δ γ ] + β z γ < P -a.s. 2.2.24 g γ and also, {2.2.24 for δ = δ /2} {2.2.24 for all δ, < δ < δ /2}.
38 N. LAZRIEVA, T. SHARIA AND T. TORONJADZE Noe ha he reverse implicaion does no hold in general see Example 3. I is no difficul o verify ha a, b and c are equivalen o ã, b and c defined as follows. ã The process β s z s γ s γ is bounded evenually. [ ] γ b β s z s γ s δ < P -a.s.. γ γ s< [ ] + γ c β s z s γ s δ = P -a.s.. γ γ s< s Also if ã holds and bc β s z s γ s δ γ s > δ /2 evenually, hen b and c hold for each δ, < δ < δ /2. Hence {ã, bc} {ã, b, c for all δ, < δ < δ /2}. However, he inverse implicaion is no rue see Examples 3 and 4. Noe ha he condiions imposed on he maringale par of Eq. 2.. in Theorems 2.2. see Eq. 2.2.4 and 2.2.2 see Eq. 2.2.8 are idenical. We, herefore, assume ha hese condiions hold in all examples given below. Example. This example illusraes ha Eq. 2.2.22 holds whereas a is violaed. Le K = γ = + and β u + /2+α, where < α < /2. Subsiuing K, γ, β in he lef-hand side of Eq. 2.2.22 we ge [ δ + /2 α + ] + d + = [ δ + /2 α ] + d +. Since [δ + /2 α ] + = evenually, he condiion 2.2.22 holds. The condiions a does no hold since γ β s z s γ s g s dγ s = + Noe ha he condiions b and c are saisfied. s + /2 α ds + /2 α as.
SEMIMARTINGALE STOCHASTIC APPROXIMATION PROCEDURE 39 I should be poined ou ha alhough Eq. 2.2.22 holds for all δ, δ >, if, e.g., d d M = +, 3/2+α he condiions 2.2.4 only holds for δ s saisfying < δ < δ = /2 + α. Example 2. In his example he condiions ã and bc hold for δ = while Eq. 2.2.24 fails for some δ, < δ < /2 = δ /2. Consider a discree ime model wih K = γ =, β u β and { /2 + a if is odd, β γ = /2 b oherwise, where < b < /2 a. Then, since 2 + a > γ s β s γ s = 2 + { a b 2 if = 2k, k =, 2,... ka b+a 2k+ if = 2k +, k =, 2,... > 2, he condiions ã and bc hold for δ =. I is easy o verify ha if /2 b < δ < /2, hen [δ β γ ] + = [ δ ] + 2 + b I { is even} = implying ha Eq. 2.2.24 does no hold for all δ wih /2 < b < δ < /2. Example 3. In his discree ime example δ = and {2.2.24 for all δ, < δ < /2} {2.2.24 for δ = /2}. Suppose ha K = γ =, β u β and [ ] + β γ = 2. log + Then for < δ < /2 and large s, and i follows ha [δ β γ ] + = [δ β γ ] + <. Bu for δ = /2, [ ] + 2 β γ Noe also ha by he Toepliz Lemma, β s γ s = s s log + I {log+>} =. [ 2 logs + ] + 2 as.