Department of Computer Science, University of Dortmund, D Dortmund / Germany

Transcription

1 THE FUNDMENTL MTRI OF THE GENERL RNDOM WLK WITH SORING OUNDRIES GUNTER RUDOLPH bstract. The general random walk on the nonnegative integers with absorbing boundaries at and n has the transition probabilities pj j, p nj nj, i;, i+ q i, and i r i, where + r i + q i. The fundamental matrix of this Markov chain is the inverse of matrix (I ; Q) where Q results from P by deleting the rows and columns and n. Entry b ij represents the expected number of occurrences of the transient state j prior to absorption if the random walk starts at state i. The absorption time as well as the absorption probabilities are easily derived once the fundamental matrix is known. Here, it is shown that the fundamental matrix can be determined in elementary manner via the adjugate of matrix (I ; Q). Key words. General random walk, fundamental matrix, absorption times, absorption probability MS subject classications. 6J5, 6J2. Introduction. Random walk models have surfaced in various disciplines. They served as initial simple models in biology (especially in genetics) and physics, but they are also useful tools in analyzing sequential test procedures in statistics or randomized algorithms in computer science to name only few elds of application. Needless to say, many results have been published for specic instantiations of the transition probabilities the general case, however, seems to be explored with less intensity. For example, El-Shehawey [] has determined the joint probability generating function of the number of occurrencies of the transient states. Its marginals may be used to derive the fundamental matrix but the expression oered in [] contains unresolved recurrence relations that make potential further calculations dicult. Therefore, this work aims at a `closed form' expression for each entry of the fundamental matrix. It will be shown that such a result can be achieved via elementary matrix theory. 2. General Random Walk with bsorbing oundaries. t rst, some notation being adopted from Minc [2] isintroduced. Next, the Markov chain model of the random walk is presented along with some basic results from Markov chain theory taken from Iosifescu [3]. Finally, the fundamental matrix of the Markov chain as well as expressions for the absorption time and absorption probabilities are determined. 2.. Notation. Let be an m n matrix. Then ( ::: h j ::: k )denotes the (m;h)(n;k) submatrix of obtained from by deleting rows ::: h and columns ::: k whereas [ ::: h j ::: k ] denotes the h k submatrix of whose (i j) entry is a i j.if i i for i ::: k then the shorthand notation ( ::: k ) resp. [ ::: k ] will be used. s usual, ; is the inverse and det is the determinant of a regular square matrix. Matrix I is the unit matrix and every entry of column vector e is Markov hain Model. The general random walk with absorbing boundaries is a time-homogeneous Markov chain ( k : k ) with state space f ::: ng Department of omputer Science, University of Dortmund, D-4422 Dortmund / Germany (rudolphls.cs.uni-dortmund.de). This work was supported by the Deutsche Forschungsgemeinschaft (DFG) as part of the ollaborative Research enter \omputational Intelligence" (SF 53).

2 2 G. RUDOLPH and transition matrix P p r q p 2 r 2 q p n;2 r n;2 q n;2 p n; r n; q n; such that Pf k+ j j k ig j for i j ::: n. Let Q P ( n), i.e., Q results from P by deleting the rows and columns and n, andset I ; Q. Then ; is the fundamental matrix associated with the transition matrix P. Let T minfk : k 2f ngg. Then E[ T j i ]a i denotes the absorption time for a random walk starting at state i where a i is the ith entry of vector a e. Thus, a i is just the sum of all entries of row i of the fundamental matrix. In case of the random walk, the absorption probabilities are Pf T j ig b i p and Pf T n j ig b i n; p n; n for i ::: n; Determination of the Fundamental Matrix. There are many methods to obtain the inverse of some regular square matrix. Here, the inverse of matrix (I;Q) is determined via its adjugate. This approach is especially useful if only few elements of the inverse are of interest. Let : d d be a regular square matrix. The adjugate adj() ofmatrix is the matrix whose (i j) entry is (;) j+i det (jji). Since ; adj() det() one obtains i+j det (jji) b ij (;) det for i j ::: d.to proceed one needs an elementary expression for the determinant of matrix. Lemma 2.. Let P be the transition matrix of the general random walk with absorbing boundaries at state and state n. Let Q P ( n) and set d I ; Q with d n ;. The determinant of d is given by for all d. det d d i d j+ Proof. (by induction) Let d. Then matrix d reduces to (p + q ) with det d p + q.since ;k i j2;k p + q the hypothesis is true for d. Now let d 2. The determinant of matrix 2 is det 2 det p + q ;q p ;p 2 p 2 + p 2 + p q 2 + q q 2 : q 2

3 Since 2 2;k i GENERL RNDOM WLK 3 2 j3;k p p 2 + p q 2 + q q 2 the hypothesis is true for d 2aswell. Suppose that the hypothesis is true for d ; and d for d 2. The determinant of matrix d+ can be expressed in terms of det d and det d; via det d+ det det d; d ;q d ;p d+ p d+ + q d+ ;q d; ;p d p d + q d ;q d ;p d+ p d+ + q d+ (p d+ + q d+ ) det d + p d+ det d;. ;p d ;q d (2.) y hypothesis, one obtains (2.2) (2.3) (p d+ + q d+ ) det d ; p d+ q d det d; q d+ det d + p d+ (det d ; q d det d;) : q d+ det d q d+ d d+ k d d i j+ d+ i d+;k i j+ d+ j(d+);k+ where eqn. (2.3) results from an index shift in eqn. (2.2). The same arguments yield and hence (2.4) q d det d; d k i p d+ (det d ; q d det d;) d j+ d+ i :

4 4 G. RUDOLPH Insertion of eqns. (2.3) and (2.4) into eqn. (2.) leads to det d+ d+ k d+;k i d+ (d+);k i d+ j(d+);k+ d+ j(d+);k+ d+ + which is the desired result. Next, one needs an elementary expression for the determinant of(jji). The rst sten this direction is similar to the approach inminc[2, pp. 47{49] who considered the more general case of establishing a general expression for a submatrix of a tridiagonal matrix. Here, the situation is less complicated. Since submatrix (jji) results from the tridiagonal matrix after the deletion of row j and column i, the submatrix is in lower triangular block formifi<j, in diagonal block form if i j, and in upper triangular block form if i>j.eachof these \blocks" is a square submatrix of. Notice that the determinant ofsuch block matrices is the product of the determinants of the diagonal blocks. s a consequence, one obtains det (jji) det([ ::: i; ]) det([i : : : j ; j i + ::: j]) det([j + ::: d]) if i<j d n ;, if i j d, and i det (jji) det([ ::: i; ]) det([j + ::: d]) det (jji) det([ ::: j; ]) det([j + ::: ij j ::: i; ]) det([i + ::: d]) if j<id. s a convention, if u>vthen det([u ::: v]). The nal step towards an elementary expression of det (jji) requires the determination of the determinants of the diagonal block matrices. n elementary expression for the matrices of the type [ ::: `]canbetaken directly from Lemma 2.. Since the structure of the matrices of the type [` + ::: d] is identical to the structure of the matrices of the type [ ::: d; `], Lemma 2. also leads to an elementary expression for the determinants of these matrices one must only take into account that the indices have the oset `. onsequently, one obtains d;` d det [` + ::: d] : u`+ v+ If i<j d then matrix [i : : : j ; j i + ::: j] reduces to a lower triangular matrix. Similarly, if j<i d then matrix [j + ::: ij j : : : i ; ] is upper triangular. It follows that and det [i : : : j ; j i + ::: j](;) j;i det [j + ::: ij j ::: i; ] (;) i;j j; ki i kj+ q k ( i<j d) p k ( j<i d):

5 b ij GENERL RNDOM WLK 5 onsequently, it has been proven: Theorem 2.2. Let :(n; ) (n ; ) be the fundamental matrix of the general random walk with absorbing boundaries at states and n. The entries b ij of matrix are " i; i;k; " j; n;k; u i; vi;k # n; n;k; u ki # " n;j; q k n; uj+ n; # for i j n ; and b ij " j; j;k; u j; vj;k # " i n; p k # "n;i; kj+ n;k; n; u n;k; ui+ n; # for n ; i>j. Thanks to Theorem 2.2 one obtains the absorption time and probability via E[ T j i ] n; j b ij resp. Pf T n j ig b i n; q n; where i ::: n;. s expected, these expressions reduce to well-known formulas if p and q i q for i ::: n;. If p q then the limit operation (qp) is necessary. 3. onclusions. losed form expressions for the entries of the fundamental matrix of the general random walk with absorbing boundaries have been derived by means of elementary matrix theory. This leads also to closed form expressions for the absorption time and the absorption probabilities. The approach taken here is especially useful if, for example, the absorption time or the absorption probabilities for a specic initial state are of interest because only few entries of the fundamental matrix must be determined in this case. REFERENES [] M.. El-Shehawey. On the frequency count for a random walk with absorbing boundaries: a carcinogenesis example. I. J. Phys., 27:735{746, 994. [2] H. Minc. Nonnegative Matrices. Wiley, New ork, 988. [3] M. Iosifescu. Finite Markov Processes and Their pplications. Wiley, hichester, 98.