FAST EXACT AFFINE PROJECTION ALGORITHM USING DISPLACEMENT STRUCTURE THEORY. Manolis C. Tsakiris and Patrick A. Naylor

Transcription

1 FAST EXACT AFFINE PROJECTION ALGORITHM USING DISPLACEMENT STRUCTURE THEORY Manolis C Tsakiris and Patrick A Naylor Dept of Electrical and Electronic Engineering, Imperial College London Communications and Signal Processing Group manolistsakiris8, pnaylor@imperialacuk ABSTRACT This paper exploits the displacement structure of the coefficient matrix of the linear system of equations pertinent to the Affine Projection Algorithm (APA), to obtain the exact solution in a way faster than any other existing exact method The main emphasis of the paper is to present the concepts of displacement structure theory and how these are applied to the APA context Index Terms Adaptive Filters, Affine Projection Algorithm, Displacement Structure Theory, Choleski Factor 1 INTRODUCTION The affine projection algorithm (APA) 1 was proposed in order to improve the convergence speed of the Normalized-Least-Mean- Squares (NLMS) algorithm for colored input signals Part of the computational complexity of APA comes from the requirement to solve a linear system of equations (LSOE) with coefficient matrix being a sample covariance matrix of the input signal The cost of solving this LSOE determines the overall cost of APA in applications where the adaptive filter length is not much larger than the projection order, such as in adaptive beamforming 2 In applications where this is not true, such as in echo cancellation 3, approximate fast versions of APA (FAPs) can be used, where the critical issue is again to solve a LSOE with the same coefficient matrix as in the exact APA Over the past few years, many efforts have been made, especially in the context of FAPs, in order to solve this LSOE in a fast and reliable way The majority of the proposed methods, eg 3, 4, 5, 6 and 7, achieve relatively low complexity, albeit at the disadvantage of leading to approximate solutions, while few only studies have been concerned with obtaining the exact solution, eg 8 and 9 In this study, the special structure of the sample covariance matrix is fully exploited by invoking the theory of displacement structure and the concept of displacement rank, in order to solve the LSOE in a fast and exact manner A numerical linear algebra algorithm for time-variant structured matrices 1 is applied into the APA context resulting in the fastest existing exact implementation of APA to the best knowledge of the authors This is done by propagating from iteration to iteration the Choleski factor of the sample covariance matrix and solving two triangular LSOE Although the presentation is done in the context of the exact APA, the core of the proposed algorithm can equally well be applied to the context of FAPs The notation used is standard with vectors and matrices denoted with boldface lower and upper case respectively and with the indexing of the columns and rows of a matrix starting from except if explicitely otherwise stated 11 The Affine Projection Algorithm In the standard system identification scenario, the APA can be described as follows Consider a collection of scalar measurements {d(i)} that arise from the model d(i) u iw o + v(i), i (1) where w o is an M 1 column vector representing the unknown finite impulse response of the system to be identified, u i is the 1 M regressor vector that captures the input data u i u(i),u(i 1),,u(i M + 1) (2) and v(i) accounts for the measurement noise At time i the APA delivers an estimate w i of the unknown w o via where b i is the solution of with U i w i w i 1 + µu b i (3) R ib i e i (4) R i U iu + ɛ I K (5) u T i, u T i 1,, u T i K+1 T (6) being the K M regressors matrix, e i the vector estimation error e i d i U iw i 1 (7) and d i d(i),d(i 1),,d(i K + 1) T the K 1 measurements vector The parameter K denotes the APA projection order, while µ is a step-size that controls convergence, ɛ is a small positive regularization parameter that enforces the positive-definiteness of R i and I K is the K K identity matrix 12 Fast Affine Projection Algorithms The fast affine projection algorithms (FAPs) are approximations of the standard APA and are well suited for echo cancellation, where usually M K An approximation adopted in this context is that e i has the following structure: e(i) e i (8) (1 µ)e i /9/$25 29 IEEE DSP 29 Authorized licensed use limited to: Imperial College London Downloaded on January 4, 21 at 8:23 from IEEE Xplore Restrictions apply

2 where e i 1 denotes the upper K 1 elements of e i 1 3 Another key approximation of FAPs is that their recursions employ an alternative weight vector, whose computation is much faster than for the weight vector of the standard APA In order for this weight vector to be updated from iteration to iteration the solution of a LSOE of the form (4) is required, with the right side now satisfying property (8) 13 Background in Solving Linear Systems of Equations AstandardapproachtowardssolvingaLSOERb e, R being K K symmetric positive-definite, is via the Choleski decomposition R LL,whereL is the unique lower triangular factor of R with positive diagonal elemens The cost is O( K3 3 )m1 (12, p144) If R is additionally Toeplitz, then the solution can be obtained via the Levisnon algorithm at O(4K 2 ) m (12, p197) Alternatively, a sequence of approximate solutions can be found usually at a smaller cost via the so-called iterative methods, typical examples of which are the Conjugate-Gradient and the Gauss-Seidel methods Existing Approaches Towards Solving the APA and FAP Linear System of Equations Returning to the LSOE (4) that arises in the APA and FAP contexts (with the approximation (8) for the latter), it is observed that the coefficient matrix R i is symmetric positive-definite and consequently the standard method to solve it is via Choleski decomposition, which as mentioned in subsection 13, requires O( K3 ) m 3 Several methods have been proposed in order to obtain the exact solution or an approximate one faster than O( K3 ) m Although this 3 work is concerned with obtaining the exact solution, it is significant to first briefly refer to the approximate methods To begin with, in 3 the inverse coefficient matrix is estimated using a sliding window fast RLS and an approximate solution is obtained at O(2N) m In4R i is assumed to be Toeplitz and solution via the Levinson algorithm is implied (O(4K 2 ) m) In 5 and 6 an approximate solution is iteratively obtained via the conjugategradient algorithm in O(2K 2 ) m and via the Gauss-Seidel algorithm in O(N 2 /p) m (p is an integer) respectively, after some significant simplification taking place by setting µ 1in (8) Finally, in 9 two approximate methods are proposed: the first assumes that R i can be regarded as constant over K sampling intervals and the second assumes as 4 that R i is Toeplitz Less work has been done in obtaining the exact solution of (4) A method that most efficiently exploits so far in the literature the structure of R i is 8, where the exact solution is obtained at O(4K 2 ) m by using the matrix-inversion-lemma in a clever way An exact approach that targets better robustness than 8 is also proposed in 9, whose cost is however proportional to K 3 Noothersignificantcontribution has come to the attention of the authors, as far as obtaining the exact solution of (4) is concerned 15 The Contribution Displacement structure theory 13 from numerical linear algebra can be applied in a novel way to fully exploit the structure of R i and solve the LSOE at O(3K 2 ) m,alowercomputationalcomplexity than that of 8 An indirect reference to the low displacement 1 In this paper the computational complexity of an operation is measured in terms of the order of required multiplications (m) rank of R i has been made in 3, where it was in passing mentioned that the so-called Generalized Levinson algorithm 11 can be used to solve (4) at O(7K 2 ) m In fact, a numerically better algorithm than the generalized Levinson, the so-called Generalized Schur algorithm 14 can be used to solve the LSOE at O(7K 2 ) m,although in the very recent 7 direct inversion requiring O(K 3 ) m is referred to In this work it is attempted to fill the gap between the APA literature and the numerical linear algebra literature, thus opening new perspectives for even faster and more reliable APA implementations This is done by observing at the first place that the displacement rank of the matrix R i is constant in time and equal to 2, regardless of the stationarity or non-stationarity of the input process In the sequel, it is shown how the Choleski factor of R i can be efficiently propagated from iteration to iteration by applying the results of 1 into the APA context The result is a clearly motivated and presented implementation of the standard regularized APA, which requires only O(3K 2 ) m to obtain the exact solution of (4) Note that this algorithm is even faster than solving the system via the Levinson algorithm, under the assumption that R i is Toeplitz This is not surprising since 1) according to the development in 11 the distance of R i from a Toeplitz matrix of the same size is zero and 2) the proposed algorithm does not compute the Choleski factor of R i directly, albeit indirectly by updating the Choleski factor of R i 1 and thus using only the minimum amount of computation 2 DISPLACEMENT STRUCTURE Let R i be a time-varying K K positive-definite matrix with lowertriangular Choleski decomposition R i L il (9) where L i is the unique lower-triangular Choleski factor of R i with positive diagonal elements, and define the matrix ZR i as ZR i R i ZR i 1Z (1) where Z is some sparse lower-triangular displacement matrix If rank( ZR i)r(i) <K, then R i is said to have displacement structure with respect to the displacement defined by Z In this work it is assumed that r(i) r for every i Since ZR i is Hermitian, its eigenvalues are all real and it is assumed that it has p positive and q negative eigenvalues with p+q r Moreover,considertheeigen-decomposition ZR i G iλ ig (11) where G i contains in its columns the eigenvectors of ZR i and Λ i is a diagonal matrix with the corresponding eigenvalues in its diagonal elements Since the eigen-decomposition (11) is not unique, the eigenvectors of ZR i can be ordered in such a way, so that the first p columns of G i contain the eigenvectors which correspond to the positive eigenvalues of ZR i, the next q columns contain the eigenvectors that correspond to the negative eigenvalues, and the remaining (K r) columns contain the eignevectors which correspond to the zero eigenvalues of ZR ibyexpandingitsrightside(11)becomes ZR i K 1 k λ i,k g i,k g,k (12) where g i,k is the k th column of G i Given the ordering of the eigenvectors in the columns of G i, the last (K r) terms of the sum in Authorized licensed use limited to: Imperial College London Downloaded on January 4, 21 at 8:23 from IEEE Xplore Restrictions apply

3 the right side of (12) will be zero and consequently r 1 ZR i λ i,k g i,k g,k (13) k By defining the scaled eigenvectors as (13) can be rewritten as g i,k λ i,k g i,k, k, 1,,r 1 (14) p 1 r 1 ZR i g i,k g,k g i,k g,k (15) k kp which can be expressed more compactly in matrix form as ZR i G ijg R i ZR i 1Z (16) where G i is the K r generator matrix defined as G i g i, g i,p 1 g i,p g i,r 1 and J is the r r signature matrix defined as 2 J I p ( I q) I p p q q p I q (17) (19) where denotes the direct sum operator Equation (16) will be refered to as the displacement equation 3 CHOLESKI FACTOR PROPAGATION The displacement equation (16) has an extremely important implication: the Choleski factor L i 1 of R i 1 can be updated to the Choleski factor L i of R i and most importantly this can be done at O(rK 2 ) m 1 In the rest of this section the general theory of how to obtain L i from L i 1 is presented Towards this end, a key lemma is invoked, known as Hyperbolic Basis Rotation Lemma: Hyperbolic Basis Rotation Lemma Consider two n m (n m) matrices A and B IfAĴA BĴB s of full rank, for some m m signature matrix Ĵ I p ( I q), p + q m, then there exists an m m Ĵ-unitary matrix H (HĴH Ĵ) such that A BH Aproofcanbefoundat1,whileamoreelegantproofcanbe found at (15, p68) Moreover, it is noted that the transformation H is highly non-unique since it can be shown that any other tranformation HC, where C is Ĵ-unitary, has the same effect, ie that of mapping B to A Now, from (16) R i ZR i 1Z + G ijg (2) which can be rewritten as Li K r I K K r I K K r L r K L i 1 Z 2 If A is p p and B is q q, then A B A diag {A, B} p q q p B G (21) (18) Since the left side of equation (21) equals to R i, which being positive-definite is also full-rank, equation (21) fits exactly to the statement of the Hyperbolic Basis Rotation Lemma with A L i K r (22) B ZL i 1 G i (23) and I Ĵ (I K J) K K r (24) Consequently, there exists a (I K J)-unitary matrix H i such that Hi L i K r (25) Equation (25) clearly reveals that knowlegde of L i 1 and G i is sufficient for the computation of L iitremainstocarefullydesignthe transformation H i which will yield L i Towards this end, note first that H i should result in a zero rightmost K r block when applied to the matrix ZL i 1 G i Moreover, H i should be designed so as to yield a lower-triangular leftmost K K block with positive diagonal elements It is shown in the sequel that if H i satisfies these two properties, then the resulting leftmost K K block is necessarily L i Schematically and assuming for simplicity K 3and r 2, H i must be designed so as to map ZL i 1 G i to a matrix of the following form: indefinite positive positive indefinite indefinite positive In other words, Hi X i K r (26) (27) where X i is lower-triangular with positive diagonal elements Now, taking the squared (I K J)-norm 3 of both sides of equation (27) leads to Hi I K K r Xi K r I K K r H L i 1 i G X r K which in view of the (I K J)-unitarity of H i 4 becomes which can be rewritten as (28) ZL i 1L 1Z + G ijg X ix (29) ZR i 1Z + G ijg X ix (3) By combining equations (3) and (2) the result is X ix R i (31) which is the lower-triangular Choleski decomposition of R i, since X i is by design lower-triangular with positive diagonal elements By the uniqueness of the Choleski decomposition it is concluded that X i L i (32) In the next section the theory of this section is applied in the APA context 3 The squared Ĵ-norm of a column-vector x is the sign-indefinite quantity x 2 x Ĵx Ĵ 4 I H K K r i H i I K K r Authorized licensed use limited to: Imperial College London Downloaded on January 4, 21 at 8:23 from IEEE Xplore Restrictions apply

4 4 ALGORITHM DEVELOPMENT Returning to the APA context, consider the R i matrix of equation (5) with its Choleski decomposition given by equation (9) Then by using the lower-shift matrix 1 (K 1) Z 1 1 (33) I K 1 (K 1) 1 it is seen that multiplication of R i 1 from the left by Z and from the right by Z amounts to shifting it downward along the main diagonal by one position while setting the first column and the first row equal to zero Consequently, the displacement equation becomes R i ZR i 1Z u iu + ɛ u iu 1 u iu K+1 u i 1u u i 1u 1 + ɛ u i 1u K+1 u i K+1u u i K+1u 1 u i K+1u K+1 + ɛ u i 1u 1 + ɛ u i 1u K+1 u i 2u 1 u i 2u K+1 u i K+1u 1 u i K+1u K+1 + ɛ u iu + ɛ u iu 1 u iu K+1 u i 1u u i K+1u (34) with the right side being a rank-2 matrix that can be factored as in (16) with u i 2 + ɛ u i 1 u u i 1 u G i u i 2 u u i 2 u (35) and J 1 1 (36) Now assume that the Choleski factor of R i 1, ie L i 1 is available According to section 3, in order to obtain L i, the matrix ZLi 1 } {{ } K K G i }{{} K 2 (37) must be (I K J)-transformed, with J now explicitely given by equation (36), to a matrix of the form X }{{} i }{{} (38) K K K 2 where X i is lower-triangular with positive diagonal entries Then X i will be the Choleski factor of R i, ie L i Details are now given on how to design an appropriate transformation H i, which performs the mapping (37) H i (38) Towards this end, consider H i as a sequence of K elementary (I K J)-unitary transformations {H i,j} K 1 j successively applied to the matrix (37), ie H i H i,h i,1 H i,k 1 Denote by l i,j the non-zero part of the j th column of L i and expand the Choleski decomposition of R i as follows R i L il l i,l, + K 1 j1 j 1 l i,j 1 j l,j (39) It is important to note that the j th term of the above sum is a K K matrix with its first j columns and rows equal to zero Now, set in (37) ZL i 1 X i, and G i G i, and consider the matrix B i, X i, G i, (4) which eventually must be transformed into the form (38) Observe that the first row of X i, is equal to zero and moreover the second entry of the first row of G i, is also zero, while its first entry is positive Consequently, H i, can be selected as I K K 2 H i, P 2 K Q K P K (41) i, where Q i, I 2 and P K denotes the orthogonal permutation matrix which permutes columns and K In this way, B i,1 B i,h i, B i,p K x 1 (K 1) 1 2 i, X i,1 G i,1 (42) where it is also mentioned for future reference that the first row of X i,1 equals to zero, since X i,1 is the lower-right (K 1) (K 1) block of the matrix ZL i 1 Now, by taking the squared (I K J)- norm of both sides of equation (42), noting that B i,h i, 2 (I K J) R i and invoking equation (39), the result is l i,l, + K 1 j1 x i,x 1 (K 1) i, + from which it is evident that j 1 l i,j 1 j l,j X i,1 (K 1) 1 X, J G 2 1 G,1 i,1 + (43) x i, l i, (44) and hence the first column of L i has been found Now, again from (43) it is seen that R i l i,l i, (K 1) (K 1) 1 X i,1x,1 + G i,1jg (45) i,1 If L i is partitioned in the following block form L i l 1 (K 1) i, L i,1 it is readily found that R i l i,l, L il l i,l, (K 1) (K 1) 1 L i,1l,1 Combining equations (45) and (47) it is seen that (46) (47) L i,1l,1 X i,1x,1 + G i,1jg,1 (48) Authorized licensed use limited to: Imperial College London Downloaded on January 4, 21 at 8:23 from IEEE Xplore Restrictions apply

5 H i,1 can now be determined Denote the first row of G i,1 by g i,1 v i,1 v i,1 (49) where v i,1 and v i,1 are scalars Since the first row of X i,1 is zero, it is infered from equation (48) that g i,1 Jg,1 > (5) since g i,1 Jg,1 equals to the squared magnitude of the upper-left 1 1 entry of L i,1, which is seen from equation (46) to be equal to the second diagonal element of L i, which by definition is positive Moreover, it is deduced from inequality (5) that v i,1 > v i,1 (51) It can easily be checked that the matrix 5 1 v Q i,1 i,1 v i,1 v i,1 2 v i,1 2 vi,1 v i,1 (52) is J-unitary and also that vi,1 v i,1 Qi,1 g i,1 Jg,1 (53) The previous analysis suggests that H i,1 can be designed as I K K 2 H i,1 P 2 K Q 1 K (54) i,1 where P 1 K permutes columns 1 and K Asaresult, B i,2 B i,1h i,1 x (K 2) 2 2 i, x i,1 X i,2 G i,2 (55) where the first row of X i,2 is equal to zero, since X i,2 consists of the (K 2) rightmost columns of X i,1 By taking the squared (I K J)-norm of both sides it is verified by using similar arguments as before that x i,1 equals to l i,1, which is the non-zero part of the second column of L i By proceeding in a similar fashion for j 2, 3,, (K 1) it can be shown that B i,k B i,h i,h i,1 H i,k 1 L i K 2 (56) 5 THE PROPOSED ALGORITHM The proposed algorithm of this paper can now be stated: Displacement-APA (DAPA) Select a filter order M,a projection order K, a positive regularization parameter ɛ, a positive step-size µ, set w 1 M 1, L 1 ɛ I K, G 1 K 2, J diag {1, 1} and iterate for i : 1 Compute 1 and set u i 2 + ɛ G i u i 1 u u i 2 u u i 1 u u i 2 u (57) 5 The matrix (52) represents an elementary hyperbolic rotation and is a generalization of the unitary Givens elementary rotation matrix 2 Using Z of equation (33) form the matrix 3 Iterate for j, 1, (K 1): (a) Form the scalars B i, ZL i 1 G i (58) v i,j B i,j(j, K) (59) v i,j B i,j(j, K + 1) (6) where B i,j(j, K) and B i,j(j, K +1)denote the (j, K) and (j, K +1)respectively entries of B i,j (b) Form the 2 2 J-unitary transformation Q i,j 1 v i,j 2 v i,j 2 v i,j v i,j vi,j (61) v i,j (c) Form the (I K J)-unitary (K + 2) (K + 2) transformation I K K 2 H i,j P 2 K Q j K i,j where P j K is the permutation matrix that permutes columns j and K (d) Apply H i,j to B i,j and obtain B i,j+1 B i,j+1 B i,jh i,j (62) 4 Obtain the Choleski factor of (U iu + ɛ I K) as the leftmost K K block of B i,k L i B i,k( : K 1, :K 1) (63) 5 Solve the two triangular systems L ic i e i and L b i c i 6 Update the weight vector w i w i 1 + µu b i (64) Note that always Q i, I 2 since the second entry of the first row of the generator matrix G i is always equal to zero This is implicitely stated in the algorithm formulation since v i, 6 COMPUTATIONAL COMPLEXITY In this section it is shown that the proposed algorithm requires O(3K 2 ) multiplications in order to compute the vector b i To begin with, the computationally intensive operations performed by the proposed algorithm towards computing L i are the elementary hyperbolic rotations (EHRs) of the rows of {G i,j} K 1 j, where G i,j B i,j(j : K 1,K : K + 1) (65) Each EHR is performed via a vector-matrix multiplication of size (1 2)(2 2) and hence requires 4 multiplications Now, at each iteration j the aforementioned EHR is performed (K j) times The total number of EHRs performed is therefore K 1 j1 (K j) where it has been taken into consideration that no EHRs take place for j ( EHRs performed being O K 2 2 ) This leads to the order of total and consequently, the order of total required multiplications is O(2K 2 ) Now, as can be seen from step 5 of the proposed algorithm, L i is used to form two triangular systems of equations These can be solved directly by forward and backward substitution at O( K2 ) m 2 each The cost required for solving these two systems is therefore O(K 2 ) multiplications Authorized licensed use limited to: Imperial College London Downloaded on January 4, 21 at 8:23 from IEEE Xplore Restrictions apply

6 db db MSD (a) Displacement APA (DAPA) standard APA using MATLAB solver iteration index k(i) (b) error convergence to limit of numerical precision iteration index Fig 1 Comparison between a standard APA implementation using the MATLAB solver for the LSOE and the proposed implementation using the propagated Choleski decomposition (DAPA) Plot (a) shows the MSD of the two algorithms and plot (b) shows the Euclidean norm of the difference of the two computed solutions of the LSOE of each iteration 7 SIMULATIONS In this section, DAPA is compared to a standard APA implementation, which solves the LSOE (4) using the MATLAB solver linsolve(r i,e i,opts), where the fields SYM and POSDEF of the structure opts have been set to true, while the others are set to false The two implementations are compared in a system identification scenario, where w o is randomly generated and of unit norm The input signal is zero-mean, unit-variance white noise filtered through the system H(z) (1 9z 1 ) 1 The measurement noise is such so that the SNR at the output of the unknown system is 3 db The algorithmic parameters are set to the standard values M 16, K 8, µ 1and ɛ 1 5 The results are averaged over 1 independent trials The Mean Square Deviation (MSD) for the two implementations is depicted at the top of figure 1, from which it is clear that they coincide The fact that the two implementations are theoretically and practically equivalent is further verified by the extremely small values of the quantity k(i) b i,(mat LAB SOLV ER) b i,dap A 2 (66) which is depicted at plot (b) of Figure 1, showing that k(i) within the numerical limits of the computations 8 CONCLUSIONS The displacement structure theory for time-variant matrices has been applied to the APA context resulting in the fastest existing exact APA implementation to the best knowledge of the authors The technique employed to solve the involved linear system of equations can also be used to derive a FAP algorithm 9 REFERENCES 1 K Ozeki and T Umeda, An adaptive filtering algorithm using an orthogonal projection to an affine subspace and its properties, Electron Commun Jpn, 1984,vol67 A,no5,pp YR Zheng and RA Goubran, Adaptive beamforming using affine projection algorithms, in Proc IEEE Int Conf Signal Process,August2,vol3,pp Steven L Gay and Sanjeev Tavanthia, The fast affine projection algorithm, in Proc IEEE Int Conf Acoust, Speech, Signal Process (ICASSP),1995,vol5,pp S Oh, D Linebarger, B Priest and B Raghothaman, A fast affine projection algorithm for an acoustic echo cancellation using a fixed-point DSP processor, in Proc IEEE ICASSP, 1997, vol 5, pp H Ding, A stable fast affine projection adaptation algorithm suitable for low-cost processors, in Proc IEEE ICASSP,2, Instabul, Turkey, pp I F Albu, J Kadlec, N Coleman and A Fagan, The Gauss- Seidel fast affine projection algorithm, In Proc IEEE Signal Process Systems Workshop, October22,SanDiegeo,CA, pp Yuriy V Zakharov, Low complexity implementation of the affine projection algorithm, IEEE signal process letters, pp 1 4, April 28 8 Q G Liu, B Champagne and K C Ho, On the use of a modified fast affine projection algorithm in subbands for acoustic echo cancellation, in Proc IEEE Digital Signal Process Workshop,1996,pp Heping Ding, Fast affine projection adaptation algorithms with stable and robust symmetric linear system solvers, IEEE trans on signal process, vol55,no5,pp ,May 27 1 Ali H Sayed, Hanoch Lev-Ari and Thomas Kailath, Timevariant displacement structure and triangular arrays, IEEE Trans on Signal Process, vol 42, NO 5 issue 8, pp , May Ph Delsarte, Y Genin and Y Kamp, On the mathematical foundations of the generalized Levinson algorithm, in Proc IEEE ICASSP,May1982,vol7,pp G H Golub and C F Van Loan, Matrix Computations, The John Hopkins University Press, Baltimore, MD, third edition, Thomas Kailath and Ali H Sayed, Displacement Structure: Theory and Applications, SIAM review, vol37,no3,pp , September T Kailath and A H Sayed, Fast Reliable Algorithms for Matrices with Structure, SIAM, Ali H Sayed, Adaptive Filters, WileyInter-Science,28 Acknowledgment The authors are thankful to Charalampos Nakos at the National Technical University of Athens for his comments on the manuscript Authorized licensed use limited to: Imperial College London Downloaded on January 4, 21 at 8:23 from IEEE Xplore Restrictions apply