An O(N log N) fast direct solver for partial Hierarchically Semi-Separable matrices With application to radial basis function interpolation

Transcription

1 An O() fast drect solver for partal Herarchcally Sem-Separable matrces Wth applcaton to radal bass functon nterpolaton Svaram Ambkasaran Erc Darve Receved: date / Accepted: date Ths artcle descrbes a fast drect solver (e, not teratve) for partal herarchcally semseparable systems Ths solver requres a storage of O() and has a computatonal complexty of O() arthmetc operatons The numercal benchmarks presented llustrate the method n the context of nterpolaton usng radal bass functons The key ngredents behnd ths fast solver are recurson, effcent low-rank factorzaton usng Chebyshev nterpolaton, and the Sherman-Morrson-Woodbury formula The algorthm and the analyss are worked out n detal The performance of the algorthm s llustrated for a varety of radal bass functons and target accuraces Keywords Fast drect solver Numercal lnear algebra Partal herarchcally sem-separable representaton Herarchcal matrx Radal bass functon 1 Background and motvaton Large dense lnear systems arsng n engneerng applcatons are often solved by teratve technques Most teratve solvers based on Krylov subspace methods such as Arnold teraton, Conjugate Gradent 41, GMRES 53, MINRES 50, Bconjugate Gradent Stablzed Method 59, QMR 4, TFQMR 3, and others, rely on matrx-vector products The number of teratons requred to acheve a target accuracy s very problem dependent In many nstances, the large condton number of the matrx or dstrbuton of the egenvalues of the matrx n the complex plane (eg, wdely spread egenvalues) result n a large number of teratons Consequently, precondtoners need to be devsed to mprove the numercal propertes of the matrx and accelerate convergence of the teratve solver Once such a method s n place, the cost of performng the matrx-vector products can be reduced usng fast summaton technques lke the fast multpole method (FMM) 6, 16, 48, the Barnes-Hut algorthm 3, panel clusterng 39, FFT, wavelet based methods, and others Of these dfferent fast summaton technques, the FMM has often been used n the context of lnear systems arsng out of boundary ntegral equatons Ths s because many kernel functons resultng from such ntegral equatons are amenable to the FMM The lterature on the FMM s vast and we refer the readers to some semnal publcatons and our prevous work 5, 14, 17, 18,, 30, 31, 65 These fast teratve solvers accelerate the soluton procedure and are able to solve (wth some prescrbed error tolerance ε) a system n lnear or almost lnear tme In ths paper however we wll focus on drect solvers, and before proceedng further, we would lke to compare some of the features of drect solvers over ther teratve counterparts: One of the most mportant advantages of a drect solver s that t scales well wth multple rght hand sdes Any drect solver nvolves two phases: the factorzaton phase and the solvng phase The factorzaton phase s ndependent of the rght hand sde and s computatonally more expensve than the solvng phase Hence, the key ngredent n fast drect solvers s to accelerate the factorzaton phase Once an effcent factorzaton s obtaned, all rght hand sdes can be solved wth relatvely low computatonal cost Iteratve solvers, on the other hand, do not modfy the matrx and rely solely on matrx vector products and other basc algebra operatons Hence, n most cases for an teratve solver, the entre procedure needs to be restarted for each rght-hand sde (although ths process can be optmzed) The authors would lke to thank the Army Hgh Performance Computng Research Center (AHPCRC) and The Global Clmate and Energy Project (GCEP) at Stanford for supportng the project Insttute for Computatonal and Mathematcal Engneerng, Stanford Unversty Department of Mechancal Engneerng, Stanford Unversty

2 For teratve solvers to be effcent, choosng a good precondtoner 7, 13, 35, 58 s mperatve, but n some cases fndng a good precondtoner s a dffcult task Many lnear systems do not have known good precondtoners, leadng to expensve lnear solves To overcome the dsadvantages of teratve solvers and to take advantage of the desrable features of drect solvers, there has been an ncreasng focus on fast drect solvers over the last decade Such solvers are concerned wth dense matrces that have sub-blocks that can be well-approxmated by low-rank matrces Of such class of matrces, the matrces termed herarchcal matrces, denoted as H-matrces and whch were ntroduced by Hackbusch et al 9, 9, n the late 1990 s, are of partcular nterest The basc dea s to sub-dvde a gven matrx nto a herarchy of rectangular blocks and approxmate them by low-rank matrces Due to ths specal structure, these dense systems are often descrbed as data sparse Ths s because these matrces can be reconstructed approxmately by consderng only a subset of ther entres Matrces that are well approxmated by ths procedure occur especally n the context of boundary ntegral equatons, nterpolaton, nverse problems, and others Ths class of matrces s very broad and ncludes for example FMM matrces A partcular class of H-matrces s the class of herarchcally sem-separable matrces (HSS) These matrces were ntroduced by Chandrasekaran et al 11, 1, 64 as a generalzaton of semseparable matrces A sem-separable matrx 57 of separablty rank p s a matrx that can be wrtten as a sum of the lower trangular part of a rank-p matrx and the upper trangular part of another rank-p matrx We refer the readers to 11, 1, 64 for more detals regardng the HSS representaton In the last few years, Greengard 3, Rokhln 4, Martnsson 44, 45, Lexng Yng 55, 56 et al have proposed varous fast drect solvers makng use of the underlyng herarchcal low-rank structure The common theme behnd all these algorthms s to construct a low-rank approxmaton for certan dense sub-blocks and perform a fast update to the soluton n a recursve manner In ths context, the present work dscusses an O() solver for herarchcal off-dagonal low-rank systems (HODLR) and O() solver for the class of partal herarchcally sem-separable systems (p-hss, a subset of HODLR matrces) The p-hss structure s dscussed n detal n secton 33 Lnear systems arsng from the dscretzaton of boundary ntegral equatons of potental theory 4, 45 n two dmensons, nterpolaton by radal bass functon along a curve 6, 35, and others, can be effcently modeled by p- HSS matrces Many papers have proposed fast algorthms for matrces that are smlar to p-hss matrces (some havng slghtly more restrctve assumptons as we wll see later), n partcular Gllman et al 6, Martnsson 44, Martnsson and Rokhln 45, and Kong et al 4 One of the advantages of ths new method s ts relatve smplcty, resultng n a low computatonal cost We show that the computatonal cost of ths method scales as O(p ) for partal herarchcally sem-separable matrces, where the rank of nteracton between the clusters s p For problems arsng out of one-dmensonal manfolds, p can be shown to be ndependent of N In other stuatons, for example when consderng D manfolds whch are cases not covered n 6, 4, 44, 45 the method n ts current form stll apples but no longer scales as O() For example, as presented n ths paper, the algorthm scales lke O(N ) for matrces arsng n 3D boundary element methods Ths results from the fact that the rank of nteracton between clusters, as dscussed n ths method, wll scale as O(N 1/ ) Overvew of method and relaton to prevous work The current work dscusses a fast drect solver for a partal herarchcally sem-separable matrx The partal herarchcally sem-separable representaton s dscussed n detal n secton 33 As wth most of the other fast drect solvers, the current solver reles on a fast low-rank factorzaton of the off-dagonal blocks of the matrx to get t nto p-hss form Once we have the p-hss representaton of the matrx, the solver reles on the Sherman-Morrson-Woodbury update to solve the lnear system 7, 40, 61 The total cost to construct the factorzaton and to solve the lnear system s O() If the matrx s exactly p-hss, the factorzaton s exact In most cases however, the p-hss matrx s only an approxmaton (wth an error controllable by choosng approprate ranks n the approxmaton), n whch case the soluton produced by the fast drect solver s only approxmate To llustrate the performance and accuracy of ths algorthm, we solve a lnear nterpolaton problem along a one dmensonal manfold usng radal bass functons n secton 5 We now dscuss the algorthm presented n ths paper n reference to exstng ones We wll restrct ourselves to exstng methods that also take advantage of the low-rank off-dagonal blocks The work by Chandrasekaran et al 1 constructs a O(N) solver for HSS systems It constructs a ULV H decomposton (U and V are untary matrces, and L s lower trangular, H s the transpose conjugate operator) of a herarchcally semseparable matrx Ther approach dffers from ours n many ways The startng pont of ther algorthm s to recognze that when we have a low-rank approxmaton of the form UBV H (U and V are thn matrces wth p columns) t s possble to apply a untary transformaton so that only the last p rows of U are non-zero Ths result s then used to reduce the sze of A recursvely ( bottom-up approach) untl we are left wth a

3 small enough lnear system that can be solved by a conventonal method They descrbe an O(N ) algorthm to construct the HSS representaton and an O(N) algorthm when the matrx s assocated wth a smooth kernel For the latter, Chebyshev polynomals are used to construct low-rank approxmatons, whch s smlar to our approach and also Fong et al The solver descrbed n the present manuscrpt constructs a dfferent one-sded factorzaton of the matrx Further, ths artcle works wth a p-hss matrx, whch s a superset of HSS matrces, as wll be explaned n detal n secton 33 The work by Rokhln and Martnsson 45 constructs an O(N) fast drect solver for boundary ntegral equatons n two-dmensons makng use of off-dagonal low-rank blocks The algorthm constructs the nverse usng a compressed block factorzaton that takes advantage of the low-rank off-dagonal blocks to factor the matrx A two-sded herarchcal factorzaton of the nverse s constructed The approach follows some of the deas n 1, and s based on applyng transformatons to low-rank matrces such that only p non-zero rows (resp columns) reman whle other rows (resp columns) are set to 0 Ths allows compressng the matrx and progressvely reducng ts sze The solver proposed n the present work, on the other hand, s conceptually smple to understand and easer to mplement Further, ths paper presents a factorzaton of the matrx (followed by a solve) whereas 45 bulds a compressed factorzaton of the nverse matrx A recently publshed work, whle ths manuscrpt was under preparaton, by Kong et al 4 proposes an O() solver for boundary ntegral equatons n two-dmensons takng advantage of off-dagonal low-rank blocks Though ther algorthm s smlar to parts of our algorthm, eg, ther algorthm also uses Sherman- Morrson-Woodbury updates, we proceed further and reduce the computatonal complexty of the algorthm to O() wth the addtonal assumpton of p-hss structure Our approach hghlghts the bottleneck n the O() algorthm, enablng us to reduce the computatonal complexty to O() In fact, our O() algorthm can be vewed as an extenson of the algorthm proposed by Kong et al 4 by makng the addtonal assumpton of p-hss structure and thereby reducng the cost from O() to O() In our benchmarks, the O() resulted n a speed-up of nearly 4 compared to the O(), even for moderately large N The speed-up n general depends on N and wll mprove even more for larger N The strategy s presented n such a way that t clearly ndcates how the structure of the matrx dctates the cost, e, the algorthm explctly reveals how the assumpton of the p-hss structure cuts down the cost from O() to O() In a smlar sprt, ths algorthm can be extended to an HSS structure to yeld an O(N) algorthm, whch wll be the subject of future work The structure of the paper s as follows The next secton dscusses some of the key deas ncludng the dfferent herarchcal representatons, low-rank approxmatons and Sherman-Morrson-Woodbury formula that are the prmary ngredents for the algorthm Secton 4 motvates and provdes an overvew of the algorthm, and dscusses the computatonal complexty of the algorthm Secton 5 dscusses the applcablty, performance and accuracy of ths solver by provdng numercal benchmarks for an nterpolaton problem on an one-dmensonal manfold usng radal bass functons The fnal secton concludes the paper by hghlghtng the capabltes of ths solver 3 Prelmnary deas In ths secton, we dscuss the key deas that wll be of help n understandng the algorthm 31 Sherman-Morrson-Woodbury formula The key ngredent n our algorthm s the Sherman-Morrson-Woodbury formula 40, 61, whch provdes a convenent way to update the soluton of a lnear system perturbed by a low-rank update Consder solvng a system of the form ( I + UV T ) x b (1) where I R N N s the dentty matrx, U, V R N p, x, b R N r, and p r N The Sherman-Morrson-Woodbury formula gves us x b U ( I + V T U ) 1 V T b () The computatonal cost for solvng (1) usng () s O(prN) We refer the readers to 40, 61 for the dervaton of the above result and the computatonal cost, though the drect proof takes only a few steps: ( I + UV T ) x ( I + UV T ) b ( I + UV T ) U ( I + V T U ) 1 V T b Eq () ( I + UV T ) b U ( I + V T U ) ( I + V T U ) 1 V T b Refactor the second term ( I + UV T ) b UV T b b

4 3 Fast low-rank factorzaton Gven a matrx A R M N, the optmal rank p approxmaton can be obtaned from ts sngular value decomposton 7 However, sngular value decomposton s computatonally expensve wth a cost of O(MN mn(m, N)) In recent years, there s an ncreasng focus 15, 5, 33, 47 on constructng fast approxmate low-rank factorzatons for matrces Technques lke adaptve cross approxmaton 5 (ACA), pseudo-skeletal approxmatons, 8 nterpolatory decomposton, randomzed algorthms 5, 43, 6, rank-revealng LU 47, 51 and QR 33 algorthms provde great ways for constructng effcent approxmate low-rank representatons The proposed algorthm for solvng the lnear system s ndependent of the algorthm to construct the low-rank factorzatons and therefore can be combned wth any low-rank factorzaton technque In the numercal llustratons dscussed n secton 5, we consder lnear systems arsng from nterpolaton schemes based on radal bass functons such as quadrcs (r +a ), nverse mult-quadrc (1/ r + a ), Gaussan (exp( r /a )), exponental (exp( r/a)), and others These radal bass functons are smooth and non-sngular For smooth kernels, nterpolaton usng Chebyshev polynomals s an attractve method to construct low-rank factorzatons, 46 Although any nterpolaton scheme can be used to construct a low-rank factorzaton, n the present work, the Chebyshev polynomals wll serve as the nterpolaton bass along wth ther roots as the nterpolaton nodes 31 Low-rank usng nterpolaton Consder a functon g(x) on the closed nterval 1, 1 A p-pont nterpolant, P p 1 (x), that approxmates g(x) can be wrtten as P p 1 (x) p g( x k )w k (x) (3) k1 where x k s are the p nterpolaton nodes and w k (x) s the nterpolatng functon correspondng to the node x k A low-rank approxmaton for the kernel K(x, y) s constructed as a p-pont nterpolant as shown below K(x, y) p w k (x)k( x k, y) (4) k1 Note that ths p-pont nterpolant s obtaned by havng p nterpolaton nodes for x only (we could nterpolate along y as well by havng nterpolaton nodes along y) Ths gves us a low-rank representaton for the kernel K(x, y) Although any nterpolaton scheme can be used to construct a low-rank approxmaton as descrbed above, the Chebyshev polynomals wll serve as the nterpolaton bass and ther roots wll serve as the nterpolaton nodes We wll brefly recall some propertes of the Chebyshev polynomals before proceedng further The Chebyshev polynomal of the frst knd of degree p, denoted by T p (x), s defned as T p (x) cos(p arccos(x)) (5) The doman of T p (x) s the closed nterval 1, 1 T p (x) has p roots located at ( ) (k 1)π x k cos, k {1,,, p} (6) p The set of roots { x k } are called Chebyshev nodes Usng the Chebyshev nodes of T p (x) as the nterpolaton nodes, the low-rank approxmaton to K(x, y) can be wrtten as p 1 K p 1 (x, y) T k (x)c k (y) (7) k0 where p p l1 c k (y) K( x l, y)t k ( x l ) f k > 0, 1 p p l1 K( x l, y) f k 0

5 By rearrangng terms, we get p K p 1 (x, y) U(x, x k )K( x k, y) (8) k1 where p 1 U(x, x k ) 1 p + T l (x)t l ( x k ) (9) p Equaton (8) gves us the desred low-rank representaton for K(x, y) If we let K mat R M N be the matrx K mat (, k) K(x, y k ), then equaton (8) can be wrtten as l1 K mat U mat V T mat (10) where U mat R M p, V mat R N p wth U mat (, j) U(x, x j ) and V mat (j, k) K( x j, y k ) U mat s termed the anterpolaton matrx snce ths anterpolates the nformaton from the cluster contanng the x s to the nterpolaton nodes of the cluster The matrx V mat computes the kernel nteracton between the nterpolaton nodes of the cluster contanng the x s and the y s 33 Herarchcal representatons We brefly dscuss a couple of herarchcal representatons, whch we wll be dealng wth n ths artcle 331 Herarchcal off-dagonal low-rank matrx A matrx, K R N N, s termed a -level herarchcal off-dagonal low-rank matrx, denoted as HODLR, f t can be wrtten n the form shown n equaton (11) K K (1) 1 U (1) 1 V (1)T 1, U (1) V (1)T,1 K (1) K () 1 1 V ()T 1, V ()T,1 K () U (1) V (1)T,1 U (1) 1 V (1)T 1, K () 3 3 V ()T 3,4 4 V ()T 4,3 K () 4 where K () R N/4 N/4, U (k) 1, U (k), V (k) 1,, V (k), 1 RN/k p and p N In general, a κ-level HODLR matrx s the one n whch, the th dagonal block at level k, where 1 k and 0 k < κ, denoted as K (k), can be wrtten as where K (k) K (k) U (k+1) K (k+1) 1 V (k+1)t, 1 U (k+1) 1 V (k+1)t 1, K (k+1) R N/k N/ k, U (k) 1, U (k), V (k) 1,, V (k), 1 RN/k p and p N The maxmum number of levels, κ, s log (N/p) The constructon of a κ-level HODLR matrx, usng nterpolaton to obtan low-rank of the off-dagonal blocks, s descrbed below 1 Let the root level (level 0) contan the locaton of all the ponts n the doman (11) (1) For all the clusters at level κ, compute the nteracton of each cluster wth tself, e, K (κ) {1,,, κ } for all 3 At all levels, k {1,,, κ}, for all the clusters, {1,,, k } and j {1,,, k 1 }, compute the nteracton wth ts sblng, usng a low-rank representaton, e, Compute the anterpolaton matrces, U (k), usng p Chebyshev nodes for the cluster at level k Compute the nteracton of the Chebyshev nodes of the cluster wth ts sblng cluster, e, V (k) j 1,j and V (k) (k) j,j 1 Note: V a,b captures the nteracton of the Chebyshev nodes of the cluster a wth ts sblng cluster b at level k Ths gves the desred HODLR representaton O(pN(1 + κ)) The total cost to construct and store a HODLR matrx s

6 33 Partal herarchcally sem-separable matrx The partal herarchcally sem-separable matrces, denoted as p-hss, are a subset of the HODLR matrces and a superset of herarchcally sem-separable (HSS) matrces Some of the notatons we use are smlar to those used by Chandrasekaran et al 11, 1 The proposed O() algorthm s applcable for ths class of matrces The recursve herarchcal structure of the p-hss representaton s seen when we consder the 4 4 block parttonng of a p-hss matrx, K The two-level p-hss representaton s as shown n equaton (13) K K (1) 1 U (1) 1 V (1)T 1, U (1) V (1)T,1 K (1) K () 1 1 V ()T 1, V ()T,1 K () 3 S() 3 4 S() 4 V (1)T,1 where U (k) 1, U (k), V (k) 1,, V (k), 1 RN/k p, S () R p p and p N The key feature s that U (1) 1 s defned n terms of 1 and and 4, e, we have U (1) 1 1 S() 1 S(), U (1) An equvalent statement s that U (1) 1 les n the span of S() 1 S() V (1)T 1, K () 3 3 V ()T 3,4 4 V ()T 4,3 K () 4 (13) (1) ; smlarly, U s defned n terms of 3 3 S() 3 4 S() 4 (14) In a κ-level p-hss representaton, f we denote the th dagonal block at level k as K (k) constructed from U (k+1) 1 and U (k+1), e, we have U (k) U (k+1) 1 S(k+1) 1 U (k+1) S (k+1) (1), then U (k) s (15) The maxmum number of levels, κ, s log (N/p) nterpolaton s dscussed below The constructon of a κ-level p-hss matrx, usng 1 Let the root level (level 0) contan the locaton of all the ponts n the doman For all the clusters at level κ, compute the nteracton of each cluster wth tself, e, K (κ) {1,,, κ } for all 3 For all the clusters at level κ, compute the anterpolaton matrces usng p Chebyshev nodes, e, U (κ) for all {1,,, κ } 4 For all clusters at level k, compute the nteracton of the Chebyshev nodes of the cluster wth the sblng of the cluster usng p Chebyshev nodes, e, V (k) 1,, V (k), 1 where k {1,,, κ} and {1,,, k 1 } 5 For all clusters at level k, compute the anterpolaton matrces from the cluster to ts parent usng p Chebyshev nodes, e, S (k) where k {, 3,, κ} and {1,,, k } Ths s done by computng S (k) (k) (r, s) U ( x r, x (k 1) ) (k) s where x r s the r th Chebyshev node of cluster at level k and x (k 1) s s the s th Chebyshev node at level (k 1) of the parent of the th (k) cluster, and U ( x r, x (k 1) ) s s gven by equaton (9) Ths gves the desred p-hss representaton The total cost to construct and store a p-hss matrx s O (pn(1 + κ)) We refer the readers to 11, 1, 64 for detaled descrpton of herarchcally sem-separable (HSS) matrces These representatons correspond to ncreasngly larger set of matrces, e, HSS p-hss HODLR The O() algorthm s applcable for HODLR matrces whle the O() algorthm s applcable for p-hss matrces The strategy s presented n a way that the O() algorthm s an extenson of the O() algorthm wth the addtonal assumpton of a p-hss structure

7 4 Algorthm In ths secton, we present the O() algorthm for HODLR matrces, and O() algorthm for p-hss matrces As dscussed n the ntroducton, any drect solver nvolves two man steps The frst step s the factorzaton step and the second step s where we use the factorzaton to obtan the fnal soluton The dfference n the computatonal cost between the O() and O() algorthm s n the factorzaton step Once the desred factorzaton has been obtaned, the computatonal cost of applyng the factorzaton to solve the system s O() rrespectve of the factorzaton algorthm For purposes of llustraton and analyss, we assume that the off-dagonal sub-blocks at each level are of the same rank and the system s splt nto two equal halves at each level 41 Factorzaton phase In ths secton, we dscuss the factorzaton of the HODLR and p-hss matrces The overall dea s to factor the underlyng matrx, K R N N, nto κ + 1 block dagonal matrces as n equaton (16): K K κ K κ 1 K κ K 1 K 0 (16) where K k R N N s a block dagonal matrx wth k dagonal blocks each of sze N N and each of the k k dagonal blocks at all levels are low-rank update to the dentty matrx A κ-level HODLR matrx, K (κ) R N N s presented n equaton (17) K (κ) where K (κ) K (κ) 1 U (κ) 1 V (κ)t 1, U (κ) V (κ)t,1 K (κ) U (κ 1) V (κ 1)T,1 R N/κ N/ κ, U (k) j, V (k) 1,, V (k), 1 U (κ 1) 1 V (κ 1)T 1, K (κ) 3 U (κ) 3 V (κ)t 3,4 U (κ) 4 V (κ)t 4,3 K (κ) 4 K (κ) κ U (κ) κ 1 V (κ)t κ 1, κ U (κ) κ V (κ)t κ, κ 1 K (κ) κ (17) RN/k p for k {1,,, κ}, j {1,,, k } and {1,,, k 1 } Recall that a κ-level p-hss matrx not only has the structure descrbed n equaton (17) but also has the addtonal structure mentoned n equaton (15) The frst step n the algorthm s to factor the block dagonal matrx shown n equaton (18) K κ K (κ) K (κ) K (κ) K (κ) 4 K (κ) κ Here s the mportant dfference between the factorzaton of a HODLR and a p-hss matrx For the HODLR structure, when we factor K κ, U (k) j at all levels k, need to be updated However, f we have the p-hss structure, we only need to update U (κ) j, snce the p-hss representaton allows us to obtan U (k) j at all the other levels through the recurrence (15) Another key observaton, vald for both the HODLR and p-hss matrces and that enables us to reduce the computatonal cost, s that we do not need to update V (k) (k) 1, and V, 1 for any level Now to factor out K κ, we need to multply by the nverse of K (κ) the correspondng N/ κ rows of U (k) for all k {1,,, κ} (a subset of the N/ k rows of U (k) ) The multplcaton by the nverse s carred out n practce by solvng the approprate lnear system HODLR case Snce each U (k) has p columns, we need to apply the nverse to a total of κp columns The computatonal cost of applyng the nverse of K (κ) to r columns s O(p 3 + rp ) and hence to factor K κ, the total cost s O(κp N), where κ log (N/p) p-hss case It s suffcent to apply the nverse to only the p columns of U (κ), for the reasons explaned above Hence, the total cost becomes O(p N) (18)

8 Factorng out K κ, we get that K (κ) K κ K (κ 1), where K (κ 1) s of the form n equaton (19) I U (κ,1) 1 V (κ)t 1, U (κ,1) U (κ 1,1) V (κ)t 1 V (κ 1)T 1,,1 I K (κ 1) U (κ 1,1) I U (κ,1) V (κ 1)T 3 V (κ)t 3,4,1 U (κ,1) 4 V (κ)t 4,3 I I U (κ,1) κ 1 V (κ)t κ 1, κ U (κ,1) κ V (κ)t κ, κ 1 I (19) Note that U (k,1) ndcates that U (k) has been updated after factorng out K κ Now that we have K (κ 1), the plan s to repeat ths process as we go up the levels For ease of understandng, let s defne I U (κ,1) 1 V (κ)t 1, U (κ,1) K (κ 1,1) V (κ)t, (0), 1 I where I R N/κ N/ κ s the dentty matrx and K (κ 1,1) R N/κ 1 N/ κ 1 Hence, we have K (κ 1,1) 1 U (κ 1,1) 1 V (κ 1)T 1, K (κ 1) U (κ 1,1) V (κ 1)T,1 K (κ 1,1) K (κ 1,1) κ 1 (1) Note that K (κ 1) s a (κ 1)-level HODLR matrx In addton, f K (κ) had a p-hss structure to begn wth, then K (κ 1) wll also have a p-hss structure, snce U (k,1) s related to U (k+1,1) 1 and U (k+1,1) 1 through (): U (k,1) U (k+1,1) 1 S (k+1) 1 U (k+1,1) S (k+1) Hence, let us factor K (κ 1) as K κ 1 K (κ ), where K κ 1 s a block dagonal matrx wth κ 1 blocks each of sze N/ κ 1 N/ κ 1 as shown n equaton (3), and K (κ ) s a (κ )-level HODLR matrx K κ 1 K (κ 1,1) K (κ 1,1) 0 K (κ 1,1) κ 1 Though the dagonal blocks of K (κ 1) are now twce n sze compared to the dagonal blocks of K (κ), all the dagonal blocks are a low-rank update to an dentty matrx, e, K (κ 1,1) R N/κ 1 N/ κ 1 n equaton (0) can be wrtten as I + Ũ (κ) and Ṽ (κ)t, where Ũ (κ) Ṽ (κ)t U (κ,1) U (κ,1) R (N/κ 1 ) p 0 V (κ)t 1, V (κ)t, 1 0 R (N/κ 1 ) p Let us now calculate the computatonal complexty for the second step n the factorzaton, e, to obtan K (κ 1) K κ 1 K (κ ) To perform ths, we frst need to multply by the nverse of K (κ 1,1) the correspondng rows of U (k,1) for all k {1,,, κ 1} HODLR case Snce K (κ 1,1) the nverse of K (κ 1,1) I +Ũ(κ) Ṽ (κ)t, by Sherman-Morrson-Woodbury formula, the cost to apply to r columns s O(N/ κ 1 (p) r) Hence, the total cost of ths step s O((κ 1)p N) () (3)

9 p-hss case It s suffcent to apply the nverse to just U (κ 1,1) Hence, the total cost s O(p N) Ths s repeated tll we reach level 0 to get a factorzaton of the form K K κ K κ 1 K 0 (4) where K k s a block dagonal matrx wth k dagonal blocks each of sze N/ k N/ k Note that for all k, except κ, all the block dagonal matrces n K k can be wrtten as a rank p update to a ( N/ k N/ k) dentty matrx Hence, the computatonal complexty at level k for factorzng K (k) as K k K (k 1), for a HODLR matrx s O(p Nk) and for a p-hss matrx s O(p N) It s mportant to note that the computatonal cost for the factorzaton at level k depends on k for a HODLR matrx, whereas t s ndependent of k for a p-hss matrx Hence, the computatonal complexty for the factorzaton s: Matrx HODLR p-hss matrx Computatonal complexty κ O(kp N) O(κ p N) O(p ) k1 κ O(p N) O(κp N) O(p ) k1 Fgure 1 provdes a pctoral descrpton of ths factorzaton K K (3) K 3 K K 1 Full rank; Low-rank; Identty matrx; Zero matrx; K 0 Fgure 1: Factorzaton of a three level HODLR/p-HSS matrx 4 Solvng phase The solvng phase s ndependent of the factorzaton phase and s the same for both the algorthms Once we have the factorzaton of the matrx K n the form K κ K κ 1 K κ K 0, then the soluton to Kx b where K R N N, b R N r s gven by x K0 1 K 1 1 Kκ 1 b Hence, we need to frst apply the nverse of K κ followed by K κ 1 all the way up to K 0 All these nverses need to be appled to an N r matrx, where r s the number of rght hand sdes In our mplementaton, the solvng phase and factorzaton phase proceed smultaneously Hence, n the benchmarks presented the factorzaton s not constructed explctly For nstance, n the frst step of the factorzaton phase of both algorthms, when we apply the nverse of K κ to K (κ) to get K (κ 1), we also apply the nverse to the r rght hand sdes Smlarly, at every step k, when we apply the nverse of K k to K (k) to get K (k 1), the nverse s also appled to the correspondng r rght hand sdes Ths s analogous to Gaussan elmnaton where the LU factorzaton and back substtuton proceed together The addtonal cost of applyng these nverses at each step to the r rght hand sdes s O(rpN) Hence, the total cost of the solve phase s O(rpN log(n)) The computatonal complexty for the algorthms dscussed are summarzed n Table 1 Table 1: Computatonal complexty; p: rank; r: number of rght-hand sdes; N: sze of matrx Phase HODLR p-hss Factorzaton O(p ) O(p ) Solvng O(pr) O(pr)

10 5 Numercal benchmark In ths secton, we present results obtaned for our test case The test case consdered s a contour deformaton problem usng radal bass functons 51 Interpolaton usng radal bass functons We brefly dscuss nterpolaton usng radal bass functons The lterature on nterpolaton usng radal bass functon s vast and we refer the reader to a few 8, 10, 0, 54, 60, 63 As wth other nterpolaton technques, the motvaton behnd nterpolaton based on radal bass functons s to approxmate the gven data by a functon defned on a large set, ensurng that the functon passes through the data ponts Let {f k } N k1 be the gven values of the data observed at N dstnct ponts say {x k } N k1 We consder the case when x k s le on a one-dmensonal manfold The motvaton behnd nterpolaton usng radal bass functons s to fnd a smooth functon s(x) such that s(x k ) f k, for all k {1,,, N} To acheve ths, the nterpolant s(x) s consdered to be of the form, N s(x) λ k φ(x x k ) + p(x) k1 where p(x) s a polynomal of degree l, λ k are a set of weghts and φ s a functon from R R We set φ(0) 0 Ths s termed the nugget effect 1, 4, 19, 1, 49 and helps n makng the system reasonably well-condtoned Wthout the nugget effect, some of the systems arsng out of radal bass nterpolaton are sngular or very close to beng sngular and hence for most practcal applcatons, a nugget s always chosen We have that p(x) P l, where P l s the space of polynomals wth degree l Let {p 0 (x), p 1 (x),, p l (x)} be a bass for P l Hence, p(x) can be wrtten as p(x) l a j p j (x) j0 Ths gves us that the nterpolant s(x) must be of the form s(x) N λ k φ(x x k ) + k1 l a j p j (x) (5) The polynomal p(x) s most often taken to be constant (l 0), lnear (l 1) (or) cubc (l 3) Further, n case of nterpolaton by radal bass functons, φ(x y) s a functon of x y In whch case, equaton (5) can be rewrtten as s(x) j0 N λ k φ ( x x k ) + k1 l a j p j (x) (6) To determne the nterpolant, we need to determne the λ k s and the a j s, a total of N + l + 1 unknowns The nterpolant n equaton (6) s requred to satsfy the nterpolaton condtons j0 s(x k ) f k, k {1,,, n} (7) Equaton (7) ensures that the nterpolant passes through the data ponts obtaned through the sde condtons gven n equaton (8) The remanng equatons are N λ k p j (x k ) 0, j {0, 1,, l} (8) k1 The sde condtons n equaton (8) ensure polynomal reproducton, e, f the data arses from a polynomal q(x) P l, then the nterpolant s also q(x) In addton, ths condton results n the fact that away from the nterpolaton ponts x k, the nterpolaton functon wll be approxmated by l j0 a jp j (x) Hence, we now have N + l + 1 equatons and N + l + 1 unknowns to be determned These equatons can be wrtten as a lnear system as shown below Φ P P T 0 λ a where Φ(, j) φ ( x x j ), P (k, j) p j 1 (x k ), λ() λ, a() a 1, f() f f 0 (9)

11 5 Problem specfcaton Intal Boundary 1 Deformed Boundary Fgure : Deformaton of a one dmensonal manfold Gven the mappng of ponts from the boundary of a dsc of unt radus to the boundary of the wggly surface, the goal s to map the nteror of the unt dsc to the nteror of the wggly surface The dsplacement of a set of N ponts on the unt crcle are specfed These N ponts are chosen unformly at random on the unt crcle, e, we sample θ from a unform dstrbuton n the nterval 0, π) The bottleneck n solvng these lnear nterpolaton problems usng radal bass functons s the soluton of the dense lnear systems Φ λ 1 P, Φ λ f (30) where Φ R N N To benchmark our algorthm, we compare the tme taken by dfferent algorthms to solve equaton (30) for a varety of commonly used radal bass functons All the algorthms were mplemented n C++ The tme taken by the O() and O() algorthms to solve a lnear system wth one rght hand sde are compared aganst the tme taken to solve one rght hand sde usng a partally pvoted LU decomposton routne n Egen 34, to hghlght the speedup attaned usng the proposed algorthm The relatve error was computed by feedng n a known (λ exact, a exact ) and comparng these wth the results obtaned by dfferent algorthms All the llustratons were run on a machne wth a sngle core 66 GHz processor and 8 GB RAM There was no necessty to parallelze the mplementaton for the present purpose due to the speedup acheved wth the proposed algorthm 53 Results and dscusson We present a detaled numercal benchmark for the radal bass functons lsted n Table For all these dfferent radal bass functons, the parameter a was chosen to be the radus of the crcle, whch n our case s 1 The ponts on the unt crcle are parameterzed as (x, y) (cos(θ), sn(θ)); r denotes the Eucldean dstance between two ponts on the unt crcle, e, the dstance between the ponts and j on the crcle s gven by ( θ θ ) j r j x x j sn Table : Radal bass functons φ(r) for whch numercal benchmarkng was performed Quadrc Mult-quadrc Inverse quadrc Inverse mult-quadrc Exponental Gaussan Logarthm 1 + (r/a) 1 + (r/a) 1/(1 + (r/a) ) 1/ 1 + (r/a) exp( r/a) exp( r /a ) log(1 + r/a) Consder for example an off-dagonal block n the matrx wth entres of the form Φ j φ(r j ) φ( x ( ( θ θ ) ) j x j ) φ sn ψ(θ θ j ), where θ, θ j I 0, π) The low-rank approxmatons are constructed by usng Chebyshev polynomals that are functons of θ In, 46, an analyss s presented where the order of Chebyshev polynomals requred to buld a low-rank approxmaton s estmated based on the growth of the kernel ψ n the complex plane along an ellpse contanng

12 I 0, π) The rank can be shown to be determned prmarly by two factors: the dstance between the poles of ψ n the complex plane and the nterval I, and the growth of ψ n the complex plane As we chose a 1 (whch determnes the locaton of the poles), we therefore expect a rapd decay of the error wth the rank We performed the followng seres of tests for each of the radal bass functons n table () In order to reduce the number of pages, not all our results are shown n ths paper We, however, ran all the calculatons and analyzed all the plots In cases where many plots were smlar, we selected a few representatve ones for ncluson n ths paper The detals of the tests that were performed are gven below We looked at how the rank of the off-dagonal blocks grows wth N for all the radal bass functons n table () The system sze was made to ncrease from 104 to 819 The decay of the sngular-values for the off-dagonal blocks can be found n fgure (3) There was no notceable growth of the rank of the off-dagonal blocks wth N r 1 + r r 1 + r Sngular Values r 1 1+r exp( r) exp( r ) log(1 + r) Sngular Values r 1 1+r exp( r) exp( r ) log(1 + r) Index Index Fgure 3: Decay of sngular values of the off-dagonal blocks for dfferent radal bass functons The sngular values are normalzed usng the largest sngular-value Left: system sze ; Rght: system sze We then consdered matrces We computed the condton number of the system and the decay of the sngular values of the largest off-dagonal block, whch s of sze The condton numbers ranged from to 10 7 as shown n table (3) Although a wde range of condton numbers were observed for these matrces, the accuracy of our algorthm was found to be largely ndependent of the condton number As explaned prevously, the accuracy s determned by the decay of the sngular values, whose behavor s dfferent from the condton number Table 3: Condton number of a system for dfferent kernels where the ponts are dstrbuted randomly on a unt crcle Kernel 1 + r 1 + r 1/(1 + r ) 1/ 1 + r exp( r) exp( r ) log(1 + r) Condton number For the lnear system, we computed: The relatve error n the soluton obtaned by usng the O() and O() algorthms as a functon of the rank of the off-dagonal blocks As explaned above, the rght-hand-sde of the system was computed from a known (λ, a), so that the exact soluton s known The assembly tme and solve tme taken for both algorthms as a functon of the rank Assembly tme denotes the tme taken to compute the desred entres n the matrx and the desred low-rank factorzatons to set up the herarchcal structure Specfcally, ths ncludes the tme taken to compute K (κ) R N/κ N/ κ, U (κ) R N/κ p where {1,, 3,, κ }, V (k) 1,, V (k), 1 RN/k p at all levels, k {1,, 3,, κ}, {1,, 3,, k 1 } and S (k) R p p at all levels where k {, 3,, κ} and {1,, 3,, k } Solve tme denotes the tme taken to compute the soluton to the lnear system, whch ncludes both the tme taken to obtan the factorzaton (secton 41) and perform the solve (secton 4)

13 Recall that the factorzaton phase and the solve phase proceed together as mentoned n secton 4 Next for all the radal bass functons, we fxed the off-dagonal rank at 30 and used Chebyshev nterpolaton for θ, to construct a low-rank representaton of the off-dagonal blocks For the O() and O() algorthms, we ncrease the system sze from 18 to We compared the two algorthms wth the partally pvoted LU algorthm For the partally pvoted LU algorthm, we ncreased the system sze from 18 to 819 Beyond ths, the partally pvoted LU algorthm took too much tme The followng comparsons were made: Relatve error n the soluton obtaned by usng the fast algorthms and the partally pvoted LU algorthms as a functon of the system sze Assembly tme taken for the fast algorthms and the partally pvoted LU algorthms as a functon of the system sze Solve tme taken for the fast algorthms and the partally pvoted LU algorthms as a functon of the system sze Next we also analyzed how the cost at each level vared for the two fast algorthms To do ths, we consdered a system sze of and fxed the rank of the off-dagonal blocks at 0 The total number of levels n ths case s 13 We measured the tme taken to assemble and solve at each level All the tme taken shown n the fgures and tables are n seconds Most of the tests returned smlar results n terms of accuracy and performance In order to reduce the number of numercal results ncluded, we present detaled tests for exp( r ) only 531 Gaussan The Gaussan radal bass functon s gven by φ(r) exp( r ) We present detaled results for ths functon as t s wdely used n many radal bass functon nterpolaton Another reason s that among all the radal bass functons we consdered, the Gaussan and the nverse quadrc show the slowest decay of sngular-values of the off-dagonal block For all the radal bass functons, the relatve error obtaned usng the algorthm s nearly the same as the relatve error obtaned usng the algorthm Ths hghlghts the fact that n our case the HODLR systems are n fact p-hss systems as well The comparson of these fast algorthms wth egen 34, an effcent C++ lnear algebra package, hghlghts the performance of these fast algorthms Snce we use a partally pvoted LU factorzaton to solve the lnear system usng egen, the total tme scales as O(N 3 ) We observe a huge reducton n runnng tme wth these fast new algorthms The relatve error between egen and the proposed algorthm s also very small because of the rapd decay of the sngular values (3) Further, between the fast algorthms, the dfference n asymptotc scalng between O() and O() s rather mportant as the fgures (4), (5) and table (4) ndcate A detaled analyss of the assembly tme and solve tme at each level for the fast algorthms s shown n fgure (6) Level 0 s the root and level 16 s the leaf Ths clearly hghlghts the dfference n the computatonal cost between the two fast algorthms For the O() algorthm, the assembly tme remans nearly the same across all levels However, the solve tme grows lnearly wth the number of levels Ths s consstent wth our analyss n secton 41 In the case of the O() algorthm, the assembly tme (except for the last few levels close to the leaf) and the solve tme both reman the same at all levels The fact that the tme taken at all levels s almost the same, s agan consstent wth our analyss n secton 41 The ncrease n assembly tme for the last few levels s due to the fact that there s a prolferaton of small problems and hence hardware effects such as the sze of the memory cache play a role Also, at the leaf level, few addtonal computatons are needed to assemble the system and hence ths too ncreases the computaton tme at the leaf level 53 Quadrc bharmonc The quadrc bharmonc radal bass functon s gven by φ(r) 1 + (r/a) The exact rank of the off-dagonal blocks s 3 snce the radal bass functon s a quadratc polynomal n r, whch can also be seen n fgure (3) However, snce we are constructng the low-rank usng θ, we have that φ(r j ) 1+4 sn ( (θ θ j )/ ) and hence around 15 terms are needed to construct a good low-rank approxmaton The relatve error as a functon of rank and system sze are plotted n fgure (7) 533 Mult-quadrc bharmonc The radal bass functon s gven by φ(r) 1 + (r/a) The decay of the sngular-values of the off-dagonal blocks s moderate and the 5 th sngular-value s close to machne precson as seen n fgure (3) The relatve error as a functon of rank and system sze are plotted n fgure (8)

14 Relatve Error Rank Assembly Tme Rank Solve Tme Rank Fgure 4: Keepng the system sze fxed at 819; Left: Relatve error; Mddle: Tme taken to assemble n seconds; Rght: Tme taken to solve n seconds; versus rank of the off-dagonal blocks Relatve Error Egen System sze Assembly Tme Egen System Sze Solve Tme Egen System Sze Fgure 5: Keepng the rank of the off-dagonal block fxed at 30; Left: Relatve error; Mddle: Tme taken to assemble n seconds; Rght: Tme taken to solve n seconds; versus system sze N Table 4: Tme taken to solve a lnear system as a functon of the system sze holdng the rank of the off-dagonal blocks fxed at 30 System Sze Tme taken Egen Assembly Solve Total Assembly Solve Total Assembly Solve Total

15 Tme taken,000 1,500 1, Tme taken Level Level Assembly Solve Assembly Solve Fgure 6: Splt up of the tme taken by the algorthms at each level for a 1, 048, 576 1, 048, 576 system Left: algorthm; Rght: algorthm Egen Relatve Error Relatve Error Rank System sze Fgure 7: Relatve error for 1 + (r/a) Left: versus rank for a system sze of 819 (the green and blue curves overlap); Rght: versus system sze keepng the off-dagonal rank as Egen Relatve Error Relatve Error Rank System sze Fgure 8: Relatve error for 1 + (r/a) Left: versus rank for a system sze of 819; Rght: versus system sze keepng the off-dagonal rank as 30

16 534 Inverse quadrc bharmonc The radal bass functon s gven by φ(r) 1 1+(r/a) The decay of the sngular-values of the off-dagonal blocks s very smlar to the decay of the Gaussan As before, the 5 th sngular-value s close to machne precson as seen n fgure (3) The relatve error as a functon of rank and system sze are plotted n fgure (9) Egen Relatve Error Relatve Error Rank System sze Fgure 9: Relatve error for 1 1+(r/a) Left: versus rank for a system sze of 819; Rght: versus system sze keepng the off-dagonal rank as Inverse mult-quadrc bharmonc 1 The radal bass functon s gven by φ(r) The decay of the sngular-values of the off-dagonal 1+(r/a) blocks s slghtly faster than nverse quadrc bharmonc but slower than mult-quadrc bharmonc As before, the 5 th sngular-value s close to machne precson as seen n fgure (3) The relatve error as a functon of rank and system sze are plotted n fgure (10) Egen Relatve Error Relatve Error Rank System sze 1 Fgure 10: Relatve error for Left: versus rank for a system sze of 819; Rght: versus system sze 1+(r/a) keepng the off-dagonal rank as Exponental The radal bass functon s gven by φ(r) exp( r/a) The decay of the sngular-values of the off-dagonal blocks s rapd The 15 th sngular-value s close to machne precson as seen n fgure (3) It s to be noted that even though the condton number of the system formed usng the exponental radal bass functon s relatvely large, e, around 10 6, ths does not seem to affect the relatve error of the soluton obtaned usng the fast algorthm as seen n fgure (11)

17 Egen Relatve Error Relatve Error Rank System sze Fgure 11: Relatve error for exp( r/a) Left: versus rank for a system sze of 819; Rght: versus system sze keepng the off-dagonal rank as Logarthm The radal bass functon s gven by φ(r) log(1 + r/a) The decay of the sngular-values of the off-dagonal blocks s smlar to exp( r/a) The 15 th sngular-value s close to machne precson as seen n fgure (3) It s also to be noted that, smlar to the exponental radal bass functon, the condton number for the system formed usng the logarthm radal bass functon s also large, e, around 10 7 However, ths does not seem to affect the relatve error of the soluton obtaned usng the fast algorthm as seen n fgure (1) Egen Relatve Error Relatve Error Rank System sze Fgure 1: Relatve error for log(1 + r/a) Left: versus rank for a system sze of 819; Rght: versus system sze keepng the off-dagonal rank as 30 6 Conclusons We have presented a new algorthm for solvng lnear systems that have a partal herarchcally sem-separable structure Such systems occur frequently n many applcatons, for example when the underlyng kernel s smooth and non-oscllatory The algorthm presented has a computatonal complexty of O() and a storage cost of O() We have llustrated the applcaton of the solver wth detaled numercal benchmarks These numercal benchmarks valdate the fact that the computatonal complexty s O() and the robustness of the method The man advantage of ths algorthm s that t s not only conceptually easy to understand and mplement, but also qute general and robust as the numercal benchmarks ndcate The algorthm was mplemented both n C++ and MATLAB The C++ mplementaton can be found at

18 acknowledgements Svaram Ambkasaran would lke to thank Krthka Narayanaswamy for proof readng the paper and helpng n generatng the fgures References 1 Andranaks I, Challenor P (01) The effect of the nugget on gaussan process emulators of computer models Computatonal Statstcs & Data Analyss Arnold W (1951) The prncple of mnmzed teratons n the soluton of the matrx egenvalue problem Quart Appl Math 9(1): Barnes J, Hut P (1986) A herarchcal O() force-calculaton algorthm Nature 34(4): Baxter B (010) The nterpolaton theory of radal bass functons arxv preprnt arxv: Beatson R, Greengard L (1997) A short course on fast multpole methods Wavelets, multlevel methods and ellptc PDEs pp Beatson R, Newsam G (199) Fast evaluaton of radal bass functons: I Computers & Mathematcs wth Applcatons 4(1): Beatson R, Cherre J, Mouat C (1999) Fast fttng of radal bass functons: Methods based on precondtoned GMRES teraton Advances n Computatonal Mathematcs 11(): Bllngs S, Beatson R, Newsam G (00) Interpolaton of geophyscal data usng contnuous global surfaces Geophyscs 67(6): Börm S, Grasedyck L, Hackbusch W (003) Herarchcal matrces Lecture notes 1 10 Buhmann M (003) Radal bass functons: theory and mplementatons, vol 1 Cambrdge Unversty Press 11 Chandrasekaran S, Dewlde P, Gu M, Pals T, Sun X, van der Veen A, Whte D (006) Some fast algorthms for sequentally semseparable representatons SIAM Journal on Matrx Analyss and Applcatons 7():341 1 Chandrasekaran S, Gu M, Pals T (006) A fast ULV decomposton solver for herarchcally semseparable representatons SIAM Journal on Matrx Analyss and Applcatons 8(3): Chen K (001) An analyss of sparse approxmate nverse precondtoners for boundary ntegral equatons SIAM Journal on Matrx Analyss and Applcatons (4): Cheng H, Greengard L, Rokhln V (1999) A fast adaptve multpole algorthm n three dmensons Journal of Computatonal Physcs 155(): Cheng H, Gmbutas Z, Martnsson P, Rokhln V (005) On the compresson of low-rank matrces SIAM Journal on Scentfc Computng 6(4): Cofman R, Rokhln V, Wandzura S (1993) The fast multpole method for the wave equaton: A pedestran prescrpton Antennas and Propagaton Magazne, IEEE 35(3): Darve E (000) The fast multpole method : Error analyss and asymptotc complexty SIAM Journal on Numercal Analyss 38(1): Darve E (000) The fast multpole method: numercal mplementaton Journal of Computatonal Physcs 160(1): Davs G, Morrs M (1997) Sx factors whch affect the condton number of matrces assocated wth krgng Mathematcal geology 9(5): De Boer A, Van der Schoot M, Bjl H (007) Mesh deformaton based on radal bass functon nterpolaton Computers & Structures 85(11-14): Detrch C, Newsam G (1995) Effcent generaton of condtonal smulatons by Chebyshev matrx polynomal approxmatons to the symmetrc square root of the covarance matrx Mathematcal geology 7():07 8

19 Fong W, Darve E (009) The black-box fast multpole method Journal of Computatonal Physcs 8(3): Freund R (1993) A transpose-free quas-mnmal resdual algorthm for non-hermtan lnear systems SIAM Journal on Scentfc Computng 14:470 4 Freund R, Nachtgal N (1991) QMR: a quas-mnmal resdual method for non-hermtan lnear systems Numersche Mathematk 60(1): Freze A, Kannan R, Vempala S (004) Fast Monte-Carlo algorthms for fndng low-rank approxmatons Journal of the ACM (JACM) 51(6): Gllman A, Young P, Martnsson P (011) A drect solver wth O(N) complexty for ntegral equatons on one-dmensonal domans arxv preprnt arxv: Golub G, Van Loan C (1996) Matrx computatons, vol 3 Johns Hopkns Unv Press 8 Gorenov S, Tyrtyshnkov E, Zamarashkn N (1997) A theory of pseudoskeleton approxmatons Lnear Algebra and ts Applcatons 61(1-3):1 1 9 Grasedyck L, Hackbusch W (003) Constructon and arthmetcs of h-matrces Computng 70(4): Greengard L, Rokhln V (1987) A fast algorthm for partcle smulatons Journal of Computatonal Physcs 73(): Greengard L, Rokhln V (1997) A new verson of the fast multpole method for the Laplace equaton n three dmensons Acta Numerca 6(1): Greengard L, Gueyffer D, Martnsson P, Rokhln V (009) Fast drect solvers for ntegral equatons n complex three-dmensonal domans Acta Numerca 18(1): Gu M, Esenstat S (1996) Effcent algorthms for computng a strong rank-revealng QR factorzaton Socety 17(4): Guennebaud G, Jacob B, et al (010) Egen v Gumerov N, Duraswam R (007) Fast radal bass functon nterpolaton va precondtoned Krylov teraton SIAM Journal on Scentfc Computng 9(5): Hackbusch W (1999) A sparse matrx arthmetc based on\ cal h-matrces part : Introducton to {\ Cal H}-matrces Computng 6(): Hackbusch W, Börm S (00) Data-sparse approxmaton by adaptve H -matrces Computng 69(1): Hackbusch W, Khoromskj BN (000) A sparse H-matrx arthmetc Computng 64(1): Hackbusch W, Nowak Z (1989) On the fast matrx multplcaton n the boundary element method by panel clusterng Numersche Mathematk 54(4): Hager W (1989) Updatng the nverse of a matrx SIAM revew pp Hestenes M, Stefel E (195) Methods of conjugate gradents for solvng lnear systems Journal of Research of the Natonal Bureau of Standards 49(6): Kong WY, Bremer J, Rokhln V (011) An adaptve fast drect solver for boundary ntegral equatons n two dmensons Appled and Computatonal Harmonc Analyss 31(3): Lberty E, Woolfe F, Martnsson P, Rokhln V, Tygert M (007) Randomzed algorthms for the low-rank approxmaton of matrces Proceedngs of the Natonal Academy of Scences 104(51):0, Martnsson P (009) A fast drect solver for a class of ellptc partal dfferental equatons Journal of Scentfc Computng 38(3): Martnsson P, Rokhln V (005) A fast drect solver for boundary ntegral equatons n two dmensons Journal of Computatonal Physcs 05(1): Messner M, Schanz M, Darve E (011) Fast drectonal multlevel summaton for oscllatory kernels based on Chebyshev nterpolaton Journal of Computatonal Physcs

20 47 Mranan L, Gu M (003) Strong rank revealng LU factorzatons Lnear Algebra and ts Applcatons 367: Nshmura N (00) Fast multpole accelerated boundary ntegral equaton methods Appled Mechancs Revews 55(4): O Dowd RJ (1991) Condtonng of coeffcent matrces of ordnary krgng Mathematcal Geology 3(5): Page CC, Saunders MA (1975) Soluton of sparse ndefnte systems of lnear equatons SIAM Journal on Numercal Analyss 1(4): Pan CT (000) On the exstence and computaton of rank-revealng LU factorzatons Lnear Algebra and ts Applcatons 316(1):199 5 Rjasanow S (00) Adaptve cross approxmaton of dense matrces IABEM 00, Internatonal Assocaton for Boundary Element Methods 53 Saad Y, Schultz M (1986) GMRES: A generalzed mnmal resdual algorthm for solvng nonsymmetrc lnear systems SIAM Journal on Scentfc and Statstcal Computng 7(3): Schaback R (1995) Creatng surfaces from scattered data usng radal bass functons Mathematcal methods for curves and surfaces pp Schmtz P, Yng L (011) A fast drect solver for ellptc problems on general meshes n D Journal of Computatonal Physcs 56 Schmtz P, Yng L (01) A fast drect solver for ellptc problems on Cartesan meshes n 3D, n revew 57 Vandebrl R, Barel M, Golub G, Mastronard N (005) A bblography on semseparable matrces Calcolo 4(3): Vavass SA (199) Precondtonng for boundary ntegral equatons SIAM journal on matrx analyss and applcatons 13(3): Van der Vorst HA (199) B-cgstab: A fast and smoothly convergng varant of b-cg for the soluton of nonsymmetrc lnear systems SIAM Journal on scentfc and Statstcal Computng 13(): Wang J, Lu G (00) A pont nterpolaton meshless method based on radal bass functons Internatonal Journal for Numercal Methods n Engneerng 54(11): Woodbury MA (1950) Invertng modfed matrces, statstcal Research Group, Memo Rep no 4, Prnceton Unversty 6 Woolfe F, Lberty E, Rokhln V, Tygert M (008) A fast randomzed algorthm for the approxmaton of matrces Appled and Computatonal Harmonc Analyss 5(3): Wu Z, Schaback R (1993) Local error estmates for radal bass functon nterpolaton of scattered data IMA Journal of Numercal Analyss 13(1): Xa J, Chandrasekaran S, Gu M, L X (010) Fast algorthms for herarchcally semseparable matrces Numercal Lnear Algebra wth Applcatons 17(6): Yng L, Bros G, Zorn D (004) A kernel-ndependent adaptve fast multpole algorthm n two and three dmensons Journal of Computatonal Physcs 196():591 66