Model order reduction via Proper Orthogonal Decomposition Reduced Basis Summer School 2015 Martin Gubisch University of Konstanz September 17, 2015 Martin Gubisch (University of Konstanz) Model order reduction via POD September 17, 2015 1 / 37 Motivation Martin Gubisch (University of Konstanz) Model order reduction via POD September 17, 2015 2 / 37
Introduction Let X be a Hilbert space and be a time interval. For a given function y L 2 (, X ) we want to find a reduced basis Ψ l = {ψ 1,..., ψ l } X of X -orthonormal elements ψ 1,..., ψ l such that y admits an optimal representation in L 2 (, span Ψ l ), in the sense that ψ 1,..., ψ l solve min ψ 1,...,ψ l X y(t) 2 y(t), ψ l X ψ l l=1 X dt s.t. ψ k, ψ l X = δ kl (1) where δ denotes the Kronecker-Delta δ kl = 1 for k = l and 0 elsewise. A solution to (1) is called a rank-l POD basis. Martin Gubisch (University of Konstanz) Model order reduction via POD September 17, 2015 3 / 37 Introduction The minimization problem (1) is equivalent to maximizing the averaged projection of the snapshots y(t), t, onto the reduced space span Ψ l : max ψ 1,...,ψ l l=1 y(t), ψ l 2 X dt s.t. ψ k, ψ l X = δ kl ; (2) especially, a given rank-l POD basis Ψ l can be expanded to a rank-(l + 1) basis by a solution ψ l+1 to max y(t), ψ 2 ψ (span Ψ l ) X dt, (3) choosing Ψ l+1 = Ψ l {ψ l+1 }; it is not required to calculate l+1 new basis elements. Martin Gubisch (University of Konstanz) Model order reduction via POD September 17, 2015 4 / 37
Basis construction Since problem (2) is a constrained optimization problem in a Banach space, we apply the Lagrange calculus: We define the Lagrange function L : X l R l l R by L (ψ, λ) = y(t), ψ l 2 X dt + l=1 λ kl ( ψ k, ψ l X δ kl ) (4) and receive the necessary first-order optimality conditions y(t), ψ l X y(t) dt = λ ll ψ l : (5) k,l=1 The POD basis elements solve an eigenvalue problem [4]. Martin Gubisch (University of Konstanz) Model order reduction via POD September 17, 2015 5 / 37 Basis construction Furthermore, the maximal value of (2) is y(t) 2 y(t), ψ l X ψ l l=1 X dt = l=1 y(t), ψ l 2 X dt = l>l λ ll, (6) so we look for eigenvectors corresponding to the l largest eigenvalues of (5). The eigenvalue problem (5) admits a solution in X l R l since the operator R(y) : X X, R(y)φ = y(t), φ X y(t) dt (φ X ) (7) is nonnegative, selfadjoint and compact. Martin Gubisch (University of Konstanz) Model order reduction via POD September 17, 2015 6 / 37
Projection error Let P l X : X span Ψl denote the orthogonal projection on the POD space, i.e. P l X φ = arg min ψ span Ψ l φ φ 2 X = φ, ψ l X ψ l (8) l=1 for each φ X. Then we have a formula for the POD projection error [10]: y(t) P l X y(t) 2 X = l>l λ l l 0. (9) Martin Gubisch (University of Konstanz) Model order reduction via POD September 17, 2015 7 / 37 Projection error Let V, H be Hilbert spaces with dense and continuous embedding V H and let y L 2 (, V). Choose X = H, then the V-orthogonal projection P l V : V span Ψ l has the following representation in the H-orthonormal basis Ψ l : P l Vφ = arg min φ span Ψ l φ φ 2 V = k,l=1 M 1 V (ψ) lk φ, ψ k H ψ l (10) where M V (ψ) R l l is the weights matrix M V (ψ) lk = ψ l, ψ k V. We get [15] y(t) PVy(t) l 2 V dt = λ l ψ l P Vψ l l 2 V l=l+1 l 0. (11) Martin Gubisch (University of Konstanz) Model order reduction via POD September 17, 2015 8 / 37
Projection error Let again be X = H. Then we have the error formula [1] y(t) PHy(t) l 2 V dt = λ l ψ l 2 V l=l+1 l 0. (12) The estimates (11) and (6) allow to bound the projection errors in V allthough the POD basis elements are orthogonal in H. This will be useful when POD is applied to partial differential equations. If y H 1 (, V), then (6) will allow not only to approximate y, but also its time derivative ẏ as we will see later. Martin Gubisch (University of Konstanz) Model order reduction via POD September 17, 2015 9 / 37 Method of snapshots We define the multiplication operator Q(y) : L 2 (, R) X and it s adjoints Q(y) : X L 2 (, R), Q(y)φ = φ(t)y(t) dt, (Q(y) φ)(t) = φ, y(t) X. (13) Then the composition satisfies R(y) = Q(y)Q(y) : V V. Let K(y) = Q(y) Q(y) : L 2 (, R) L 2 (, R). Since R(y) and K(y) have the same eigenvalues with the same multiplicities (except of possible 0), a POD basis is also given by a spectral decomposition (λ l, φ l ) l=1,...,l L 2 (, R) l R l of (K(y)φ)(t) = φ(s)y(s) ds, y(t) = φ(s) y(s), y(t) X ds, (14) X choosing ψ l = 1 λl Q(y)φ l V. Martin Gubisch (University of Konstanz) Model order reduction via POD September 17, 2015 10 / 37
Discretization Let X n X be a finite-dimensional subspace of X spanned by the linearly independent elements ϕ 1,..., ϕ n V with mass matrix M(ϕ) R n n, M(ϕ) jj = ϕ j, ϕ j X. Let {t 1,..., t m }. We replace y L 2 (, V) by y R n m such that y(t i ) n j=1 y jiϕ j and consider min ψ 1,...,ψ l R n m α i y i i=1 2 y i, ψ l R n ϕ ψ l l=1 R n ϕ s.t. ψ k, ψ l R n ϕ = δ kl (15) with the weighted scalar product, R n ϕ =, M(ϕ) R n and time weights α i > 0. Then a POD basis can be calculated by a decomposition of one of the matrices R(y) = ym(α)y T M(ϕ) R n n, K(y) = y T M(ϕ)yM(α) R m m (16) where M(α) jj = α j δ jj [4]. Martin Gubisch (University of Konstanz) Model order reduction via POD September 17, 2015 11 / 37 Discretization 1 Let ( ψ l, λ l ) be a decomposition of the symmetrized matrix R(y) = M(ϕ) 1/2 ym(α)y T M(ϕ) 1/2, then appropriate POD elements are ψ l = M(ϕ) 1/2 ψl R n. We require the decomposition of an n n matrix, the root of the mass matrix and solution steps for the transformation ψ ψ. 2 Let ( ψ l, λ l ) be a decomposition of the symmetrized matrix K(y) = M(α) 1/2 y T M(ϕ)yM(α) 1/2, then appropriate POD elements are ψ l = λ 1/2 l ym(α) 1/2 ψl : An m m matrix has to be decomposed, but no matrix roots or solving steps are needed. 3 A singular value decomposition ( ψ l, λ l, ψ l ) of M(ϕ) 1/2 ym(α) 1/2 is costly, but more robust; the eigenvalues corresponding to the POD elements ψ l are λ l = λ 2 l. Martin Gubisch (University of Konstanz) Model order reduction via POD September 17, 2015 12 / 37
Discretization % ------------------------------------------------------------------------- % POD determines a weighted POD matrix by solving the eigenvalue problem % R(Y)*psi = Y*diag(T)*Y *W*psi = lambda*psi. % % Input: % Y... nxm snapshot matrix % l... 1x1 pod basis rank % W... nxn space weights % T... mx1 time weights % % Output: % Psi... nxl pod basis % Lambda... lx1 eigenvalues % ------------------------------------------------------------------------- function [Psi,Lambda] = pod(y,l,w,t) % Build up pod operator R(Y) Alpha = spdiags(t,0,size(t,1),size(t,1)); % Alpha... mxm R = Y*Alpha*Y *W; % R... nxn % Solve eigenvalue problem by EIG [Psi,Lambda] = eig(r); [Lambda,Ord] = sort(diag(lambda), descend ); % Lambda... nx1 Lambda = Lambda(1:l,1); % Lambda... lx1 Psi = Psi(:,Ord); % Psi... nxn Psi = Psi(:,1:l); % Psi... nxl end Martin Gubisch (University of Konstanz) Model order reduction via POD September 17, 2015 13 / 37 Application: Filtering and compression of frogs Martin Gubisch (University of Konstanz) Model order reduction via POD September 17, 2015 14 / 37
Application: Filtering and compression of frogs Martin Gubisch (University of Konstanz) Model order reduction via POD September 17, 2015 15 / 37 Application: Filtering and compression of frogs Martin Gubisch (University of Konstanz) Model order reduction via POD September 17, 2015 16 / 37
Application: Filtering and compression of frogs Martin Gubisch (University of Konstanz) Model order reduction via POD September 17, 2015 17 / 37 September 17, 2015 18 / 37 Application: Filtering and compression of frogs Martin Gubisch (University of Konstanz) Model order reduction via POD
Application: Filtering and compression of frogs Martin Gubisch (University of Konstanz) Model order reduction via POD September 17, 2015 19 / 37 September 17, 2015 20 / 37 Application: Filtering and compression of frogs Martin Gubisch (University of Konstanz) Model order reduction via POD
Application: Filtering and compression of frogs Martin Gubisch (University of Konstanz) Model order reduction via POD September 17, 2015 21 / 37 Application: Filtering and compression of frogs Martin Gubisch (University of Konstanz) Model order reduction via POD September 17, 2015 22 / 37
Application: Filtering and compression of random data Martin Gubisch (University of Konstanz) Model order reduction via POD September 17, 2015 23 / 37 Application: Filtering and compression of random data Martin Gubisch (University of Konstanz) Model order reduction via POD September 17, 2015 24 / 37
Application: Filtering and compression of dynamical flows Martin Gubisch (University of Konstanz) Model order reduction via POD September 17, 2015 25 / 37 Application: Filtering and compression of dynamical flows Martin Gubisch (University of Konstanz) Model order reduction via POD September 17, 2015 26 / 37
Application: Frogs, randoms and dynamics 1 Photo compression with POD works quite well: from 10% of the original data onwards, the visible approximation errors vanish. 2 For random data, POD is completely useless (and it should be: it recognizes structurs only if they are there...). Taking 95% of the original data still does not lead to appropriate results although the original data storage is exceeded by 150%. 3 Dynamical flows in nice settings (such as diffusion dominated processes with smooth data) are reconstructed perfectly: In the example above, we just used two POD basis functions... Martin Gubisch (University of Konstanz) Model order reduction via POD September 17, 2015 27 / 37 Reduced order modeling Let Ψ l = {ψ 1,..., ψ l } be an orthonormal system in X {V, H}. In the following, we interprete the parabolic partial differential equation ẏ(t) + Ay(t) = f (t) in V, y(0) = y in H (17) as a variational problem in the reduced space span Ψ l V: ẏ(t), φ V,V + a(y(t), φ) = f (t), φ V,V, y(0), φ H = y, φ H (18) for all φ span Ψ l. The solution to (18) has the form y l H 1 (, V), y l (t) = y l (t)ψ l (19) l=1 Martin Gubisch (University of Konstanz) Model order reduction via POD September 17, 2015 28 / 37
Reduced order modeling The coefficient function y H 1 (, R l ) is given by the system of ordinary differential equations M(ψ)ẏ(t) + A(ψ)y(t) = f(ψ; t), M(ψ)y(0) = y (ψ); (20) M(ψ), A(ψ) R l l are defined as M(ψ) kl = ψ k, ψ l H, A(ψ) kl = a(ψ k, ψ l ) and the reduced data functions are f(ψ) L 2 (, R l ), and y (ψ) R l, (20) admits the unique solution f(ψ; t) l = f (t), ψ l V,V, y (ψ) l = y, ψ l H. y(t) = e tm(ψ) 1 A(ψ) y + t e (τ t)m(ψ) 1 A(ψ) M(ψ) 1 f(ψ; τ) dτ. (21) 0 Martin Gubisch (University of Konstanz) Model order reduction via POD September 17, 2015 29 / 37 Model reduction error Assume y = 0. Let X = V, (ψ 1,..., ψ l ) be a POD basis satisfying the problem ψ l, y(t) V y(t) dt = λ l ψ l and let y H 1 (, R l ) be the solution to the reduced problem (20). Then there exists a constant C > 0 just depending on the final time and the geometric data such that the ROM error can be estimated by y 2 y l ψ l l=1 L 2 (,V) H 1 (,V ) ( C l=l+1 λ l + ỹ(t) P l V ỹ(t) 2 L 2 (,V ) ). (22) Martin Gubisch (University of Konstanz) Model order reduction via POD September 17, 2015 30 / 37
Model reduction error Let X = V, (ψ 1,..., ψ l ) be a POD basis satisfying the problem ψ l, y(t) V y(t) dt + ψ l, ẏ(t) V ẏ(t) dt = λ l ψ l and let y H 1 (, R l ) be the solution to the reduced problem (20). Then there exists a constant C > 0 just depending on the final time and the geometric data such that the ROM error can be estimated by y 2 y l ψ l l=1 L 2 (,V) H 1 (,V ) C λ l. (23) l=l+1 Martin Gubisch (University of Konstanz) Model order reduction via POD September 17, 2015 31 / 37 Model reduction error Let X = H, (ψ 1,..., ψ l ) be a POD basis satisfying the problem ψ l, y(t) H y(t) dt + ψ l, ẏ(t) H ẏ(t) dt = λ l ψ l and let y H 1 (, R l ) be the solution to the reduced problem (20). Then there exists a constant C > 0 just depending on the final time and the geometric data such that the ROM error can be estimated by y 2 y l ψ l l=1 L 2 (,V) H 1 (,V ) C λ l ψ l P Vψ l l 2 V. (24) l=l+1 Martin Gubisch (University of Konstanz) Model order reduction via POD September 17, 2015 32 / 37
Model reduction error Let X = V, (ψ 1,..., ψ l ) be a POD basis satisfying the problem ψ l, y(t) V y(t) dt = λ l ψ l and let y H 1 (, R l ) be the solution to the reduced problem (20). Then there exists a constant C > 0 just depending on the final time and the geometric data such that the ROM error can be estimated by y 2 y l ψ l l=1 L 2 (,V) C λ l ψ l 2 V. (25) l=l+1 Martin Gubisch (University of Konstanz) Model order reduction via POD September 17, 2015 33 / 37 Motivation 1 Combination of POD with nonlinear PDE solvers such as Sequential Quadratic Programming [2], Trust Region Method [14] or Primal Dual Active Set Method [5]. 2 How does the reduction error react if the state y which builds up R(y) corresponds to a different source term f then the reduced system [6]? 3 In this case, the presented a-priori estimates are not valid any more. The design of efficient a-posteriori error bounds [16], especially for nonlinear equations [7], is in work. 4 The a-priori bounds [10] and convergence rates [9] are available if an appropriate POD basis update strategy (OS-POD) [11], [3] is used. 5 Applications to optimal control [5], parameter identification [12] and inverse problems [13]. 6 Combination of POD model reduction and Greedy algorithm in the reduced basis context [8]. Martin Gubisch (University of Konstanz) Model order reduction via POD September 17, 2015 34 / 37
References References I [1] Chapelle, D., Gariah, A. & Sainte-Marie, J.: Galerkin approximation with Proper Orthogonal Decomposition: New error estimates and illustrative examples. ESAIM: M2AN, vol. 46, no. 4: pp. 731 757, 2011. [2] Gräßle, C.: POD based inexact SQP methods for optimal control problems governed by a semilinear heat equation. Master s thesis, Universität Konstanz, 2014. [3] Grimm, E., Gubisch, M. & Volkwein, S.: A-Posteriori Error Analysis and OS-POD for Optimal Control. Comp. Eng., vol. 3: pp. 125 129, 2014. [4] Gubisch, M. & Volkwein, S.: POD Reduced-Order Modelling for PDE Constrained Optimization. proceeding Model Reduction and Approximation (Luminy), submitted, Oct. 2013. URL http://nbn-resolving.de/urn:nbn:de:bsz:352-250378. [5] Gubisch, M. & Volkwein, S.: POD a-posteriori error analysis for optimal control Problems with mixed constraints. Comp. Opt. Appl., vol. 58, no. 3: pp. 619 644, 2014. [6] Hinze, M. & Volkwein, S.: Error estimates for abstract linear-quadratic optimal control problems using POD. Comput. Optim. Appl., vol. 39: pp. 319 345, 2008. [7] Kammann, E., Tröltzsch, F. & Volkwein, S.: A method of a-posteriori error estimation with application to proper orthogonal decomposition. ESAIM M2AN, vol. 47, no. 2: pp. 555 581, 2013. [8] Kärcher, M. & Grepl, M. A.: A posteriori error estimation for reduced order solutions of parametrized parabolic optimal control problems. Math. Mod. Num., vol. 48, no. 6: pp. 1615 1638, 2014. Martin Gubisch (University of Konstanz) Model order reduction via POD September 17, 2015 35 / 37 References II References [9] Kunisch, K. & Müller, M.: Uniform convergence of the Pod method and applications to optimal control. Disc. Cont. Dyn. Syst., vol. 35, no. 9: pp. 4477 4501, 2015. [10] Kunisch, K. & Volkwein, S.: Galerkin proper orthogonal decomposition methods for parabolic problems. Numer. Math., vol. 90, no. 1: pp. 117 148, 2001. [11] Kunisch, K. & Volkwein, S.: Proper orthogonal decomposition for optimality systems. ESAIM M2AN, vol. 42: pp. 1 23, 2008. [12] Lass, O. & Volkwein, S.: Parameter identification for nonlinear elliptic-parabolic systems with application in lithium-ion battery modeling. Comp. Opt. Appl., vol. 62: pp. 217 239, 2015. [13] Ostrowsky, Z., Białecki, R. & Kassab, A.: Advances in Application of POD in Inverse Problems. Proc. Inv,. Prob. Eng., vol. 5: pp. 1 10, 2005. [14] Rogg, S.: Trust Region POD for Optimal Boundary Control of a Semilinear Heat Equation. Master s thesis, Universität Konstanz, 2014. [15] Singler, J. R.: New POD error expressions, error bounds, and asymptotic results for reduced order models. SIAM J. Numer. Anal., vol. 52, no. 2: pp. 852 876, 2014. [16] Tröltzsch, F. & Volkwein, S.: POD a-posteriori error estimates for linear-quadratic optimal control problems. Comput. Optim. Appl., vol. 44: pp. 319 345, 2008. Martin Gubisch (University of Konstanz) Model order reduction via POD September 17, 2015 36 / 37
References Motivation Martin Gubisch (University of Konstanz) Model order reduction via POD September 17, 2015 37 / 37