Diagonal preconditioning for first order primaldual algorithms in convex optimization


 Lambert Bruce
 1 years ago
 Views:
Transcription
1 Diagonal preconditioning for first order primaldual algorithms in conve optimization Thomas Pock Institute for Computer Graphics and Vision Graz University of Technology Antonin Chambolle CMAP & CNRS École Polytechnique Abstract In this paper we study preconditioning techniques for the firstorder primaldual algorithm proposed in [5]. In particular, we propose simple and easy to compute diagonal preconditioners for which convergence of the algorithm is guaranteed without the need to compute any step size parameters. As a byproduct, we show that for a certain instance of the preconditioning, the proposed algorithm is equivalent to the old and widely unknown alternating step method for monotropic programg [7]. We show numerical results on general linear programg problems and a few standard computer vision problems. In all eamples, the preconditioned algorithm significantly outperforms the algorithm of [5].. Introduction In [5, 8, 3] firstorder primaldual algorithms are studied to solve a certain class of conve optimization problems with known saddlepoint structure. ma K, y + G) F y), ) X y Y where X and Y are finitedimensional vector spaces equipped with standard inner products,. K : X Y is a linear operator and G : X R } and F : Y R } are conve functions with known structure. The iterates of the algorithm studied in [5] to solve ) are very simple: k+ =I + τ G) k τk T y k ) y k+ =I + σ F ) y k + σk k+ + θ k+ k ))) ) They basically consist of alternating a gradient ascend in the dual variable and a gradient descend in the primal The first author acknowledges support from the Austrian Science Fund FWF) under the grant P49N3. Figure. On problems with irregular structure, the proposed preconditioned algorithm PPD) converges significantly faster than the algorithm of [5] PD). variable. Additionally, the algorithm performs an overrelaation step in the primal variable. A fundamental assumption of the algorithm is that the functions F and G are of simple structure, meaning that the socalled proimity or resolvent operators I + σ F ) and I + τ G) have closedform solutions or can be efficiently computed with a high precision. Their eact definitions will be given in Section.. The parameters τ, σ > 0 are the primal and dual step sizes and θ [0, ] controls the amount of overrelaation in. It is shown in [5] that the algorithm converges as long as θ = and the primal and dual step sizes τ and σ are chosen such that τσl <, where L = K is the operator norm of K. It is further shown that a suitably defined partial primaldual gap of the average of the sequence 0, y 0 ),..., k, y k )) vanishes with rate O/k) for the complete class of problems covered by ). For problems with more regularity, the authors propose acceleration schemes based on nonempirical choices on τ, σ and θ. In particular they show that they can achieve O/k ) for problems where G of F is uniformly conve and Oω k ), ω < for problems where both G and F are uniformly conve. See [5] for more details. A common feature of all numerical eamples in [5] is that the involved linear operators K have a simple structure which makes it very easy to estimate L. We observed that for problems where the operator K has a more compli
2 cated structure, L cannot be estimated easily or it might be very large such that the convergence of the algorithm significantly slows down. As we will see, linear operators with irregular structure frequently arise in many different vision problems. In this work, we study preconditioning techniques for the primaldual algorithm ). This allows us to overcome the aforementioned shortcogs. The proposed preconditioned algorithm has several advantages. Firstly, it avoids the estimation of the operator norm of K, secondly, it significantly accelerates the convergence on problems with irregular K and thirdly, it leaves the computational compleity of the iterations basically unchanged. Figure shows convergence plots on two LP problems with such an irregular structure. The proposed algorithm can better adapt to the problem structure, leading to faster convergence. The rest of the paper is as follows. In Section. we fi some preliary definitions which will be used throughout the paper. In Section we present the preconditioned primaldual algorithm and give conditions under which convergence of the algorithm is guaranteed. We propose a family of simple and easy to compute diagonal preconditioners, which turn out to be very efficient on many problems. In Section.3 we establish connections to the old and widely unknown alternating step method for monotropic programg [7]. In Section 3 we detail eperimental results of the proposed algorithm. In the last Section we draw some conclusions and show directions for future work... Preliaries We consider finitedimensional vector spaces X and Y, where n = dim X and m = dim Y with inner products, X = T,,, X, y, y Y = Σ y, y, y, y Y, where T and Σ are a symmetric, positive definite preconditioning matrices. We further define the norms in the usual way as X =, X, y Y = y, y Y. We will make frequent use of the socalled resolvent or proimity operator of a function G). Given a point ˆ X, it is defined as the solution of the auiliary imization problem = arg G) + ˆ X The unique imizer to the above problem is characterized by the optimality condition G) + T ˆ) 0, whose optimal solution can be written in operator form as = I + T G) ˆ). 3). Preconditioned primaldual algorithm In this work, we propose the following preconditioned firstorder primaldual algorithm: Choose symmetric and positive definite matrices T, Σ, θ [0, ], 0, y 0 ) X Y. Then for k 0, update k, y k ) as follows: k+ =I + T G) k TK T y k ) y k+ =I + Σ F ) y k + ΣK k+ +θ k+ k ))) 4) Comparing the iterates 4) of the proposed algorithm to ), one can see that the global steps τ and σ have been replaced by the preconditioning matrices T and Σ. It is known that ) converges as long as θ = and τσ K <. Hence, a natural question is now to establish conditions on T and Σ and θ which ensure convergence of the proposed preconditioned algorithm. In very recent work [0], it has been shown that the iterates ) can be written in form of a proimal point algorithm [4], which greatly simplifies the convergence analysis. From the optimality conditions of the iterates 4) and the conveity of G and F it follows that for any, y) X Y the iterates k+ and y k+ satisfy ) ) ) k+ k+ y y k+,f y k+ +M k+ k y k+ y k 0, where ) ) k+ G F y k+ = k+ ) + K T y k+ F y k+ ) K k+, and [ T K M = T θk Σ 5) ]. 6) It is easy to check, that the variational inequality 5) now takes the form of a proimal point algorithm [0, 4, 6]. In the net Section we will establish conditions on θ, T and Σ which ensure convergence of the algorithm... Convergence of the algorithm We can make direct use of the convergence analysis developed in [0, 4, 6]. In fact, convergence of 5) can be guaranteed as long as the matri M is symmetric and positive definite. In the following Lemma we establish conditions on θ, T and Σ which indeed ensure these properties of M. Lemma. Let θ =, T and Σ symmetric positive definite maps satisfying Σ KT <, 7)
3 then the matri M defined in 6) is symmetric and positive definite, more precisely it holds, y) T, M, y) T > 0, 8) for any, y) T X Y and, y) T 0. Proof. Due to the structure of M, symmetry follows directly from the symmetry of T, Σ and the fact that θ =. Epanding the inner product in 8) we have T, + Σ y, y K, y > 0 X + y Y K, y > 0. 9) Since T and Σ are symmetric and positive definite, the above scalar product can be rewritten as K, y = Σ KT T, Σ y Using the CauchySchwarz inequality and the fact that ab ca + b /c for any a, b and c > 0, we obtain K, y Σ KT T Σ y c Σ KT X + ) c y Y In view of 7), it is clear that there eists a ε > 0 such that it holds + ε) Σ KT =. With this relation, we can perform suitable choices of c and /c such that our estimate becomes ) Σ KT X K, y + ε) Σ KT + y Y + ε) ) X + y ) Y. + ε Inserting this estimate into 9), we obtain ε +ε ) X + y Y ) > 0. Since ε > 0, it becomes clear that the condition 7) ensures positive definiteness of M. Remark. Choosing T = τ I and Σ = σi, for some τ, σ > 0, the assertion 7) reduces to τσ K <, which is the original condition on the step widths τ and σ to ensure convergence of the primaldual algorithm [5]. Note that this criterion requires to compute or to estimate) the operator norm of K which can be difficult in some situations. Theorem. Let θ = and T, Σ be symmetric and positive definite matrices such that Σ KT <. Then, the sequence n, y n ) generated by the iterates 4) converges weakly to an optimal solution, y ) of the saddlepoint problem ). Proof. As shown in Lemma, the conditions θ = and Σ KT < ensure that the matri M defined in 6) is symmetric and positive definite. Hence, the variational inequality 5) takes the form of a proimal point algorithm. Convergence of the iterates 4) to an optimal solution, y ) of ) then follows from the weak convergence of the proimal point algorithm Theorem of [4]). In the net Section we study simple diagonal preconditioners which obey 7) and hence ensure the convergence of the algorithm. Furthermore, computing these preconditioners only requires to compute row and column wise norms of K, which can be done very efficiently... A family of diagonal preconditioners In general, Σ and T could be any symmetric, positive definite maps. However, since it is a fundamental requirement of the proposed algorithm, that the resolvent operators are simple, some care has to be taken in choosing T and Σ. In particular, if G and F are separable in j and y i, the resolvent operators remain simple, if Σ and T are restricted to diagonal matrices. In the following we study a family of preconditioners which satisfy all these requirements and additionally guarantee the convergence of the algorithm. Lemma. Let T = diagτ ), where τ = τ,...τ n ) and Σ = diagσ), where σ = σ,..., σ m ). In particular, τ j = m i= K i,j, σ α i = n j= K 0) i,j α then for any α [0, ] Σ KT Σ KT = sup X, 0. ) Proof. In order to prove the inequality, we need to find an upper bound on Σ KT. We have Σ KT = Σ KT, Σ KT m n = σi K i,j τj j = i= m i= m i= σ i σ i j= n K i,j τj j j= n K i,j α Ki,j α τ j j j= Then, by applying the CauchySchwarz inequality we obtain m n n Σ KT σ i K i,j α K i,j α τ j j i= j= j=
4 By definition of σ i and τ j, the above inequality can be simplified to Σ KT = m i= j= n K i,j α τ j j n m ) K i,j α τ j j =. j= i= Substituting back into the definition of the operator norm, we finally obtain Σ KT Σ KT = sup X, 0 = ) Remark. To ensure positive definiteness in 8) inequality ) has to be strict. This requirement can be easily satisfied by multiplying τ and σ by appropriate positive constants µ and ν such that µν <. However, in practice, we did not observe any convergence problems of the algorithm for µ = ν =. Remark 3. Basically, the diagonal preconditioning leads to dimensiondependent times steps τ j and σ i instead of global steps τ and σ of the algorithm in [5] and hence do not change the computational compleity of the algorithm..3. A remark on the alternating step method A monotropic program is defined as a conve optimization problem of the form h), s.t. A = b, 3) where R n, h) = n j= h j j ) with h j ) being conve functions, A R m n and b R m. Let us denote by a j the j th column of A, by r i ) the i th entry of the vector r) = b A and by q i the number of nonzero entries in the i th row of A. In [7], the alternating step method for monotropic programg has been introduced as a special case of the alternating direction method of multipliers. It is defined as the iterative scheme k+ j = arg j h j j ) + a j, π k j + j λ a j [ k j + ]) m a i,jr i k } ) a j i= q i π k+ i = πi k + λ q i r i k+ ) 4) where π R m is the dual variable and λ > 0 is a parameter which weights between the strengths of the primal and dual updates. Let us now cast 3) in the form of ). Indeed, it can be written very easily as a saddlepoint problem by introducing Lagrange multipliers y for the constraint A = b, giving us ma b A, y + h). 5) y The iterates 4) now become k+ = I + T h) k + TA T y k ) y k+ = y k + Σb A k+ k )) 6) Let us now perform the particular choice α = 0 in 0) to compute T, Σ and perform an additional scaling by the factors /λ and λ, respectively see Remark ). We consequently find that τ j = λ a j and σ i = λ q i, where we used the convention that 0 0 = 0. By a change of variables in the duals, π k = y k Σb A k ), it can be checked that in this particular setting, 6) is equivalent to the alternating step method 4). 3. Numerical results In this Section we perform several numerical eperiments using the proposed algorithm. We first show applications to general linear programg problems and show that the proposed algorithm leads to a significantly faster convergence compared to the algorithm in [5]. Furthermore, we show that for large scale problems, a simple Matlab implementation of the proposed algorithm significantly outperforms a highly optimized interior point solver of Matlab. Finally, we show applications of the proposed algorithm to classical computer vision problems such as graph cuts and imal partitioning problems, where we show that a GPU implementation of the proposed algorithm can easily compete with specialized algorithms such as the maflow algorithm of Boykov and Kolmogorov [3]. In our numerical eperiments, we use the following algorithms and parameter settings: IP: Matlab s LP interior point solver linprog with settings optimized for large scale problems. PD: The primaldual algorithm ) where we set τ = L and σ = L and L = K is estimated using the Matlab command normest. PPD: Preconditioned primal dual algorithm 4) with the preconditioners T and Σ defined in 0) and using α =. In general, the solutions of the problems we consider here are not unique. Therefore, we use the value of the objective functions as a criterion to detere the convergence
5 of the algorithms. We define the relative error of the objective function as δ k = E E k / E, where E refers to the optimal value and E k is the value of the current iterate. Depending of the particular problem, E might be the value of either the primal or the dual problem. In order to detere the optimal value of the objective function we run one method with a very high accuracy. During the evaluation, we run the algorithms until δ k is below a fied threshold of tol = 0 4. Note that for practical purposes, δ k can also be bounded by the primaldual gap. All eperiments were eecuted on a.73 GHz i7 CPU running Matlab and a NVidia GTX480 GPU using the CUDA framework. 3.. Linear programg Linear programs LPs) define an important class of optimization problems in computer vision. See for eample the recent work [5] for a huge scale) LP formulation of curvature dependent segmentation and inpainting models. Therefore, there is a strong demand for efficient solver which can handle very large problems. Here, we consider LPs in socalled equality form: c T s.t. A = b, 0, 7) where, c R n, b R m and A R m n. Note that it is straightforward to transform other standard forms to this form []. The above LP can be easily reformulated in terms of a saddlepoint problem by introducing Lagrange multipliers y to account for the equality constraints: ma A b, y + c,, s.t. 0 y Applying PPD to 7) we obtain the following simple iterates k+ = proj [0, ) k TA T y k + c) ) y k+ = y k + ΣA k+ k ) b), 8) where proj [0, ) ) can be implemented by simple truncation operations. In our first eample, we compare IP, PD and PPD on two standard LP test problems that come along with the Matlab package. To gain more insight to the proposed preconditioning technique, we compare the global steps of PD with the dimensiondependent adaptive steps of PPD. sc50b: This problem consists of 48 variables, 30 inequalities and 0 equalities. We find L = and set τ = σ = for PD. For PPD we find that the average primal step is τ = and the average dual step is σ = Therefore, PPD can take in average approimately.8 times larger primal and dual steps than PD. densecolumn: This problem consists of 677 variables and 67 equalities with one dense column in the linear operator. We find L = and consequently set τ = σ = for PD. For PPD, we find the average primal and dual steps to be τ = and σ = Hence PPD can take on average approimately 40 times larger primal and 5 times larger dual steps. Table shows the results of the performance evaluation on the standard LP test problems. The optimal value of the objective function was detered by running IP with a very high accuracy. This eamples clearly demonstrate the advantage of the proposed algorithm. On sc50b P PD is.4 times faster than PD while on densecolumn, PPD is more than 00 times faster than PD. Especially, densecolumn has a very irregular structure which can be handled much better by the preconditioned algorithm. Clearly, on smallscale problems, the simple primaldual algorithms can not compete with the highly optimized IP. However, as we will see in the net section, this is quite different for largescale problems. Figure depicts convergence plots for PD and PPD for the two LP test problems. IP PD PPD sc50b 0.0s.75s 0.49s densecolumn 0.7s 68.5s 0.6s Table. Comparison of of IP, PD and PPD on two standard LP test problems. 3.. TVL image denoising The TVL model [, 6] is a popular image denoising model. It consists of total variation TV) regularization and a L data fidelity term. It is known that the TVL model is very effective in removing strong outliers such as salt and pepper noise in images. We assume discrete images defined on a regular Cartesian grid of size M N with indices i, j). The anisotropic) TVL model in a discrete setting is defined as the l imization problem u Du l + λ u f l, 9) where f = f i,j ) R MN is the noisy input image and u = u i,j ) R MN is the denoised image. The free parameter λ > 0 is used to control the tradeoff between smoothing and data fidelity. The linear operator D R MN MN is the discretized gradient operator defined by ) Du) Du) i,j = i,j Du), 0) i,j where Du) i,j = u i+,j u i,j if i < M 0 if i = M,
6 a) 3 phases b) Output c) 4 phases d) Output e) Natural f) Segmentation 8 phases) Figure 3. Figures ab) and cd) show the result of standard Dirichlet problems. Figures ef) show the segmentation of a natural image into 8 regions. usual isotropic total variation. The only modification is to replace the projections of the duals onto bo constraints by projections onto l balls. Figure shows the result of PD and PPD applied to a binary image segmentation problem. Let I [0, ] 3MN be the input image. The unary terms were computed by first computing the average RGB values µ f R 3 and µ b R 3 in the user specified foreground and background regions and then by setting wi,j u = α I i,j µ f I i,j µ b ). The binary terms were computed by w b, i,j = ep β I i+,j I i,j ) and w b, i,j = ep β I i,j+ I i,j ), where we set α = and β = 0. We used the popular maimum flow algorithm of Boykov and Kolmogorov [3] to compute the optimal value of the primal objective function ). We then ran PD and PPD until the relative error δ k was below tol. Table 3 presents the tigs. Although a Matlab implementation of PPD cannot compete with the highly optimized maimum flow algorithm, a parallel GPU implementation is already significantly faster. MAXFLOW PD PPD PPDGPU 0.60s 5.75s 8.56s 0.045s Table 3. Comparison of the proposed algorithm for graph cut segmentation Minimal partitions In the last eample we quickly eplain the application of the proposed preconditioned primaldual algorithm for computing imal partitions. Following [5] and references therein, a conve relaation of the multilabel Potts model can be computed by solving the saddle point problem where S = ma u S q B k Du l, q l + u l, f l, 3) l= u = u l ) k l= : } k u l =, u l 0, l= 4) is the usual simple constraint and the conve set B = Bm n, 5) m<n k can be written as the intersection of balls of the form Bm n = q = ql ), ql )) k l= : q m q n ) i,j, i, j }, 6) The linear operator D is the discretized gradient operator as defined in 0), u = u l ) k l=, is the labeling function which represents a relaed assignment of the image piels to each region, f = f l ) k l= is some given weight function and q = ql ), q l ))k l= are the dual variables. The original approach taken in [5] is to compute the projections with respect to S and B by solving auiliary optimization problems. This approach has the disadvantage that the inner optimization problems have to be computed with sufficient accuracy such that the overall algorithm converges, which can heavily slow down the algorithm. Here, we take a different approach, which is closely related to the multiple operator splitting method proposed in []. We first introduce slack variables b for the epressions q m q n in 6). Then we introduce Lagrange multipliers λ to account for the equality constraints l u l = in 4) and µ to account for the equality constraints b n m = q m q n. Together, this yields the following augmented saddlepoint problem u,µ ma q,b,λ l k= Du l, q l + u l, f l + P u, λ + b Qq, µ s.t. u l ) i,j 0, b n m) i,j, 7) where the linear operators P and Q are such that P u) i,j = k u l ) i,j, Qq) n m) i,j = q m q n ) i,j. l= Note that 7) now depends only on simple pointwise constraints for which the projections can be implemented very efficiently. To cast the above problem into the form of ), it remains to arrange all primal variables in a vector, all
7 dual variables into a vector y and all linear operators into a global linear operator K. Then, applying the preconditioning techniques proposed in this paper leads to an algorithm that is guaranteed to converge to the optimal solution without the need to solve any inner optimization problems. Figure 3 shows some results of standard imal partitioning and segmentation problems. We compared the original approach solving inner optimization problems and using PD to PPD applied to 7). We first precomputed the optimal solution using a large number of iterations and then recorded the time until the error is below a threshold of tol. The tigs are presented in Table 4. In all cases, the proposed algorithm clearly outperforms the original approach of [5]. PD PPD Speedup Synthetic 3 phases).7s 75.65s.9 Synthetic 4 phases) 39.0s s.6 Natural 8 phases) 59.85s 3.76s 5. Table 4. Comparison of the proposed algorithm on partitioning problems. 4. Conclusion In this paper we have proposed a simple preconditioning technique to improve the performance of the firstorder primaldual algorithm proposed in [3, 5]. The proposed diagonal preconditioners can be computed efficiently and guarantee the convergence of the algorithm without the need to estimate any step size parameters. In several numerical eperiments, we have shown that the proposed algorithm significantly outperforms the algorithm in [5]. Furthermore, on large scale linear programg problems, an unoptimized implementation of the proposed algorithm easily outperforms a highly optimized interior point solver and a GPU implementation of the proposed algorithm can easily compete with specialized combinatorial algorithms for computing imum cuts. We believe that the proposed algorithm can become a standard algorithm in computer vision since it can be applied to a large class of conve optimization problems arising in computer vision and has the potential for parallel computing. Future work will mainly concentrate on the development of more sophisticated preconditioners that are different from diagonal matrices. References [] A. Bhusnurmath and C. Taylor. Graph cuts via l norm imization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30:866 87, 008. [] S. Boyd and L. Vandenberghe. Conve Optimization. Cambridge University Press, 004. [3] Y. Boykov and V. Kolmogorov. An eperimental comparison of cut/maflow algorithms for energy imization in computer vision. In A. J. M. Figueiredo, J. Zerubia, editor, Energy Minimization Methods in Computer Vision and Pattern Recognition, volume 34 of LNCS, pages Springer, 00. [4] A. Chambolle. Total variation imization and a class of binary MRF models. In Energy Minimization Methods in Computer Vision and Pattern Recognition, pages 36 5, 005. [5] A. Chambolle and T. Pock. A firstorder primaldual algorithm for conve problems with applications to imaging. Journal of Mathematical Imaging and Vision, 00. online first. [6] T. Chan and S. Esedoglu. Aspects of total variation regularized L function approimation. SIAM J. Appl. Math., 655):87 837, 004. [7] J. Eckstein. Splitting Methods for Monotone Operators with Application to Parallel Optimization. PhD thesis, MIT, 989. [8] E. Esser, X. Zhang, and T. Chan. A general framework for a class of first order primaldual algorithms for conve optimization in imaging science. SIAM J. Imaging Sci., 3:05 046, 00. [9] L. Ford and D. Fulkerson. Flows in Networks. Princeton University Press, Princeton, New Jersey, 96. [0] B. He and X. Yuan. Convergence analysis of primaldual algorithms for total variation image restoration. Technical report, Nanjing University, China, 00. [] J. Lellmann, D. Breitenreicher, and C. Schnörr. Fast and eact primaldual iterations for variational problems in computer vision. In K. Daniilidis, P. Maragos, and N. Paragios, editors, European Conference on Computer Vision ECCV), volume 63 of LNCS, pages Springer Berlin / Heidelberg, 00. [] M. Nikolova. A variational approach to remove outliers and impulse noise. Journal of Mathematical Imaging and Vision, 0):99 0, 004. [3] T. Pock, D. Cremers, H. Bischof, and A. Chambolle. An algorithm for imizing the MumfordShah functional. In International Conference on Computer Vision ICCV), 009. [4] R. T. Rockafellar. Monotone operators and the proimal point algorithm. SIAM Journal on Control and Optimization, 45): , 976. [5] T. Schoenemann, F. Kahl, and D. Cremers. Curvature regularity for regionbased image segmentation and inpainting: A linear programg relaation. In IEEE International Conference on Computer Vision ICCV), Kyoto, Japan, 009. [6] X. Zhang, M. Burger, and S. Osher. A unified primaldual algorithm framework based onbregman iteration. Journal of Scientific Computing, 46:0 46, 0.
Optical Flow. Thomas Pock
Optical Flow Thomas Pock 1 Optical Flow (II) Content Global approaches (HornSchunck, TVL1) Coarsetofine warping 2 The Horn and Schunck (HS) Method 3 The Horn and Schunck (HS) Method Global energy to
More informationLecture 5: Conic Optimization: Overview
EE 227A: Conve Optimization and Applications January 31, 2012 Lecture 5: Conic Optimization: Overview Lecturer: Laurent El Ghaoui Reading assignment: Chapter 4 of BV. Sections 3.13.6 of WTB. 5.1 Linear
More informationMoving Least Squares Approximation
Chapter 7 Moving Least Squares Approimation An alternative to radial basis function interpolation and approimation is the socalled moving least squares method. As we will see below, in this method the
More informationProximal mapping via network optimization
L. Vandenberghe EE236C (Spring 234) Proximal mapping via network optimization minimum cut and maximum flow problems parametric minimum cut problem application to proximal mapping Introduction this lecture:
More informationRegression Using Support Vector Machines: Basic Foundations
Regression Using Support Vector Machines: Basic Foundations Technical Report December 2004 Aly Farag and Refaat M Mohamed Computer Vision and Image Processing Laboratory Electrical and Computer Engineering
More informationOn Convergence Rate of ConcaveConvex Procedure
On Convergence Rate of ConcaveConve Procedure Ian EnHsu Yen r00922017@csie.ntu.edu.tw PoWei Wang b97058@csie.ntu.edu.tw Nanyun Peng Johns Hopkins University Baltimore, MD 21218 npeng1@jhu.edu ShouDe
More informationc 2006 Society for Industrial and Applied Mathematics
SIAM J. OPTIM. Vol. 17, No. 4, pp. 943 968 c 2006 Society for Industrial and Applied Mathematics STATIONARITY RESULTS FOR GENERATING SET SEARCH FOR LINEARLY CONSTRAINED OPTIMIZATION TAMARA G. KOLDA, ROBERT
More informationDuality of linear conic problems
Duality of linear conic problems Alexander Shapiro and Arkadi Nemirovski Abstract It is well known that the optimal values of a linear programming problem and its dual are equal to each other if at least
More informationModern Optimization Methods for Big Data Problems MATH11146 The University of Edinburgh
Modern Optimization Methods for Big Data Problems MATH11146 The University of Edinburgh Peter Richtárik Week 3 Randomized Coordinate Descent With Arbitrary Sampling January 27, 2016 1 / 30 The Problem
More informationThe Method of Lagrange Multipliers
The Method of Lagrange Multipliers S. Sawyer October 25, 2002 1. Lagrange s Theorem. Suppose that we want to maximize (or imize a function of n variables f(x = f(x 1, x 2,..., x n for x = (x 1, x 2,...,
More informationDuality in General Programs. Ryan Tibshirani Convex Optimization 10725/36725
Duality in General Programs Ryan Tibshirani Convex Optimization 10725/36725 1 Last time: duality in linear programs Given c R n, A R m n, b R m, G R r n, h R r : min x R n c T x max u R m, v R r b T
More informationPractical Guide to the Simplex Method of Linear Programming
Practical Guide to the Simplex Method of Linear Programming Marcel Oliver Revised: April, 0 The basic steps of the simplex algorithm Step : Write the linear programming problem in standard form Linear
More informationVariational approach to restore pointlike and curvelike singularities in imaging
Variational approach to restore pointlike and curvelike singularities in imaging Daniele Graziani joint work with Gilles Aubert and Laure BlancFéraud Roma 12/06/2012 Daniele Graziani (Roma) 12/06/2012
More informationP164 Tomographic Velocity Model Building Using Iterative Eigendecomposition
P164 Tomographic Velocity Model Building Using Iterative Eigendecomposition K. Osypov* (WesternGeco), D. Nichols (WesternGeco), M. Woodward (WesternGeco) & C.E. Yarman (WesternGeco) SUMMARY Tomographic
More information(57) (58) (59) (60) (61)
Module 4 : Solving Linear Algebraic Equations Section 5 : Iterative Solution Techniques 5 Iterative Solution Techniques By this approach, we start with some initial guess solution, say for solution and
More informationAdaptive Online Gradient Descent
Adaptive Online Gradient Descent Peter L Bartlett Division of Computer Science Department of Statistics UC Berkeley Berkeley, CA 94709 bartlett@csberkeleyedu Elad Hazan IBM Almaden Research Center 650
More informationMassive Data Classification via Unconstrained Support Vector Machines
Massive Data Classification via Unconstrained Support Vector Machines Olvi L. Mangasarian and Michael E. Thompson Computer Sciences Department University of Wisconsin 1210 West Dayton Street Madison, WI
More informationInner Product Spaces and Orthogonality
Inner Product Spaces and Orthogonality week 34 Fall 2006 Dot product of R n The inner product or dot product of R n is a function, defined by u, v a b + a 2 b 2 + + a n b n for u a, a 2,, a n T, v b,
More informationAn Overview Of Software For Convex Optimization. Brian Borchers Department of Mathematics New Mexico Tech Socorro, NM 87801 borchers@nmt.
An Overview Of Software For Convex Optimization Brian Borchers Department of Mathematics New Mexico Tech Socorro, NM 87801 borchers@nmt.edu In fact, the great watershed in optimization isn t between linearity
More informationFurther Study on Strong Lagrangian Duality Property for Invex Programs via Penalty Functions 1
Further Study on Strong Lagrangian Duality Property for Invex Programs via Penalty Functions 1 J. Zhang Institute of Applied Mathematics, Chongqing University of Posts and Telecommunications, Chongqing
More information! Solve problem to optimality. ! Solve problem in polytime. ! Solve arbitrary instances of the problem. #approximation algorithm.
Approximation Algorithms 11 Approximation Algorithms Q Suppose I need to solve an NPhard problem What should I do? A Theory says you're unlikely to find a polytime algorithm Must sacrifice one of three
More informationMathematics 31 Precalculus and Limits
Mathematics 31 Precalculus and Limits Overview After completing this section, students will be epected to have acquired reliability and fluency in the algebraic skills of factoring, operations with radicals
More informationNumerical Methods For Image Restoration
Numerical Methods For Image Restoration CIRAM Alessandro Lanza University of Bologna, Italy Faculty of Engineering CIRAM Outline 1. Image Restoration as an inverse problem 2. Image degradation models:
More information4.6 Linear Programming duality
4.6 Linear Programming duality To any minimization (maximization) LP we can associate a closely related maximization (minimization) LP. Different spaces and objective functions but in general same optimal
More informationSOLVING LINEAR SYSTEMS
SOLVING LINEAR SYSTEMS Linear systems Ax = b occur widely in applied mathematics They occur as direct formulations of real world problems; but more often, they occur as a part of the numerical analysis
More informationPolynomial and Synthetic Division. Long Division of Polynomials. Example 1. 6x 2 7x 2 x 2) 19x 2 16x 4 6x3 12x 2 7x 2 16x 7x 2 14x. 2x 4.
_.qd /7/5 9: AM Page 5 Section.. Polynomial and Synthetic Division 5 Polynomial and Synthetic Division What you should learn Use long division to divide polynomials by other polynomials. Use synthetic
More informationTESLA Report 200303
TESLA Report 233 A multigrid based 3D spacecharge routine in the tracking code GPT Gisela Pöplau, Ursula van Rienen, Marieke de Loos and Bas van der Geer Institute of General Electrical Engineering,
More informationDecember 4, 2013 MATH 171 BASIC LINEAR ALGEBRA B. KITCHENS
December 4, 2013 MATH 171 BASIC LINEAR ALGEBRA B KITCHENS The equation 1 Lines in twodimensional space (1) 2x y = 3 describes a line in twodimensional space The coefficients of x and y in the equation
More informationExample 4.1 (nonlinear pendulum dynamics with friction) Figure 4.1: Pendulum. asin. k, a, and b. We study stability of the origin x
Lecture 4. LaSalle s Invariance Principle We begin with a motivating eample. Eample 4.1 (nonlinear pendulum dynamics with friction) Figure 4.1: Pendulum Dynamics of a pendulum with friction can be written
More informationA Negative Result Concerning Explicit Matrices With The Restricted Isometry Property
A Negative Result Concerning Explicit Matrices With The Restricted Isometry Property Venkat Chandar March 1, 2008 Abstract In this note, we prove that matrices whose entries are all 0 or 1 cannot achieve
More informationA fast multiclass SVM learning method for huge databases
www.ijcsi.org 544 A fast multiclass SVM learning method for huge databases Djeffal Abdelhamid 1, Babahenini Mohamed Chaouki 2 and TalebAhmed Abdelmalik 3 1,2 Computer science department, LESIA Laboratory,
More informationThe Trimmed Iterative Closest Point Algorithm
Image and Pattern Analysis (IPAN) Group Computer and Automation Research Institute, HAS Budapest, Hungary The Trimmed Iterative Closest Point Algorithm Dmitry Chetverikov and Dmitry Stepanov http://visual.ipan.sztaki.hu
More informationMATH 304 Linear Algebra Lecture 20: Inner product spaces. Orthogonal sets.
MATH 304 Linear Algebra Lecture 20: Inner product spaces. Orthogonal sets. Norm The notion of norm generalizes the notion of length of a vector in R n. Definition. Let V be a vector space. A function α
More informationCompletely Positive Cone and its Dual
On the Computational Complexity of Membership Problems for the Completely Positive Cone and its Dual Peter J.C. Dickinson Luuk Gijben July 3, 2012 Abstract Copositive programming has become a useful tool
More informationSupport Vector Machine. Tutorial. (and Statistical Learning Theory)
Support Vector Machine (and Statistical Learning Theory) Tutorial Jason Weston NEC Labs America 4 Independence Way, Princeton, USA. jasonw@neclabs.com 1 Support Vector Machines: history SVMs introduced
More informationBig Data  Lecture 1 Optimization reminders
Big Data  Lecture 1 Optimization reminders S. Gadat Toulouse, Octobre 2014 Big Data  Lecture 1 Optimization reminders S. Gadat Toulouse, Octobre 2014 Schedule Introduction Major issues Examples Mathematics
More informationDiagonal, Symmetric and Triangular Matrices
Contents 1 Diagonal, Symmetric Triangular Matrices 2 Diagonal Matrices 2.1 Products, Powers Inverses of Diagonal Matrices 2.1.1 Theorem (Powers of Matrices) 2.2 Multiplying Matrices on the Left Right by
More information3. Linear Programming and Polyhedral Combinatorics
Massachusetts Institute of Technology Handout 6 18.433: Combinatorial Optimization February 20th, 2009 Michel X. Goemans 3. Linear Programming and Polyhedral Combinatorics Summary of what was seen in the
More information(a) The transpose of a lower triangular matrix is upper triangular, and the transpose of an upper triangular matrix is lower triangular.
Theorem.7.: (Properties of Triangular Matrices) (a) The transpose of a lower triangular matrix is upper triangular, and the transpose of an upper triangular matrix is lower triangular. (b) The product
More informationInner products on R n, and more
Inner products on R n, and more Peyam Ryan Tabrizian Friday, April 12th, 2013 1 Introduction You might be wondering: Are there inner products on R n that are not the usual dot product x y = x 1 y 1 + +
More informationSection 33 Approximating Real Zeros of Polynomials
 Approimating Real Zeros of Polynomials 9 Section  Approimating Real Zeros of Polynomials Locating Real Zeros The Bisection Method Approimating Multiple Zeros Application The methods for finding zeros
More informationOn SDP and CPrelaxations and on connections between SDP, CP and SIP
On SDP and CPrelaations and on connections between SDP, CP and SIP Georg Still and Faizan Ahmed University of Twente p 1/12 1. IP and SDP, CPrelaations Integer program: IP) : min T Q s.t. a T j =
More information10. Proximal point method
L. Vandenberghe EE236C Spring 201314) 10. Proximal point method proximal point method augmented Lagrangian method MoreauYosida smoothing 101 Proximal point method a conceptual algorithm for minimizing
More information15.1. Exact Differential Equations. Exact FirstOrder Equations. Exact Differential Equations Integrating Factors
SECTION 5. Eact FirstOrder Equations 09 SECTION 5. Eact FirstOrder Equations Eact Differential Equations Integrating Factors Eact Differential Equations In Section 5.6, ou studied applications of differential
More informationAnytime plan TalkMore plan
CONDENSED L E S S O N 6.1 Solving Sstems of Equations In this lesson ou will represent situations with sstems of equations use tables and graphs to solve sstems of linear equations A sstem of equations
More informationFleet Assignment Using Collective Intelligence
Fleet Assignment Using Collective Intelligence Nicolas E Antoine, Stefan R Bieniawski, and Ilan M Kroo Stanford University, Stanford, CA 94305 David H Wolpert NASA Ames Research Center, Moffett Field,
More information1.5 SOLUTION SETS OF LINEAR SYSTEMS
12 CHAPTER 1 Linear Equations in Linear Algebra 1.5 SOLUTION SETS OF LINEAR SYSTEMS Many of the concepts and computations in linear algebra involve sets of vectors which are visualized geometrically as
More informationLAGRANGIAN RELAXATION TECHNIQUES FOR LARGE SCALE OPTIMIZATION
LAGRANGIAN RELAXATION TECHNIQUES FOR LARGE SCALE OPTIMIZATION Kartik Sivaramakrishnan Department of Mathematics NC State University kksivara@ncsu.edu http://www4.ncsu.edu/ kksivara SIAM/MGSA Brown Bag
More information2.3 Convex Constrained Optimization Problems
42 CHAPTER 2. FUNDAMENTAL CONCEPTS IN CONVEX OPTIMIZATION Theorem 15 Let f : R n R and h : R R. Consider g(x) = h(f(x)) for all x R n. The function g is convex if either of the following two conditions
More informationEigenvalues of C SMS (16) 1 A. Eigenvalues of A (32) 1 A (16,32) 1 A
Negative Results for Multilevel Preconditioners in Image Deblurring C. R. Vogel Department of Mathematical Sciences Montana State University Bozeman, MT 597170240 USA Abstract. A onedimensional deconvolution
More informationTHE NT (NEW TECHNOLOGY) HYPOTHESIS ABSTRACT
THE NT (NEW TECHNOLOGY) HYPOTHESIS Michele Impedovo Università Bocconi di Milano, Italy michele.impedovo@unibocconi.it ABSTRACT I teach a firstyear undergraduate mathematics course at a business university.
More informationGROUP SPARSE OPTIMIZATION BY ALTERNATING DIRECTION METHOD. April 19, 2011
GROUP SPARSE OPTIMIZATION BY ALTERNATING DIRECTION METHOD WEI DENG, WOTAO YIN, AND YIN ZHANG April 19, 2011 Abstract. This paper proposes efficient algorithms for group sparse optimization with mied l
More informationNotes on Cholesky Factorization
Notes on Cholesky Factorization Robert A. van de Geijn Department of Computer Science Institute for Computational Engineering and Sciences The University of Texas at Austin Austin, TX 78712 rvdg@cs.utexas.edu
More informationBlind Deconvolution of Corrupted Barcode Signals
Blind Deconvolution of Corrupted Barcode Signals Everardo Uribe and Yifan Zhang Advisors: Ernie Esser and Yifei Lou Interdisciplinary Computational and Applied Mathematics Program University of California,
More informationRoots of Equations (Chapters 5 and 6)
Roots of Equations (Chapters 5 and 6) Problem: given f() = 0, find. In general, f() can be any function. For some forms of f(), analytical solutions are available. However, for other functions, we have
More informationComputing a Nearest Correlation Matrix with Factor Structure
Computing a Nearest Correlation Matrix with Factor Structure Nick Higham School of Mathematics The University of Manchester higham@ma.man.ac.uk http://www.ma.man.ac.uk/~higham/ Joint work with Rüdiger
More informationDual Methods for Total VariationBased Image Restoration
Dual Methods for Total VariationBased Image Restoration Jamylle Carter Institute for Mathematics and its Applications University of Minnesota, Twin Cities Ph.D. (Mathematics), UCLA, 2001 Advisor: Tony
More informationTutorial: Operations Research in Constraint Programming
Tutorial: Operations Research in Constraint Programming John Hooker Carnegie Mellon University May 2009 Revised June 2009 May 2009 Slide 1 Motivation Benders decomposition allows us to apply CP and OR
More informationDefinition of a Linear Program
Definition of a Linear Program Definition: A function f(x 1, x,..., x n ) of x 1, x,..., x n is a linear function if and only if for some set of constants c 1, c,..., c n, f(x 1, x,..., x n ) = c 1 x 1
More informationFactorization Theorems
Chapter 7 Factorization Theorems This chapter highlights a few of the many factorization theorems for matrices While some factorization results are relatively direct, others are iterative While some factorization
More informationAnalysis of Approximation Algorithms for kset Cover using FactorRevealing Linear Programs
Analysis of Approximation Algorithms for kset Cover using FactorRevealing Linear Programs Stavros Athanassopoulos, Ioannis Caragiannis, and Christos Kaklamanis Research Academic Computer Technology Institute
More informationApplied Algorithm Design Lecture 5
Applied Algorithm Design Lecture 5 Pietro Michiardi Eurecom Pietro Michiardi (Eurecom) Applied Algorithm Design Lecture 5 1 / 86 Approximation Algorithms Pietro Michiardi (Eurecom) Applied Algorithm Design
More information9. Numerical linear algebra background
Convex Optimization Boyd & Vandenberghe 9. Numerical linear algebra background matrix structure and algorithm complexity solving linear equations with factored matrices LU, Cholesky, LDL T factorization
More informationSection 7.2 Linear Programming: The Graphical Method
Section 7.2 Linear Programming: The Graphical Method Man problems in business, science, and economics involve finding the optimal value of a function (for instance, the maimum value of the profit function
More informationChapter 6: Sensitivity Analysis
Chapter 6: Sensitivity Analysis Suppose that you have just completed a linear programming solution which will have a major impact on your company, such as determining how much to increase the overall production
More informationTaylor Series and Asymptotic Expansions
Taylor Series and Asymptotic Epansions The importance of power series as a convenient representation, as an approimation tool, as a tool for solving differential equations and so on, is pretty obvious.
More informationTD(0) Leads to Better Policies than Approximate Value Iteration
TD(0) Leads to Better Policies than Approximate Value Iteration Benjamin Van Roy Management Science and Engineering and Electrical Engineering Stanford University Stanford, CA 94305 bvr@stanford.edu Abstract
More informationNonlinear Optimization: Algorithms 3: Interiorpoint methods
Nonlinear Optimization: Algorithms 3: Interiorpoint methods INSEAD, Spring 2006 JeanPhilippe Vert Ecole des Mines de Paris JeanPhilippe.Vert@mines.org Nonlinear optimization c 2006 JeanPhilippe Vert,
More informationRouting in Line Planning for Public Transport
KonradZuseZentrum für Informationstechnik Berlin Takustraße 7 D14195 BerlinDahlem Germany MARC E. PFETSCH RALF BORNDÖRFER Routing in Line Planning for Public Transport Supported by the DFG Research
More information15.062 Data Mining: Algorithms and Applications Matrix Math Review
.6 Data Mining: Algorithms and Applications Matrix Math Review The purpose of this document is to give a brief review of selected linear algebra concepts that will be useful for the course and to develop
More informationIntroduction to Support Vector Machines. Colin Campbell, Bristol University
Introduction to Support Vector Machines Colin Campbell, Bristol University 1 Outline of talk. Part 1. An Introduction to SVMs 1.1. SVMs for binary classification. 1.2. Soft margins and multiclass classification.
More informationSHARP BOUNDS FOR THE SUM OF THE SQUARES OF THE DEGREES OF A GRAPH
31 Kragujevac J. Math. 25 (2003) 31 49. SHARP BOUNDS FOR THE SUM OF THE SQUARES OF THE DEGREES OF A GRAPH Kinkar Ch. Das Department of Mathematics, Indian Institute of Technology, Kharagpur 721302, W.B.,
More informationPOLYNOMIAL FUNCTIONS
POLYNOMIAL FUNCTIONS Polynomial Division.. 314 The Rational Zero Test.....317 Descarte s Rule of Signs... 319 The Remainder Theorem.....31 Finding all Zeros of a Polynomial Function.......33 Writing a
More informationApplications to Data Smoothing and Image Processing I
Applications to Data Smoothing and Image Processing I MA 348 Kurt Bryan Signals and Images Let t denote time and consider a signal a(t) on some time interval, say t. We ll assume that the signal a(t) is
More information! Solve problem to optimality. ! Solve problem in polytime. ! Solve arbitrary instances of the problem. !approximation algorithm.
Approximation Algorithms Chapter Approximation Algorithms Q Suppose I need to solve an NPhard problem What should I do? A Theory says you're unlikely to find a polytime algorithm Must sacrifice one of
More informationPolynomial Degree and Finite Differences
CONDENSED LESSON 7.1 Polynomial Degree and Finite Differences In this lesson you will learn the terminology associated with polynomials use the finite differences method to determine the degree of a polynomial
More informationA Network Flow Approach in Cloud Computing
1 A Network Flow Approach in Cloud Computing Soheil Feizi, Amy Zhang, Muriel Médard RLE at MIT Abstract In this paper, by using network flow principles, we propose algorithms to address various challenges
More informationMathematical finance and linear programming (optimization)
Mathematical finance and linear programming (optimization) Geir Dahl September 15, 2009 1 Introduction The purpose of this short note is to explain how linear programming (LP) (=linear optimization) may
More informationMATH 590: Meshfree Methods
MATH 590: Meshfree Methods Chapter 7: Conditionally Positive Definite Functions Greg Fasshauer Department of Applied Mathematics Illinois Institute of Technology Fall 2010 fasshauer@iit.edu MATH 590 Chapter
More informationLinear Programming I
Linear Programming I November 30, 2003 1 Introduction In the VCR/guns/nuclear bombs/napkins/star wars/professors/butter/mice problem, the benevolent dictator, Bigus Piguinus, of south Antarctica penguins
More informationOPRE 6201 : 2. Simplex Method
OPRE 6201 : 2. Simplex Method 1 The Graphical Method: An Example Consider the following linear program: Max 4x 1 +3x 2 Subject to: 2x 1 +3x 2 6 (1) 3x 1 +2x 2 3 (2) 2x 2 5 (3) 2x 1 +x 2 4 (4) x 1, x 2
More information1 Introduction. Linear Programming. Questions. A general optimization problem is of the form: choose x to. max f(x) subject to x S. where.
Introduction Linear Programming Neil Laws TT 00 A general optimization problem is of the form: choose x to maximise f(x) subject to x S where x = (x,..., x n ) T, f : R n R is the objective function, S
More informationBranchandPrice Approach to the Vehicle Routing Problem with Time Windows
TECHNISCHE UNIVERSITEIT EINDHOVEN BranchandPrice Approach to the Vehicle Routing Problem with Time Windows Lloyd A. Fasting May 2014 Supervisors: dr. M. Firat dr.ir. M.A.A. Boon J. van Twist MSc. Contents
More informationSegmentationFree Dynamic Scene Deblurring
SegmentationFree Dynamic Scene Deblurring Tae Hyun Kim and Kyoung Mu Lee Department of ECE, ASRI, Seoul National University, 151742, Seoul, Korea {lliger9, kyoungmu}@snu.ac.kr, http://cv.snu.ac.kr (a)
More informationDuality in Linear Programming
Duality in Linear Programming 4 In the preceding chapter on sensitivity analysis, we saw that the shadowprice interpretation of the optimal simplex multipliers is a very useful concept. First, these shadow
More informationOn the representability of the biuniform matroid
On the representability of the biuniform matroid Simeon Ball, Carles Padró, Zsuzsa Weiner and Chaoping Xing August 3, 2012 Abstract Every biuniform matroid is representable over all sufficiently large
More informationPerspective Shape from Shading and Viscosity Solutions
Perspective Shape from Shading and Viscosity Solutions E.Prados O.Faugeras ODYSSEE Lab. INRIA  ENS Paris  ENPC/CERMICS http://wwwsop.inria.fr/odyssee/team ICCV 04, 15th October 2003, Nice/France Abstract
More informationThe Goldberg Rao Algorithm for the Maximum Flow Problem
The Goldberg Rao Algorithm for the Maximum Flow Problem COS 528 class notes October 18, 2006 Scribe: Dávid Papp Main idea: use of the blocking flow paradigm to achieve essentially O(min{m 2/3, n 1/2 }
More informationSupport Vector Machines
Support Vector Machines Charlie Frogner 1 MIT 2011 1 Slides mostly stolen from Ryan Rifkin (Google). Plan Regularization derivation of SVMs. Analyzing the SVM problem: optimization, duality. Geometric
More informationMath 115A HW4 Solutions University of California, Los Angeles. 5 2i 6 + 4i. (5 2i)7i (6 + 4i)( 3 + i) = 35i + 14 ( 22 6i) = 36 + 41i.
Math 5A HW4 Solutions September 5, 202 University of California, Los Angeles Problem 4..3b Calculate the determinant, 5 2i 6 + 4i 3 + i 7i Solution: The textbook s instructions give us, (5 2i)7i (6 + 4i)(
More informationA LagrangianDNN Relaxation: a Fast Method for Computing Tight Lower Bounds for a Class of Quadratic Optimization Problems
A LagrangianDNN Relaxation: a Fast Method for Computing Tight Lower Bounds for a Class of Quadratic Optimization Problems Sunyoung Kim, Masakazu Kojima and KimChuan Toh October 2013 Abstract. We propose
More informationSolving Systems of Linear Equations
LECTURE 5 Solving Systems of Linear Equations Recall that we introduced the notion of matrices as a way of standardizing the expression of systems of linear equations In today s lecture I shall show how
More informationThe Steepest Descent Algorithm for Unconstrained Optimization and a Bisection Linesearch Method
The Steepest Descent Algorithm for Unconstrained Optimization and a Bisection Linesearch Method Robert M. Freund February, 004 004 Massachusetts Institute of Technology. 1 1 The Algorithm The problem
More informationmost 4 Mirka Miller 1,2, Guillermo PinedaVillavicencio 3, The University of Newcastle Callaghan, NSW 2308, Australia University of West Bohemia
Complete catalogue of graphs of maimum degree 3 and defect at most 4 Mirka Miller 1,2, Guillermo PinedaVillavicencio 3, 1 School of Electrical Engineering and Computer Science The University of Newcastle
More informationNonlinear Iterative Partial Least Squares Method
Numerical Methods for Determining Principal Component Analysis Abstract Factors Béchu, S., RichardPlouet, M., Fernandez, V., Walton, J., and Fairley, N. (2016) Developments in numerical treatments for
More informationPRIME FACTORS OF CONSECUTIVE INTEGERS
PRIME FACTORS OF CONSECUTIVE INTEGERS MARK BAUER AND MICHAEL A. BENNETT Abstract. This note contains a new algorithm for computing a function f(k) introduced by Erdős to measure the minimal gap size in
More informationNMR Measurement of T1T2 Spectra with Partial Measurements using Compressive Sensing
NMR Measurement of T1T2 Spectra with Partial Measurements using Compressive Sensing Alex Cloninger Norbert Wiener Center Department of Mathematics University of Maryland, College Park http://www.norbertwiener.umd.edu
More informationNumerical Analysis Lecture Notes
Numerical Analysis Lecture Notes Peter J. Olver 5. Inner Products and Norms The norm of a vector is a measure of its size. Besides the familiar Euclidean norm based on the dot product, there are a number
More informationSECTION 14 Absolute Value in Equations and Inequalities
14 Absolute Value in Equations and Inequalities 37 SECTION 14 Absolute Value in Equations and Inequalities Absolute Value and Distance Absolute Value in Equations and Inequalities Absolute Value and
More informationRSVM: Reduced Support Vector Machines
page 1 RSVM: Reduced Support Vector Machines YuhJye Lee and Olvi L. Mangasarian 1 Introduction Abstract An algorithm is proposed which generates a nonlinear kernelbased separating surface that requires
More information