# Diagonal preconditioning for first order primal-dual algorithms in convex optimization

Save this PDF as:

Size: px
Start display at page:

Download "Diagonal preconditioning for first order primal-dual algorithms in convex optimization"

## Transcription

1 Diagonal preconditioning for first order primal-dual algorithms in conve optimization Thomas Pock Institute for Computer Graphics and Vision Graz University of Technology Antonin Chambolle CMAP & CNRS École Polytechnique Abstract In this paper we study preconditioning techniques for the first-order primal-dual algorithm proposed in [5]. In particular, we propose simple and easy to compute diagonal preconditioners for which convergence of the algorithm is guaranteed without the need to compute any step size parameters. As a by-product, we show that for a certain instance of the preconditioning, the proposed algorithm is equivalent to the old and widely unknown alternating step method for monotropic programg [7]. We show numerical results on general linear programg problems and a few standard computer vision problems. In all eamples, the preconditioned algorithm significantly outperforms the algorithm of [5].. Introduction In [5, 8, 3] first-order primal-dual algorithms are studied to solve a certain class of conve optimization problems with known saddle-point structure. ma K, y + G) F y), ) X y Y where X and Y are finite-dimensional vector spaces equipped with standard inner products,. K : X Y is a linear operator and G : X R } and F : Y R } are conve functions with known structure. The iterates of the algorithm studied in [5] to solve ) are very simple: k+ =I + τ G) k τk T y k ) y k+ =I + σ F ) y k + σk k+ + θ k+ k ))) ) They basically consist of alternating a gradient ascend in the dual variable and a gradient descend in the primal The first author acknowledges support from the Austrian Science Fund FWF) under the grant P49-N3. Figure. On problems with irregular structure, the proposed preconditioned algorithm P-PD) converges significantly faster than the algorithm of [5] PD). variable. Additionally, the algorithm performs an overrelaation step in the primal variable. A fundamental assumption of the algorithm is that the functions F and G are of simple structure, meaning that the so-called proimity or resolvent operators I + σ F ) and I + τ G) have closed-form solutions or can be efficiently computed with a high precision. Their eact definitions will be given in Section.. The parameters τ, σ > 0 are the primal and dual step sizes and θ [0, ] controls the amount of overrelaation in. It is shown in [5] that the algorithm converges as long as θ = and the primal and dual step sizes τ and σ are chosen such that τσl <, where L = K is the operator norm of K. It is further shown that a suitably defined partial primal-dual gap of the average of the sequence 0, y 0 ),..., k, y k )) vanishes with rate O/k) for the complete class of problems covered by ). For problems with more regularity, the authors propose acceleration schemes based on non-empirical choices on τ, σ and θ. In particular they show that they can achieve O/k ) for problems where G of F is uniformly conve and Oω k ), ω < for problems where both G and F are uniformly conve. See [5] for more details. A common feature of all numerical eamples in [5] is that the involved linear operators K have a simple structure which makes it very easy to estimate L. We observed that for problems where the operator K has a more compli-

2 cated structure, L cannot be estimated easily or it might be very large such that the convergence of the algorithm significantly slows down. As we will see, linear operators with irregular structure frequently arise in many different vision problems. In this work, we study preconditioning techniques for the primal-dual algorithm ). This allows us to overcome the aforementioned shortcogs. The proposed preconditioned algorithm has several advantages. Firstly, it avoids the estimation of the operator norm of K, secondly, it significantly accelerates the convergence on problems with irregular K and thirdly, it leaves the computational compleity of the iterations basically unchanged. Figure shows convergence plots on two LP problems with such an irregular structure. The proposed algorithm can better adapt to the problem structure, leading to faster convergence. The rest of the paper is as follows. In Section. we fi some preliary definitions which will be used throughout the paper. In Section we present the preconditioned primal-dual algorithm and give conditions under which convergence of the algorithm is guaranteed. We propose a family of simple and easy to compute diagonal preconditioners, which turn out to be very efficient on many problems. In Section.3 we establish connections to the old and widely unknown alternating step method for monotropic programg [7]. In Section 3 we detail eperimental results of the proposed algorithm. In the last Section we draw some conclusions and show directions for future work... Preliaries We consider finite-dimensional vector spaces X and Y, where n = dim X and m = dim Y with inner products, X = T,,, X, y, y Y = Σ y, y, y, y Y, where T and Σ are a symmetric, positive definite preconditioning matrices. We further define the norms in the usual way as X =, X, y Y = y, y Y. We will make frequent use of the so-called resolvent or proimity operator of a function G). Given a point ˆ X, it is defined as the solution of the auiliary imization problem = arg G) + ˆ X The unique imizer to the above problem is characterized by the optimality condition G) + T ˆ) 0, whose optimal solution can be written in operator form as = I + T G) ˆ). 3). Preconditioned primal-dual algorithm In this work, we propose the following preconditioned first-order primal-dual algorithm: Choose symmetric and positive definite matrices T, Σ, θ [0, ], 0, y 0 ) X Y. Then for k 0, update k, y k ) as follows: k+ =I + T G) k TK T y k ) y k+ =I + Σ F ) y k + ΣK k+ +θ k+ k ))) 4) Comparing the iterates 4) of the proposed algorithm to ), one can see that the global steps τ and σ have been replaced by the preconditioning matrices T and Σ. It is known that ) converges as long as θ = and τσ K <. Hence, a natural question is now to establish conditions on T and Σ and θ which ensure convergence of the proposed preconditioned algorithm. In very recent work [0], it has been shown that the iterates ) can be written in form of a proimal point algorithm [4], which greatly simplifies the convergence analysis. From the optimality conditions of the iterates 4) and the conveity of G and F it follows that for any, y) X Y the iterates k+ and y k+ satisfy ) ) ) k+ k+ y y k+,f y k+ +M k+ k y k+ y k 0, where ) ) k+ G F y k+ = k+ ) + K T y k+ F y k+ ) K k+, and [ T K M = T θk Σ 5) ]. 6) It is easy to check, that the variational inequality 5) now takes the form of a proimal point algorithm [0, 4, 6]. In the net Section we will establish conditions on θ, T and Σ which ensure convergence of the algorithm... Convergence of the algorithm We can make direct use of the convergence analysis developed in [0, 4, 6]. In fact, convergence of 5) can be guaranteed as long as the matri M is symmetric and positive definite. In the following Lemma we establish conditions on θ, T and Σ which indeed ensure these properties of M. Lemma. Let θ =, T and Σ symmetric positive definite maps satisfying Σ KT <, 7)

3 then the matri M defined in 6) is symmetric and positive definite, more precisely it holds, y) T, M, y) T > 0, 8) for any, y) T X Y and, y) T 0. Proof. Due to the structure of M, symmetry follows directly from the symmetry of T, Σ and the fact that θ =. Epanding the inner product in 8) we have T, + Σ y, y K, y > 0 X + y Y K, y > 0. 9) Since T and Σ are symmetric and positive definite, the above scalar product can be rewritten as K, y = Σ KT T, Σ y Using the Cauchy-Schwarz inequality and the fact that ab ca + b /c for any a, b and c > 0, we obtain K, y Σ KT T Σ y c Σ KT X + ) c y Y In view of 7), it is clear that there eists a ε > 0 such that it holds + ε) Σ KT =. With this relation, we can perform suitable choices of c and /c such that our estimate becomes ) Σ KT X K, y + ε) Σ KT + y Y + ε) ) X + y ) Y. + ε Inserting this estimate into 9), we obtain ε +ε ) X + y Y ) > 0. Since ε > 0, it becomes clear that the condition 7) ensures positive definiteness of M. Remark. Choosing T = τ I and Σ = σi, for some τ, σ > 0, the assertion 7) reduces to τσ K <, which is the original condition on the step widths τ and σ to ensure convergence of the primal-dual algorithm [5]. Note that this criterion requires to compute or to estimate) the operator norm of K which can be difficult in some situations. Theorem. Let θ = and T, Σ be symmetric and positive definite matrices such that Σ KT <. Then, the sequence n, y n ) generated by the iterates 4) converges weakly to an optimal solution, y ) of the saddle-point problem ). Proof. As shown in Lemma, the conditions θ = and Σ KT < ensure that the matri M defined in 6) is symmetric and positive definite. Hence, the variational inequality 5) takes the form of a proimal point algorithm. Convergence of the iterates 4) to an optimal solution, y ) of ) then follows from the weak convergence of the proimal point algorithm Theorem of [4]). In the net Section we study simple diagonal preconditioners which obey 7) and hence ensure the convergence of the algorithm. Furthermore, computing these preconditioners only requires to compute row and column wise norms of K, which can be done very efficiently... A family of diagonal preconditioners In general, Σ and T could be any symmetric, positive definite maps. However, since it is a fundamental requirement of the proposed algorithm, that the resolvent operators are simple, some care has to be taken in choosing T and Σ. In particular, if G and F are separable in j and y i, the resolvent operators remain simple, if Σ and T are restricted to diagonal matrices. In the following we study a family of preconditioners which satisfy all these requirements and additionally guarantee the convergence of the algorithm. Lemma. Let T = diagτ ), where τ = τ,...τ n ) and Σ = diagσ), where σ = σ,..., σ m ). In particular, τ j = m i= K i,j, σ α i = n j= K 0) i,j α then for any α [0, ] Σ KT Σ KT = sup X, 0. ) Proof. In order to prove the inequality, we need to find an upper bound on Σ KT. We have Σ KT = Σ KT, Σ KT m n = σi K i,j τj j = i= m i= m i= σ i σ i j= n K i,j τj j j= n K i,j α Ki,j α τ j j j= Then, by applying the Cauchy-Schwarz inequality we obtain m n n Σ KT σ i K i,j α K i,j α τ j j i= j= j=

4 By definition of σ i and τ j, the above inequality can be simplified to Σ KT = m i= j= n K i,j α τ j j n m ) K i,j α τ j j =. j= i= Substituting back into the definition of the operator norm, we finally obtain Σ KT Σ KT = sup X, 0 = ) Remark. To ensure positive definiteness in 8) inequality ) has to be strict. This requirement can be easily satisfied by multiplying τ and σ by appropriate positive constants µ and ν such that µν <. However, in practice, we did not observe any convergence problems of the algorithm for µ = ν =. Remark 3. Basically, the diagonal preconditioning leads to dimension-dependent times steps τ j and σ i instead of global steps τ and σ of the algorithm in [5] and hence do not change the computational compleity of the algorithm..3. A remark on the alternating step method A monotropic program is defined as a conve optimization problem of the form h), s.t. A = b, 3) where R n, h) = n j= h j j ) with h j ) being conve functions, A R m n and b R m. Let us denote by a j the j th column of A, by r i ) the i th entry of the vector r) = b A and by q i the number of non-zero entries in the i th row of A. In [7], the alternating step method for monotropic programg has been introduced as a special case of the alternating direction method of multipliers. It is defined as the iterative scheme k+ j = arg j h j j ) + a j, π k j + j λ a j [ k j + ]) m a i,jr i k } ) a j i= q i π k+ i = πi k + λ q i r i k+ ) 4) where π R m is the dual variable and λ > 0 is a parameter which weights between the strengths of the primal and dual updates. Let us now cast 3) in the form of ). Indeed, it can be written very easily as a saddle-point problem by introducing Lagrange multipliers y for the constraint A = b, giving us ma b A, y + h). 5) y The iterates 4) now become k+ = I + T h) k + TA T y k ) y k+ = y k + Σb A k+ k )) 6) Let us now perform the particular choice α = 0 in 0) to compute T, Σ and perform an additional scaling by the factors /λ and λ, respectively see Remark ). We consequently find that τ j = λ a j and σ i = λ q i, where we used the convention that 0 0 = 0. By a change of variables in the duals, π k = y k Σb A k ), it can be checked that in this particular setting, 6) is equivalent to the alternating step method 4). 3. Numerical results In this Section we perform several numerical eperiments using the proposed algorithm. We first show applications to general linear programg problems and show that the proposed algorithm leads to a significantly faster convergence compared to the algorithm in [5]. Furthermore, we show that for large scale problems, a simple Matlab implementation of the proposed algorithm significantly outperforms a highly optimized interior point solver of Matlab. Finally, we show applications of the proposed algorithm to classical computer vision problems such as graph cuts and imal partitioning problems, where we show that a GPU implementation of the proposed algorithm can easily compete with specialized algorithms such as the maflow algorithm of Boykov and Kolmogorov [3]. In our numerical eperiments, we use the following algorithms and parameter settings: IP: Matlab s LP interior point solver linprog with settings optimized for large scale problems. PD: The primal-dual algorithm ) where we set τ = L and σ = L and L = K is estimated using the Matlab command normest. P-PD: Preconditioned primal dual algorithm 4) with the preconditioners T and Σ defined in 0) and using α =. In general, the solutions of the problems we consider here are not unique. Therefore, we use the value of the objective functions as a criterion to detere the convergence

5 of the algorithms. We define the relative error of the objective function as δ k = E E k / E, where E refers to the optimal value and E k is the value of the current iterate. Depending of the particular problem, E might be the value of either the primal or the dual problem. In order to detere the optimal value of the objective function we run one method with a very high accuracy. During the evaluation, we run the algorithms until δ k is below a fied threshold of tol = 0 4. Note that for practical purposes, δ k can also be bounded by the primal-dual gap. All eperiments were eecuted on a.73 GHz i7 CPU running Matlab and a NVidia GTX480 GPU using the CUDA framework. 3.. Linear programg Linear programs LPs) define an important class of optimization problems in computer vision. See for eample the recent work [5] for a huge scale) LP formulation of curvature dependent segmentation and inpainting models. Therefore, there is a strong demand for efficient solver which can handle very large problems. Here, we consider LPs in so-called equality form: c T s.t. A = b, 0, 7) where, c R n, b R m and A R m n. Note that it is straight-forward to transform other standard forms to this form []. The above LP can be easily reformulated in terms of a saddle-point problem by introducing Lagrange multipliers y to account for the equality constraints: ma A b, y + c,, s.t. 0 y Applying P-PD to 7) we obtain the following simple iterates k+ = proj [0, ) k TA T y k + c) ) y k+ = y k + ΣA k+ k ) b), 8) where proj [0, ) ) can be implemented by simple truncation operations. In our first eample, we compare IP, PD and P-PD on two standard LP test problems that come along with the Matlab package. To gain more insight to the proposed preconditioning technique, we compare the global steps of PD with the dimension-dependent adaptive steps of P-PD. sc50b: This problem consists of 48 variables, 30 inequalities and 0 equalities. We find L = and set τ = σ = for PD. For P-PD we find that the average primal step is τ = and the average dual step is σ = Therefore, P-PD can take in average approimately.8 times larger primal and dual steps than PD. densecolumn: This problem consists of 677 variables and 67 equalities with one dense column in the linear operator. We find L = and consequently set τ = σ = for PD. For P-PD, we find the average primal and dual steps to be τ = and σ = Hence P-PD can take on average approimately 40 times larger primal and 5 times larger dual steps. Table shows the results of the performance evaluation on the standard LP test problems. The optimal value of the objective function was detered by running IP with a very high accuracy. This eamples clearly demonstrate the advantage of the proposed algorithm. On sc50b P- PD is.4 times faster than PD while on densecolumn, P-PD is more than 00 times faster than PD. Especially, densecolumn has a very irregular structure which can be handled much better by the preconditioned algorithm. Clearly, on small-scale problems, the simple primal-dual algorithms can not compete with the highly optimized IP. However, as we will see in the net section, this is quite different for large-scale problems. Figure depicts convergence plots for PD and P-PD for the two LP test problems. IP PD P-PD sc50b 0.0s.75s 0.49s densecolumn 0.7s 68.5s 0.6s Table. Comparison of of IP, PD and P-PD on two standard LP test problems. 3.. TV-L image denoising The TV-L model [, 6] is a popular image denoising model. It consists of total variation TV) regularization and a L data fidelity term. It is known that the TV-L model is very effective in removing strong outliers such as salt and pepper noise in images. We assume discrete images defined on a regular Cartesian grid of size M N with indices i, j). The anisotropic) TV-L model in a discrete setting is defined as the l imization problem u Du l + λ u f l, 9) where f = f i,j ) R MN is the noisy input image and u = u i,j ) R MN is the denoised image. The free parameter λ > 0 is used to control the trade-off between smoothing and data fidelity. The linear operator D R MN MN is the discretized gradient operator defined by ) Du) Du) i,j = i,j Du), 0) i,j where Du) i,j = u i+,j u i,j if i < M 0 if i = M,

6 a) 3 phases b) Output c) 4 phases d) Output e) Natural f) Segmentation 8 phases) Figure 3. Figures a-b) and c-d) show the result of standard Dirichlet problems. Figures e-f) show the segmentation of a natural image into 8 regions. usual isotropic total variation. The only modification is to replace the projections of the duals onto bo constraints by projections onto l balls. Figure shows the result of PD and P-PD applied to a binary image segmentation problem. Let I [0, ] 3MN be the input image. The unary terms were computed by first computing the average RGB values µ f R 3 and µ b R 3 in the user specified foreground and background regions and then by setting wi,j u = α I i,j µ f I i,j µ b ). The binary terms were computed by w b, i,j = ep β I i+,j I i,j ) and w b, i,j = ep β I i,j+ I i,j ), where we set α = and β = 0. We used the popular maimum flow algorithm of Boykov and Kolmogorov [3] to compute the optimal value of the primal objective function ). We then ran PD and P-PD until the relative error δ k was below tol. Table 3 presents the tigs. Although a Matlab implementation of P-PD cannot compete with the highly optimized maimum flow algorithm, a parallel GPU implementation is already significantly faster. MAXFLOW PD P-PD P-PD-GPU 0.60s 5.75s 8.56s 0.045s Table 3. Comparison of the proposed algorithm for graph cut segmentation Minimal partitions In the last eample we quickly eplain the application of the proposed preconditioned primal-dual algorithm for computing imal partitions. Following [5] and references therein, a conve relaation of the multi-label Potts model can be computed by solving the saddle point problem where S = ma u S q B k Du l, q l + u l, f l, 3) l= u = u l ) k l= : } k u l =, u l 0, l= 4) is the usual simple constraint and the conve set B = Bm n, 5) m<n k can be written as the intersection of balls of the form Bm n = q = ql ), ql )) k l= : q m q n ) i,j, i, j }, 6) The linear operator D is the discretized gradient operator as defined in 0), u = u l ) k l=, is the labeling function which represents a relaed assignment of the image piels to each region, f = f l ) k l= is some given weight function and q = ql ), q l ))k l= are the dual variables. The original approach taken in [5] is to compute the projections with respect to S and B by solving auiliary optimization problems. This approach has the disadvantage that the inner optimization problems have to be computed with sufficient accuracy such that the overall algorithm converges, which can heavily slow down the algorithm. Here, we take a different approach, which is closely related to the multiple operator splitting method proposed in []. We first introduce slack variables b for the epressions q m q n in 6). Then we introduce Lagrange multipliers λ to account for the equality constraints l u l = in 4) and µ to account for the equality constraints b n m = q m q n. Together, this yields the following augmented saddle-point problem u,µ ma q,b,λ l k= Du l, q l + u l, f l + P u, λ + b Qq, µ s.t. u l ) i,j 0, b n m) i,j, 7) where the linear operators P and Q are such that P u) i,j = k u l ) i,j, Qq) n m) i,j = q m q n ) i,j. l= Note that 7) now depends only on simple pointwise constraints for which the projections can be implemented very efficiently. To cast the above problem into the form of ), it remains to arrange all primal variables in a vector, all

7 dual variables into a vector y and all linear operators into a global linear operator K. Then, applying the preconditioning techniques proposed in this paper leads to an algorithm that is guaranteed to converge to the optimal solution without the need to solve any inner optimization problems. Figure 3 shows some results of standard imal partitioning and segmentation problems. We compared the original approach solving inner optimization problems and using PD to P-PD applied to 7). We first precomputed the optimal solution using a large number of iterations and then recorded the time until the error is below a threshold of tol. The tigs are presented in Table 4. In all cases, the proposed algorithm clearly outperforms the original approach of [5]. PD P-PD Speedup Synthetic 3 phases).7s 75.65s.9 Synthetic 4 phases) 39.0s s.6 Natural 8 phases) 59.85s 3.76s 5. Table 4. Comparison of the proposed algorithm on partitioning problems. 4. Conclusion In this paper we have proposed a simple preconditioning technique to improve the performance of the first-order primal-dual algorithm proposed in [3, 5]. The proposed diagonal preconditioners can be computed efficiently and guarantee the convergence of the algorithm without the need to estimate any step size parameters. In several numerical eperiments, we have shown that the proposed algorithm significantly outperforms the algorithm in [5]. Furthermore, on large scale linear programg problems, an unoptimized implementation of the proposed algorithm easily outperforms a highly optimized interior point solver and a GPU implementation of the proposed algorithm can easily compete with specialized combinatorial algorithms for computing imum cuts. We believe that the proposed algorithm can become a standard algorithm in computer vision since it can be applied to a large class of conve optimization problems arising in computer vision and has the potential for parallel computing. Future work will mainly concentrate on the development of more sophisticated preconditioners that are different from diagonal matrices. References [] A. Bhusnurmath and C. Taylor. Graph cuts via l norm imization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30:866 87, 008. [] S. Boyd and L. Vandenberghe. Conve Optimization. Cambridge University Press, 004. [3] Y. Boykov and V. Kolmogorov. An eperimental comparison of -cut/ma-flow algorithms for energy imization in computer vision. In A. J. M. Figueiredo, J. Zerubia, editor, Energy Minimization Methods in Computer Vision and Pattern Recognition, volume 34 of LNCS, pages Springer, 00. [4] A. Chambolle. Total variation imization and a class of binary MRF models. In Energy Minimization Methods in Computer Vision and Pattern Recognition, pages 36 5, 005. [5] A. Chambolle and T. Pock. A first-order primal-dual algorithm for conve problems with applications to imaging. Journal of Mathematical Imaging and Vision, 00. online first. [6] T. Chan and S. Esedoglu. Aspects of total variation regularized L function approimation. SIAM J. Appl. Math., 655):87 837, 004. [7] J. Eckstein. Splitting Methods for Monotone Operators with Application to Parallel Optimization. PhD thesis, MIT, 989. [8] E. Esser, X. Zhang, and T. Chan. A general framework for a class of first order primal-dual algorithms for conve optimization in imaging science. SIAM J. Imaging Sci., 3:05 046, 00. [9] L. Ford and D. Fulkerson. Flows in Networks. Princeton University Press, Princeton, New Jersey, 96. [0] B. He and X. Yuan. Convergence analysis of primal-dual algorithms for total variation image restoration. Technical report, Nanjing University, China, 00. [] J. Lellmann, D. Breitenreicher, and C. Schnörr. Fast and eact primal-dual iterations for variational problems in computer vision. In K. Daniilidis, P. Maragos, and N. Paragios, editors, European Conference on Computer Vision ECCV), volume 63 of LNCS, pages Springer Berlin / Heidelberg, 00. [] M. Nikolova. A variational approach to remove outliers and impulse noise. Journal of Mathematical Imaging and Vision, 0-):99 0, 004. [3] T. Pock, D. Cremers, H. Bischof, and A. Chambolle. An algorithm for imizing the Mumford-Shah functional. In International Conference on Computer Vision ICCV), 009. [4] R. T. Rockafellar. Monotone operators and the proimal point algorithm. SIAM Journal on Control and Optimization, 45): , 976. [5] T. Schoenemann, F. Kahl, and D. Cremers. Curvature regularity for region-based image segmentation and inpainting: A linear programg relaation. In IEEE International Conference on Computer Vision ICCV), Kyoto, Japan, 009. [6] X. Zhang, M. Burger, and S. Osher. A unified primal-dual algorithm framework based onbregman iteration. Journal of Scientific Computing, 46:0 46, 0.

### Optical Flow. Thomas Pock

Optical Flow Thomas Pock 1 Optical Flow (II) Content Global approaches (Horn-Schunck, TV-L1) Coarse-to-fine warping 2 The Horn and Schunck (HS) Method 3 The Horn and Schunck (HS) Method Global energy to

### Lecture 5: Conic Optimization: Overview

EE 227A: Conve Optimization and Applications January 31, 2012 Lecture 5: Conic Optimization: Overview Lecturer: Laurent El Ghaoui Reading assignment: Chapter 4 of BV. Sections 3.1-3.6 of WTB. 5.1 Linear

### Moving Least Squares Approximation

Chapter 7 Moving Least Squares Approimation An alternative to radial basis function interpolation and approimation is the so-called moving least squares method. As we will see below, in this method the

### Proximal mapping via network optimization

L. Vandenberghe EE236C (Spring 23-4) Proximal mapping via network optimization minimum cut and maximum flow problems parametric minimum cut problem application to proximal mapping Introduction this lecture:

### Regression Using Support Vector Machines: Basic Foundations

Regression Using Support Vector Machines: Basic Foundations Technical Report December 2004 Aly Farag and Refaat M Mohamed Computer Vision and Image Processing Laboratory Electrical and Computer Engineering

### On Convergence Rate of Concave-Convex Procedure

On Convergence Rate of Concave-Conve Procedure Ian En-Hsu Yen r00922017@csie.ntu.edu.tw Po-Wei Wang b97058@csie.ntu.edu.tw Nanyun Peng Johns Hopkins University Baltimore, MD 21218 npeng1@jhu.edu Shou-De

### By W.E. Diewert. July, Linear programming problems are important for a number of reasons:

APPLIED ECONOMICS By W.E. Diewert. July, 3. Chapter : Linear Programming. Introduction The theory of linear programming provides a good introduction to the study of constrained maximization (and minimization)

### c 2006 Society for Industrial and Applied Mathematics

SIAM J. OPTIM. Vol. 17, No. 4, pp. 943 968 c 2006 Society for Industrial and Applied Mathematics STATIONARITY RESULTS FOR GENERATING SET SEARCH FOR LINEARLY CONSTRAINED OPTIMIZATION TAMARA G. KOLDA, ROBERT

### Absolute Value Programming

Computational Optimization and Aplications,, 1 11 (2006) c 2006 Springer Verlag, Boston. Manufactured in The Netherlands. Absolute Value Programming O. L. MANGASARIAN olvi@cs.wisc.edu Computer Sciences

### Duality of linear conic problems

Duality of linear conic problems Alexander Shapiro and Arkadi Nemirovski Abstract It is well known that the optimal values of a linear programming problem and its dual are equal to each other if at least

### The Method of Lagrange Multipliers

The Method of Lagrange Multipliers S. Sawyer October 25, 2002 1. Lagrange s Theorem. Suppose that we want to maximize (or imize a function of n variables f(x = f(x 1, x 2,..., x n for x = (x 1, x 2,...,

### ME128 Computer-Aided Mechanical Design Course Notes Introduction to Design Optimization

ME128 Computer-ided Mechanical Design Course Notes Introduction to Design Optimization 2. OPTIMIZTION Design optimization is rooted as a basic problem for design engineers. It is, of course, a rare situation

### The multi-integer set cover and the facility terminal cover problem

The multi-integer set cover and the facility teral cover problem Dorit S. Hochbaum Asaf Levin December 5, 2007 Abstract The facility teral cover problem is a generalization of the vertex cover problem.

### Modern Optimization Methods for Big Data Problems MATH11146 The University of Edinburgh

Modern Optimization Methods for Big Data Problems MATH11146 The University of Edinburgh Peter Richtárik Week 3 Randomized Coordinate Descent With Arbitrary Sampling January 27, 2016 1 / 30 The Problem

### max cx s.t. Ax c where the matrix A, cost vector c and right hand side b are given and x is a vector of variables. For this example we have x

Linear Programming Linear programming refers to problems stated as maximization or minimization of a linear function subject to constraints that are linear equalities and inequalities. Although the study

### Practical Guide to the Simplex Method of Linear Programming

Practical Guide to the Simplex Method of Linear Programming Marcel Oliver Revised: April, 0 The basic steps of the simplex algorithm Step : Write the linear programming problem in standard form Linear

### Duality in General Programs. Ryan Tibshirani Convex Optimization 10-725/36-725

Duality in General Programs Ryan Tibshirani Convex Optimization 10-725/36-725 1 Last time: duality in linear programs Given c R n, A R m n, b R m, G R r n, h R r : min x R n c T x max u R m, v R r b T

### Chapter 5. The Inverse; Numerical Methods

Vector Spaces in Physics 8/6/ Chapter. The nverse Numerical Methods n the Chapter we discussed the solution of systems of simultaneous linear algebraic equations which could be written in the form C -

### Gradient Methods. Rafael E. Banchs

Gradient Methods Rafael E. Banchs INTRODUCTION This report discuss one class of the local search algorithms to be used in the inverse modeling of the time harmonic field electric logging problem, the Gradient

### Variational approach to restore point-like and curve-like singularities in imaging

Variational approach to restore point-like and curve-like singularities in imaging Daniele Graziani joint work with Gilles Aubert and Laure Blanc-Féraud Roma 12/06/2012 Daniele Graziani (Roma) 12/06/2012

### (57) (58) (59) (60) (61)

Module 4 : Solving Linear Algebraic Equations Section 5 : Iterative Solution Techniques 5 Iterative Solution Techniques By this approach, we start with some initial guess solution, say for solution and

### P164 Tomographic Velocity Model Building Using Iterative Eigendecomposition

P164 Tomographic Velocity Model Building Using Iterative Eigendecomposition K. Osypov* (WesternGeco), D. Nichols (WesternGeco), M. Woodward (WesternGeco) & C.E. Yarman (WesternGeco) SUMMARY Tomographic

Adaptive Online Gradient Descent Peter L Bartlett Division of Computer Science Department of Statistics UC Berkeley Berkeley, CA 94709 bartlett@csberkeleyedu Elad Hazan IBM Almaden Research Center 650

### Chapter 5.1 Systems of linear inequalities in two variables.

Chapter 5. Systems of linear inequalities in two variables. In this section, we will learn how to graph linear inequalities in two variables and then apply this procedure to practical application problems.

### Massive Data Classification via Unconstrained Support Vector Machines

Massive Data Classification via Unconstrained Support Vector Machines Olvi L. Mangasarian and Michael E. Thompson Computer Sciences Department University of Wisconsin 1210 West Dayton Street Madison, WI

### Iterative Techniques in Matrix Algebra. Jacobi & Gauss-Seidel Iterative Techniques II

Iterative Techniques in Matrix Algebra Jacobi & Gauss-Seidel Iterative Techniques II Numerical Analysis (9th Edition) R L Burden & J D Faires Beamer Presentation Slides prepared by John Carroll Dublin

### Inner Product Spaces and Orthogonality

Inner Product Spaces and Orthogonality week 3-4 Fall 2006 Dot product of R n The inner product or dot product of R n is a function, defined by u, v a b + a 2 b 2 + + a n b n for u a, a 2,, a n T, v b,

### Advanced results on variational inequality formulation in oligopolistic market equilibrium problem

Filomat 26:5 (202), 935 947 DOI 0.2298/FIL205935B Published by Faculty of Sciences and Mathematics, University of Niš, Serbia Available at: http://www.pmf.ni.ac.rs/filomat Advanced results on variational

### LINEAR PROGRAMMING P V Ram B. Sc., ACA, ACMA Hyderabad

LINEAR PROGRAMMING P V Ram B. Sc., ACA, ACMA 98481 85073 Hyderabad Page 1 of 19 Question: Explain LPP. Answer: Linear programming is a mathematical technique for determining the optimal allocation of resources

### Introduction and message of the book

1 Introduction and message of the book 1.1 Why polynomial optimization? Consider the global optimization problem: P : for some feasible set f := inf x { f(x) : x K } (1.1) K := { x R n : g j (x) 0, j =

### A Negative Result Concerning Explicit Matrices With The Restricted Isometry Property

A Negative Result Concerning Explicit Matrices With The Restricted Isometry Property Venkat Chandar March 1, 2008 Abstract In this note, we prove that matrices whose entries are all 0 or 1 cannot achieve

### Further Study on Strong Lagrangian Duality Property for Invex Programs via Penalty Functions 1

Further Study on Strong Lagrangian Duality Property for Invex Programs via Penalty Functions 1 J. Zhang Institute of Applied Mathematics, Chongqing University of Posts and Telecommunications, Chongqing

### ! Solve problem to optimality. ! Solve problem in poly-time. ! Solve arbitrary instances of the problem. #-approximation algorithm.

Approximation Algorithms 11 Approximation Algorithms Q Suppose I need to solve an NP-hard problem What should I do? A Theory says you're unlikely to find a poly-time algorithm Must sacrifice one of three

### SOLVING LINEAR SYSTEMS

SOLVING LINEAR SYSTEMS Linear systems Ax = b occur widely in applied mathematics They occur as direct formulations of real world problems; but more often, they occur as a part of the numerical analysis

### 4.6 Linear Programming duality

4.6 Linear Programming duality To any minimization (maximization) LP we can associate a closely related maximization (minimization) LP. Different spaces and objective functions but in general same optimal

### Numerical Methods For Image Restoration

Numerical Methods For Image Restoration CIRAM Alessandro Lanza University of Bologna, Italy Faculty of Engineering CIRAM Outline 1. Image Restoration as an inverse problem 2. Image degradation models:

### Mathematics 31 Pre-calculus and Limits

Mathematics 31 Pre-calculus and Limits Overview After completing this section, students will be epected to have acquired reliability and fluency in the algebraic skills of factoring, operations with radicals

### A fast multi-class SVM learning method for huge databases

www.ijcsi.org 544 A fast multi-class SVM learning method for huge databases Djeffal Abdelhamid 1, Babahenini Mohamed Chaouki 2 and Taleb-Ahmed Abdelmalik 3 1,2 Computer science department, LESIA Laboratory,

### An Overview Of Software For Convex Optimization. Brian Borchers Department of Mathematics New Mexico Tech Socorro, NM 87801 borchers@nmt.

An Overview Of Software For Convex Optimization Brian Borchers Department of Mathematics New Mexico Tech Socorro, NM 87801 borchers@nmt.edu In fact, the great watershed in optimization isn t between linearity

### TESLA Report 2003-03

TESLA Report 23-3 A multigrid based 3D space-charge routine in the tracking code GPT Gisela Pöplau, Ursula van Rienen, Marieke de Loos and Bas van der Geer Institute of General Electrical Engineering,

### Linear Systems. Singular and Nonsingular Matrices. Find x 1, x 2, x 3 such that the following three equations hold:

Linear Systems Example: Find x, x, x such that the following three equations hold: x + x + x = 4x + x + x = x + x + x = 6 We can write this using matrix-vector notation as 4 {{ A x x x {{ x = 6 {{ b General

### December 4, 2013 MATH 171 BASIC LINEAR ALGEBRA B. KITCHENS

December 4, 2013 MATH 171 BASIC LINEAR ALGEBRA B KITCHENS The equation 1 Lines in two-dimensional space (1) 2x y = 3 describes a line in two-dimensional space The coefficients of x and y in the equation

### Example 4.1 (nonlinear pendulum dynamics with friction) Figure 4.1: Pendulum. asin. k, a, and b. We study stability of the origin x

Lecture 4. LaSalle s Invariance Principle We begin with a motivating eample. Eample 4.1 (nonlinear pendulum dynamics with friction) Figure 4.1: Pendulum Dynamics of a pendulum with friction can be written

### Polynomial and Synthetic Division. Long Division of Polynomials. Example 1. 6x 2 7x 2 x 2) 19x 2 16x 4 6x3 12x 2 7x 2 16x 7x 2 14x. 2x 4.

_.qd /7/5 9: AM Page 5 Section.. Polynomial and Synthetic Division 5 Polynomial and Synthetic Division What you should learn Use long division to divide polynomials by other polynomials. Use synthetic

### MATH 304 Linear Algebra Lecture 20: Inner product spaces. Orthogonal sets.

MATH 304 Linear Algebra Lecture 20: Inner product spaces. Orthogonal sets. Norm The notion of norm generalizes the notion of length of a vector in R n. Definition. Let V be a vector space. A function α

### Chapter 2 Limits Functions and Sequences sequence sequence Example

Chapter Limits In the net few chapters we shall investigate several concepts from calculus, all of which are based on the notion of a limit. In the normal sequence of mathematics courses that students

### Nonnegative Matrix Factorization: Algorithms, Complexity and Applications

Nonnegative Matrix Factorization: Algorithms, Complexity and Applications Ankur Moitra Massachusetts Institute of Technology July 6th, 2015 Ankur Moitra (MIT) NMF July 6th, 2015 m M n W m M = A inner dimension

### Big Data - Lecture 1 Optimization reminders

Big Data - Lecture 1 Optimization reminders S. Gadat Toulouse, Octobre 2014 Big Data - Lecture 1 Optimization reminders S. Gadat Toulouse, Octobre 2014 Schedule Introduction Major issues Examples Mathematics

### MATH 2030: MATRICES. Av = 3

MATH 030: MATRICES Introduction to Linear Transformations We have seen that we ma describe matrices as smbol with simple algebraic properties like matri multiplication, addition and scalar addition In

### Eigenvalues of C SMS (16) 1 A. Eigenvalues of A (32) 1 A (16,32) 1 A

Negative Results for Multilevel Preconditioners in Image Deblurring C. R. Vogel Department of Mathematical Sciences Montana State University Bozeman, MT 59717-0240 USA Abstract. A one-dimensional deconvolution

### Support Vector Machine. Tutorial. (and Statistical Learning Theory)

Support Vector Machine (and Statistical Learning Theory) Tutorial Jason Weston NEC Labs America 4 Independence Way, Princeton, USA. jasonw@nec-labs.com 1 Support Vector Machines: history SVMs introduced

### Diagonal, Symmetric and Triangular Matrices

Contents 1 Diagonal, Symmetric Triangular Matrices 2 Diagonal Matrices 2.1 Products, Powers Inverses of Diagonal Matrices 2.1.1 Theorem (Powers of Matrices) 2.2 Multiplying Matrices on the Left Right by

### Completely Positive Cone and its Dual

On the Computational Complexity of Membership Problems for the Completely Positive Cone and its Dual Peter J.C. Dickinson Luuk Gijben July 3, 2012 Abstract Copositive programming has become a useful tool

### 2.5 Gaussian Elimination

page 150 150 CHAPTER 2 Matrices and Systems of Linear Equations 37 10 the linear algebra package of Maple, the three elementary 20 23 1 row operations are 12 1 swaprow(a,i,j): permute rows i and j 3 3

### 1.5 SOLUTION SETS OF LINEAR SYSTEMS

1-2 CHAPTER 1 Linear Equations in Linear Algebra 1.5 SOLUTION SETS OF LINEAR SYSTEMS Many of the concepts and computations in linear algebra involve sets of vectors which are visualized geometrically as

### Linear Systems COS 323

Linear Systems COS 323 Last time: Constrained Optimization Linear constrained optimization Linear programming (LP) Simplex method for LP General optimization With equality constraints: Lagrange multipliers

### Section 3-3 Approximating Real Zeros of Polynomials

- Approimating Real Zeros of Polynomials 9 Section - Approimating Real Zeros of Polynomials Locating Real Zeros The Bisection Method Approimating Multiple Zeros Application The methods for finding zeros

### Chapter 15 Introduction to Linear Programming

Chapter 15 Introduction to Linear Programming An Introduction to Optimization Spring, 2014 Wei-Ta Chu 1 Brief History of Linear Programming The goal of linear programming is to determine the values of

### 3. Linear Programming and Polyhedral Combinatorics

Massachusetts Institute of Technology Handout 6 18.433: Combinatorial Optimization February 20th, 2009 Michel X. Goemans 3. Linear Programming and Polyhedral Combinatorics Summary of what was seen in the

### The Trimmed Iterative Closest Point Algorithm

Image and Pattern Analysis (IPAN) Group Computer and Automation Research Institute, HAS Budapest, Hungary The Trimmed Iterative Closest Point Algorithm Dmitry Chetverikov and Dmitry Stepanov http://visual.ipan.sztaki.hu

### (a) The transpose of a lower triangular matrix is upper triangular, and the transpose of an upper triangular matrix is lower triangular.

Theorem.7.: (Properties of Triangular Matrices) (a) The transpose of a lower triangular matrix is upper triangular, and the transpose of an upper triangular matrix is lower triangular. (b) The product

### Lecture 7 - Linear Programming

COMPSCI 530: Design and Analysis of Algorithms DATE: 09/17/2013 Lecturer: Debmalya Panigrahi Lecture 7 - Linear Programming Scribe: Hieu Bui 1 Overview In this lecture, we cover basics of linear programming,

### 10. Proximal point method

L. Vandenberghe EE236C Spring 2013-14) 10. Proximal point method proximal point method augmented Lagrangian method Moreau-Yosida smoothing 10-1 Proximal point method a conceptual algorithm for minimizing

### Inner products on R n, and more

Inner products on R n, and more Peyam Ryan Tabrizian Friday, April 12th, 2013 1 Introduction You might be wondering: Are there inner products on R n that are not the usual dot product x y = x 1 y 1 + +

### On SDP- and CP-relaxations and on connections between SDP-, CP and SIP

On SDP- and CP-relaations and on connections between SDP-, CP and SIP Georg Still and Faizan Ahmed University of Twente p 1/12 1. IP and SDP-, CP-relaations Integer program: IP) : min T Q s.t. a T j =

### Lecture 1 Newton s method.

Lecture 1 Newton s method. 1 Square roots 2 Newton s method. The guts of the method. A vector version. Implementation. The existence theorem. Basins of attraction. The Babylonian algorithm for finding

### 15.1. Exact Differential Equations. Exact First-Order Equations. Exact Differential Equations Integrating Factors

SECTION 5. Eact First-Order Equations 09 SECTION 5. Eact First-Order Equations Eact Differential Equations Integrating Factors Eact Differential Equations In Section 5.6, ou studied applications of differential

### Anytime plan TalkMore plan

CONDENSED L E S S O N 6.1 Solving Sstems of Equations In this lesson ou will represent situations with sstems of equations use tables and graphs to solve sstems of linear equations A sstem of equations

### Blind Deconvolution of Corrupted Barcode Signals

Blind Deconvolution of Corrupted Barcode Signals Everardo Uribe and Yifan Zhang Advisors: Ernie Esser and Yifei Lou Interdisciplinary Computational and Applied Mathematics Program University of California,

### LAGRANGIAN RELAXATION TECHNIQUES FOR LARGE SCALE OPTIMIZATION

LAGRANGIAN RELAXATION TECHNIQUES FOR LARGE SCALE OPTIMIZATION Kartik Sivaramakrishnan Department of Mathematics NC State University kksivara@ncsu.edu http://www4.ncsu.edu/ kksivara SIAM/MGSA Brown Bag

### Roots of Equations (Chapters 5 and 6)

Roots of Equations (Chapters 5 and 6) Problem: given f() = 0, find. In general, f() can be any function. For some forms of f(), analytical solutions are available. However, for other functions, we have

### Fleet Assignment Using Collective Intelligence

Fleet Assignment Using Collective Intelligence Nicolas E Antoine, Stefan R Bieniawski, and Ilan M Kroo Stanford University, Stanford, CA 94305 David H Wolpert NASA Ames Research Center, Moffett Field,

### 2.3 Convex Constrained Optimization Problems

42 CHAPTER 2. FUNDAMENTAL CONCEPTS IN CONVEX OPTIMIZATION Theorem 15 Let f : R n R and h : R R. Consider g(x) = h(f(x)) for all x R n. The function g is convex if either of the following two conditions

### LECTURE NOTES: FINITE ELEMENT METHOD

LECTURE NOTES: FINITE ELEMENT METHOD AXEL MÅLQVIST. Motivation The finite element method has two main strengths... Geometry. Very complex geometries can be used. This is probably the main reason why finite

### GROUP SPARSE OPTIMIZATION BY ALTERNATING DIRECTION METHOD. April 19, 2011

GROUP SPARSE OPTIMIZATION BY ALTERNATING DIRECTION METHOD WEI DENG, WOTAO YIN, AND YIN ZHANG April 19, 2011 Abstract. This paper proposes efficient algorithms for group sparse optimization with mied l

### Notes on Cholesky Factorization

Notes on Cholesky Factorization Robert A. van de Geijn Department of Computer Science Institute for Computational Engineering and Sciences The University of Texas at Austin Austin, TX 78712 rvdg@cs.utexas.edu

### Duality in Linear Programming

Duality in Linear Programming 4 In the preceding chapter on sensitivity analysis, we saw that the shadow-price interpretation of the optimal simplex multipliers is a very useful concept. First, these shadow

### THE NT (NEW TECHNOLOGY) HYPOTHESIS ABSTRACT

THE NT (NEW TECHNOLOGY) HYPOTHESIS Michele Impedovo Università Bocconi di Milano, Italy michele.impedovo@uni-bocconi.it ABSTRACT I teach a first-year undergraduate mathematics course at a business university.

### Definition of a Linear Program

Definition of a Linear Program Definition: A function f(x 1, x,..., x n ) of x 1, x,..., x n is a linear function if and only if for some set of constants c 1, c,..., c n, f(x 1, x,..., x n ) = c 1 x 1

### Practical Numerical Training UKNum

Practical Numerical Training UKNum 7: Systems of linear equations C. Mordasini Max Planck Institute for Astronomy, Heidelberg Program: 1) Introduction 2) Gauss Elimination 3) Gauss with Pivoting 4) Determinants

### Section Notes 3. The Simplex Algorithm. Applied Math 121. Week of February 14, 2011

Section Notes 3 The Simplex Algorithm Applied Math 121 Week of February 14, 2011 Goals for the week understand how to get from an LP to a simplex tableau. be familiar with reduced costs, optimal solutions,

### Perspective Shape from Shading and Viscosity Solutions

Perspective Shape from Shading and Viscosity Solutions E.Prados O.Faugeras ODYSSEE Lab. INRIA - ENS Paris - ENPC/CERMICS http://www-sop.inria.fr/odyssee/team ICCV 04, 15th October 2003, Nice/France Abstract

### Dual Methods for Total Variation-Based Image Restoration

Dual Methods for Total Variation-Based Image Restoration Jamylle Carter Institute for Mathematics and its Applications University of Minnesota, Twin Cities Ph.D. (Mathematics), UCLA, 2001 Advisor: Tony

### Computing a Nearest Correlation Matrix with Factor Structure

Computing a Nearest Correlation Matrix with Factor Structure Nick Higham School of Mathematics The University of Manchester higham@ma.man.ac.uk http://www.ma.man.ac.uk/~higham/ Joint work with Rüdiger

### Section 7.2 Linear Programming: The Graphical Method

Section 7.2 Linear Programming: The Graphical Method Man problems in business, science, and economics involve finding the optimal value of a function (for instance, the maimum value of the profit function

### RSVM: Reduced Support Vector Machines

page 1 RSVM: Reduced Support Vector Machines Yuh-Jye Lee and Olvi L. Mangasarian 1 Introduction Abstract An algorithm is proposed which generates a nonlinear kernel-based separating surface that requires

### POLYNOMIAL FUNCTIONS

POLYNOMIAL FUNCTIONS Polynomial Division.. 314 The Rational Zero Test.....317 Descarte s Rule of Signs... 319 The Remainder Theorem.....31 Finding all Zeros of a Polynomial Function.......33 Writing a

### Analysis of Approximation Algorithms for k-set Cover using Factor-Revealing Linear Programs

Analysis of Approximation Algorithms for k-set Cover using Factor-Revealing Linear Programs Stavros Athanassopoulos, Ioannis Caragiannis, and Christos Kaklamanis Research Academic Computer Technology Institute

### Factorization Theorems

Chapter 7 Factorization Theorems This chapter highlights a few of the many factorization theorems for matrices While some factorization results are relatively direct, others are iterative While some factorization

### 9. Numerical linear algebra background

Convex Optimization Boyd & Vandenberghe 9. Numerical linear algebra background matrix structure and algorithm complexity solving linear equations with factored matrices LU, Cholesky, LDL T factorization

### Tutorial: Operations Research in Constraint Programming

Tutorial: Operations Research in Constraint Programming John Hooker Carnegie Mellon University May 2009 Revised June 2009 May 2009 Slide 1 Motivation Benders decomposition allows us to apply CP and OR

### Applied Algorithm Design Lecture 5

Applied Algorithm Design Lecture 5 Pietro Michiardi Eurecom Pietro Michiardi (Eurecom) Applied Algorithm Design Lecture 5 1 / 86 Approximation Algorithms Pietro Michiardi (Eurecom) Applied Algorithm Design

### 1. Periodic Fourier series. The Fourier expansion of a 2π-periodic function f is:

CONVERGENCE OF FOURIER SERIES 1. Periodic Fourier series. The Fourier expansion of a 2π-periodic function f is: with coefficients given by: a n = 1 π f(x) a 0 2 + a n cos(nx) + b n sin(nx), n 1 f(x) cos(nx)dx

### 15.062 Data Mining: Algorithms and Applications Matrix Math Review

.6 Data Mining: Algorithms and Applications Matrix Math Review The purpose of this document is to give a brief review of selected linear algebra concepts that will be useful for the course and to develop

### Routing in Line Planning for Public Transport

Konrad-Zuse-Zentrum für Informationstechnik Berlin Takustraße 7 D-14195 Berlin-Dahlem Germany MARC E. PFETSCH RALF BORNDÖRFER Routing in Line Planning for Public Transport Supported by the DFG Research

### Chapter 6: Sensitivity Analysis

Chapter 6: Sensitivity Analysis Suppose that you have just completed a linear programming solution which will have a major impact on your company, such as determining how much to increase the overall production

### Nonlinear Optimization: Algorithms 3: Interior-point methods

Nonlinear Optimization: Algorithms 3: Interior-point methods INSEAD, Spring 2006 Jean-Philippe Vert Ecole des Mines de Paris Jean-Philippe.Vert@mines.org Nonlinear optimization c 2006 Jean-Philippe Vert,

### Good luck, veel succes!

Final exam Advanced Linear Programming, May 7, 13.00-16.00 Switch off your mobile phone, PDA and any other mobile device and put it far away. No books or other reading materials are allowed. This exam

### Introduction to Support Vector Machines. Colin Campbell, Bristol University

Introduction to Support Vector Machines Colin Campbell, Bristol University 1 Outline of talk. Part 1. An Introduction to SVMs 1.1. SVMs for binary classification. 1.2. Soft margins and multi-class classification.

### most 4 Mirka Miller 1,2, Guillermo Pineda-Villavicencio 3, The University of Newcastle Callaghan, NSW 2308, Australia University of West Bohemia

Complete catalogue of graphs of maimum degree 3 and defect at most 4 Mirka Miller 1,2, Guillermo Pineda-Villavicencio 3, 1 School of Electrical Engineering and Computer Science The University of Newcastle