27. OKTOBER 2010 Optimization - Elements of Calculus - Basics of constrained and unconstrained optimization @
Optimization the general problem Optimization minimize, subject to Ω = (,, ), optimization variables, or decision variables :, objective function Ω, is the constraint set or feasible set, sometimes Ω =, where = 1,, }, where = 1,, : constraint functions optimal solution or minimizer, is the smallest value of among = (,, ), satisfying the constraint
Local minimizer Definition 6.1 Suppose :, is a real-valued function defined on the set Ω. A point is a local minimizer of over Ω if there exists > 0 such that ( ) for all Ω and <. A point is a global minimizer of over Ω if ( ) for all Ω.
Partiel derivatives and the Hessian matrix
Directional derivatives
Level sets and contour plots MatLab demo: ContoursInMatlab.m
Exercise on Derivatives Find a point (x,y) where the gradient parallel to the y-axis for the following function:, = 3x 4xy + 7y In the point you found above, what is the directional derivation in the direction (1,0)?
Feasible directions
First Order Necessary Condition (FONC) Less important More important
FONC example (Thm. 6.1)
FONC example (Cor. 6.1)
Quadratic forms using matrices Examples from Lay sec. 7.2
Quadratic forms using matrices Examples from Lay sec. 7.2
Qudratic forms using matrices Examples from Lay sec. 7.2
Qudratic forms change of varibles Examples from Lay sec. 7.2
Qudratic forms and extremal points Examples from Lay sec. 7.2
Exercise on Qudratic forms Examples from Lay sec. 7.2
Optimization
Optimization
Optimization