Nonlinear Optimization By The Sequential Unconstrained Minimization Technique Using Conjugate Gradient Methods

Nonlinear Optimization by the Sequential Unconstrained Minimization Technique Using Conjugate Gradient Methods

Nonlinear Programming 4

Nonlinear Programming, 4 focuses on linear, quadratic, and nonlinear programming, unconstrained minimization, nonsmooth and discrete optimization, ellipsoidal methods, linear complementarity problems, and software evaluation. The selection first elaborates on an upper triangular matrix method for quadratic programming, solving quadratic programs by an exact penalty function, and QP-based methods for large-scale nonlinearly constrained optimization. Discussions focus on large-scale linearly constrained optimization, search direction for superbasic variables, finite convergence, basic properties, comparison of three active set methods, and QP-based methods for dense problems. The book then examines an iterative linear programming algorithm based on an augmented Lagrangian and iterative algorithms for singular minimization problems. The publication ponders on the derivation of symmetric positive definite secant updates, preconditioned conjugate gradient methods, and finding the global minimum of a function of one variable using the method of constant signed higher order derivatives. Topics include effects of calculation errors, application to polynomial minimization, using moderate additional storage, updating Cholesky factors, and utilizing sparse second order information. The selection is a valuable source of data for researchers interested in nonlinear programming.

Linear and Nonlinear Programming

The 5th edition of this classic textbook covers the central concepts of practical optimization techniques, with an emphasis on methods that are both state-of-the-art and popular. One major insight is the connection between the purely analytical character of an optimization problem and the behavior of algorithms used to solve that problem. End-of-chapter exercises are provided for all chapters. The material is organized into three separate parts. Part I offers a self-contained introduction to linear programming. The presentation in this part is fairly conventional, covering the main elements of the underlying theory of linear programming, many of the most effective numerical algorithms, and many of its important special applications. Part II, which is independent of Part I, covers the theory of unconstrained optimization, including both derivations of the appropriate optimality conditions and an introduction to basic algorithms. This part of the book explores the general properties of algorithms and defines various notions of convergence. In turn, Part III extends the concepts developed in the second part to constrained optimization problems. Except for a few isolated sections, this part is also independent of Part I. As such, Parts II and III can easily be used without reading Part I and, in fact, the book has been used in this way at many universities. New to this edition are popular topics in data science and machine learning, such as the Markov Decision Process, Farkas’ lemma, convergence speed analysis, duality theories and applications, various first-order methods, stochastic gradient method, mirror-descent method, Frank-Wolf method, ALM/ADMM method, interior trust-region method for non-convex optimization, distributionally robust optimization, online linear programming, semidefinite programming for sensor-network localization, and infeasibility detection for nonlinear optimization.