Practical Optimization

In the intervening years since this book was published in 1981, the field of optimization has been exceptionally lively. This fertility has involved not only progress in theory, but also faster numerical algorithms and extensions into unexpected or previously unknown areas such as semidefinite programming. Despite these changes, many of the important principles and much of the intuition can be found in this Classics version of Practical Optimization.

This book provides model algorithms and pseudocode, useful tools for users who prefer to write their own code as well as for those who want to understand externally provided code. It presents algorithms in a step-by-step format, revealing the overall structure of the underlying procedures and thereby allowing a high-level perspective on the fundamental differences. And it contains a wealth of techniques and strategies that are well suited for optimization in the twenty-first century, and particularly in the now-flourishing fields of data science, “big data,” and machine learning.

Practical Optimization is appropriate for advanced undergraduates, graduate students, and researchers interested in methods for solving optimization problems.

Philip E. Gill is Distinguished Professor of Mathematics and co-director of the Center for Computational Mathematics at the University of California, San Diego (UCSD). He received the Diploma of Imperial College in 1971 at the Imperial College of Science, Technology and Medicine, University of London, and his Ph.D. in applied mathematics in 1974, also at Imperial College. Before joining UCSD in 1988, he was a researcher at the National Physical Laboratory in the United Kingdom and the Systems Optimization Laboratory in the Operations Research Department at Stanford. His research areas are numerical analysis, applied mathematics, and scientific computation, with a focus on numerical optimization. He has more than 40 years of experience in the development of optimization algorithms and the production of software that solves continuous optimization problems. He is the co-author of the computer packages NPSOL (for nonlinear optimization; with W. Murray and M. Saunders) and SNOPT (for large-scale sparse nonlinear optimization; with W. Murray, M. Saunders, and E. Wong), which have been distributed to thousands of universities, research laboratories and industrial sites around the world. He was elected Fellow of the Society of Industrial and Applied Mathematics (SIAM) in 2014. He was co-chair of the 1999 SIAM Conference on Optimization, and he has served on the organizing committees of eight national and international conferences on optimization. He has been an associate editor for SIAM Journal on Optimization, SIAM Journal on Matrix Analysis and Applications, and Mathematical Programming Computation. He is the co-author (with W. Murray and M. Wright) of two textbooks on optimization, the most recent of which (Numerical Linear Algebra and Optimization), was published by Addison-Wesley Publishing in 1991. He is an author or co-author of more than 65 refereed papers and has given more than 50 invited presentations at major conferences.

Walter Murray has been a professor at Stanford University since 1979. He is the director of the Systems Optimization Laboratory (SOL), and was a previous director of the program in Scientific Computing and Computational Mathematics (SCCM) and of the Institute for Computational and Mathematical Engineering (ICME) both at Stanford University. He received his Ph.D. from the University of London in 1969 while he was working in the Mathematics Division of the National Physical Laboratory. Professor Murray's research areas are numerical analysis, applied mathematics, and scientific computation. His primary research has been in numerical optimization that includes nonlinear, nonconvex, and discrete problems. His whole career has been dedicated to the development of optimization algorithms and the production of software that solves continuous optimization problems. Professor Murray has been principal advisor to 40 doctoral students from 19 countries and five continents who have graduated from a variety of universities (Stanford, London, Oxford, Royal Institute of Technology in Stockholm, South Australia).

Margaret H. Wright is Silver Professor of Computer Science and Mathematics in the Courant Institute of Mathematical Sciences, New York University. She received her B.S. (mathematics) and M.S. and Ph.D. (computer science) from Stanford University. She holds an honorary doctorate in mathematics from the University of Waterloo, and an honorary doctorate of technology from the Swedish Royal Institute of Technology (KTH). Before joining NYU, she was a researcher in the Systems Optimization Laboratory in the Department of Operations Research at Stanford University and then a Distinguished Member of Technical Staff at Bell Laboratories. Her research focuses on continuous optimization, numerical linear algebra, scientific computing, and methods for solving real-world problems. She (with P. Gill and W. Murray) has co-authored two books about numerical optimization. She is a member of the (U.S.) National Academy of Engineering (NAE), the American Academy of Arts and Sciences, and the National Academy of Sciences (NAS). She chaired the NAS Applied Mathematics Section and the NAE Committee on Membership. She is a Fellow of the Society for Industrial and Applied Mathematics (SIAM), the American Mathematical Society, and the Institute for Operations Research and Management Science (INFORMS), and an honorary member of the London Mathematical Society. Her SIAM activities include service as president and membership in the Council and Board of Trustees. She has been editor-in-chief of SIAM Review and an associate editor of SIAM Journal on Scientific Computing, SIAM Journal on Optimization, and Mathematical Programming Computation. She received the John von Neumann lecture prize from SIAM in 2019.