9R13. Optimal Control. - R Vinter (Dept of Elec Eng, Imperial Col of Sci, Tech, and Med, London, SW7 2BT, UK). Birkhauser Boston, Cambridge MA. 2000. 507 pp. ISBN 0-8176-4075-4. $79.95.
Reviewed by S Sieniutycz (Dept of Chemical Eng, Fac of Chem and Process Eng, 1 Warynskiego St, Warszawa, 00-645, Poland).
In recent decades, control science has attained a high level of competence in advanced design of practical devices, complex industrial systems, robotics, and flying objects. One of the key concepts of the classical optimal control and its distinguishing feature is that it can take account of dynamic and pathwise constraints. Early key aspects were the Pontryagin’s maximum principle and an intuitive understanding of the relationship between the optimal profit function and the Hamilton-Jacobi equation of dynamic programming. The nature of the maximum principle and the techniques first applied to prove it, based on approximation of reachable sets, suggested that an essentially new sort of necessary conditions was required to deal with the constraints of the optimal control problems and new techniques to derive them. However, recent developments in optimal control, aimed at extending the range of application of available necessary conditions of optimality, stress its similarities rather than its differences with the variational calculus. In fact, the book in question represents one of the approaches of this sort in which constraints of optimal control are replaced by extended-valued penalty terms in the integral to be extremized. In this way, problems in optimal control are reformulated as extended problems in the calculus of variations with nonsmooth data. It is then possible to derive, in the optimal control context, optimality conditions of remarkable generality, analogous to classical necessary conditions in the variational calculus, in which classical derivatives are replaced by certain generalized derivatives of nonsmooth functions. A crucial role in these developments is played by the nonsmooth analysis which gives meaning to generalized derivatives and provides the mathematical apparatus for interpreting generalized solutions to the Hamilton-Jacobi equation. One highlight of these approaches is the clarification of the relationship between the optimal profit function and the Hamilton-Jacobi equation.
Of many books on optimal control theory written to date, this is one of a few that penetrates the subject matter in both a non-standard and fundamental way. Optimal Control is the one of the first books to provide a comprehensive treatment bringing together many of the important advances in nonsmooth optimal control concerning such basic issues as: necessary optimality conditions, minimizer regularity, and global optimality associated with the Hamilton-Jacobi theory. The analysis is largely self-contained and provides a unified perspective on those optimization problems which are beyond the realm of conventional analytical and computational techniques. Moreover, this analysis includes many of the unifying properties and simplifications discovered in recent research.
While the book constitutes an advanced text, it is intended for a relatively broad audience comprising postgraduates, researchers, and professionals in system science, process control, optimization, and applied mathematics. The book is written well; it has a readable preface, self-contained main-body text consisting of 12 chapters, a suitable reference section, and a good subject index. It also has simple and clear, good quality figures whose number is perhaps too small relative to needs of the whole treatment. To warrant the self-contained structure of the treatise, five preparatory chapters are included on nonsmooth analysis, measurable multifunctions, and differential inclusions.
The basic intention of the book is to bring together as a single treatise many important developments in optimal control based on the nonsmooth analysis in recent years and thereby render them accessible to a broader audience. With regards to nonsmooth optimal control, FH Clarke’s book Optimization and Nonsmooth Analysis (Wiley, New York, 1983), which was crucial for winning an audience for the field, remains the standard reference. The present book extends the range of topics covered therein by including some contemporary evergreen problems which are still at the research stage.
The Preface well defines the main goals of the book and important breakthroughs which culminated in setting the nonsmooth analysis and optimization as a new field. Chapter 1 is a brief overview of background of optimal control in historical and contemporary context, along with a passage from smooth to nonsmooth optimization problems. Throughout this chapter the principal results of the classical theory of optimal control and a basic philosophy of new approaches which abandon the concept of continuous state are well summarized. Chapter 2 introduces and treats measurable multifunctions and differential inclusions. Chapter 3 deals with variational principles. The author’s standpoint stresses the important role of exact penalization in the derivation of optimality conditions for constrained optimization problems. Mini-max theorems are pointed out as powerful tools in nonconvex optimization. Chapters 4 and 5 concern those aspects of nonsmooth analysis which are required to support future chapters on necessary conditions and dynamic programming of nonsmooth systems. These chapters also contain discussion of problems unsolved to date which are associated with nonsmooth analysis and the field of subdifferential calculus.
Chapters 6 is crucial for development of the new theory of maximum principle. Essentially three different versions of the maximum principle are proved, namely, for necessary conditions for problems with smooth data and free right endpoints, for problems with smooth data and general endpoint constraints, and, finally, for nonsmooth problems with general endpoint constraints. In the most general (nonsmooth) case, the approach used by the author is similar to those in fundamental works of AD Ioffe and FH Clarke: the nonsmooth maximum principle is a collorary of the generalized Euler-Lagrange condition of variational calculus and the Hamiltonian inclusion is used as a stepping stone to derive a nonsmooth maximum principle.
Extended Euler equations, related Hamiltonian conditions, and equations of motion generalizing the unconstrained theory are derived in Chapter 7. Chapter 8, based on the author’s work with FH Clarke, presents a number of valuable results supporting the new theory via a synthesis of some original results on necessary conditions for free end-time problems, terminating with a free end-time maximum principle. Chapter 9 develops the maximum principle for nonsmooth systems with state constraints, whereas Chapter 10 deals with necessary conditions for differential inclusion problems with state constraints. Chapter 11, which is based largely on the author’s original research, is a modern exposition of regularity of minimizers. The closing chapter, 12, deals with Bellman’s dynamic programming in the context of relation between smooth and nonsmooth problems. Among a number of basic issues, a dual problem is set up, in which Hamilton-Jacobi inequality features as a constraint. All problems are presented in the book at an advanced level. As they are not always easy to understand for a layman, some experience in the topic and knowledge based on other sources will be helpful. For these purposes, Clarke’s valuable book (cited above) can be recommended.
However, in the context of mathematical physics problems, the contemporary trend of nonsmooth analyses can also be seen in a different way: we may tend to use them either to include systems which are discontinuous by nature due to suitable discretizing of ODEs or to describe the inherently discrete systems. When considered physical objectives have these features, tasks of nonsmooth analyses can be seen somewhat differently. First, the present tendencies in physics are towards constructing numerical integration schemes for ordinary differential equations (ODEs) in such a way that a qualitative property of the solution of the ODE is exactly preserved. For Poisson-structure-preserving integration schemes (symplectic integrators), symmetries and related invariants, see, for example, a number of papers by RI McLachlan and GRW Quispel in the physics literature; for example: RI McLachlan and GRW Quispel, Physica D112, 298–309 (1998), and references therein. Examples are symplectic integrators for Hamiltonian OD equations, volume-preserving integrators for divergence-free OD equations, time-reversing symmetries preserving integrators, and integrators preserving the structure of gradient and Lyapunov systems.
Second, for optimally-controlled systems, there are discrete canonical (symplecticlike) structures, which exist for discrete algorithms with optimization-determined, free variable intervals of time along the optimal path, as shown, for example in the monograph: S Sieniutycz, Hamilton-Jacobi-Bellman Framework for Optimal Control in Multistage Energy Systems, Physics Reports 326, No 4, March 2000, 165–285, Elsevier, Amsterdam, 2000 (ISBN:0370–1573).
For the latter approaches, the constraint on the size of the time interval is absent or any inequality imposed on this interval is inoperative, in which cases an enlarged Hamiltonian vanishes and the discrete optimal set becomes canonical. The optimal-performance-based choice of time intervals, which involves global or integral criteria, may be compared with the first group of special-purpose integration methods for OD equations. Of course, all these are different problems than those considered in the book under review. Yet, it would be interesting to fill the gap between the two classes of problems discussed above and non-smooth optimization problems analyzed in the book.
To conclude, Optimal Control is not an easy book, but it is a deep, ambitious, and a novel book; a rigorous approach which should be read by researchers and graduate students interested in revision and extension of recent findings in optimal control theory and solution of evergreen problems, in particular nonsmooth maximum principles. The book is well written and well edited in terms of organization, technical writing, and the use of illustrations; its is also attractively printed. This reviewer warmly recommends that mathematically-oriented individuals and scientific libraries do purchase this inspiring and valuable book.
