Infinite Horizon Optimal Control : Deterministic and Stochastic Systems

This work presents a systematic account of the development of infinite horizon optimal control. It is demonstrated that the usual concept of optimality is insufficient as it assumes, a priori, that the objective functional is finite. To avoid this difficulty a hierarchy of optimality concepts have been developed, which serves as a background for the treatment of the usual questions encountered in optimization. Namely, the book investigates necessary conditions for optimality, sufficient conditions for optimality, and the existence of optimal solutions.

1 Dynamical Systems with Unbounded Time Interval in Engineering, Ecology and Economics.- 1.1 Introduction.- 1.2 The regulator problem.- 1.3 The pest control problem and other problems of optimal control of interacting species.- 1.4 The optimal economic growth problem.- 1.5 Definition of optimality on an unbounded time interval.- 1.6 Uniformly optimal solutions are agreeable.- 2 Necessary Conditions and Sufficient Conditions for Optimality.- 2.1 Introduction.- 2.2 The maximum principle with a finite horizon.- 2.3 The optimality principle.- 2.4 A maximum principle for an infinite horizon control problem.- 2.5 Sufficient conditions for overtaking optimality.- 3 Asymptotic Stability and the Turnpike Property in Some Simple Control Problems.- 3.1 Introduction.- 3.2 Saddle point property of the Hamiltonian in a convex problem of Lagrange. Implications on local asymptotic stability of optimally controlled systems.- 3.3 An exact turnpike property: Optimal fish harvest.- 3.4 Use of a phase diagram for a one-state-variable control problem: The simple optimal economic growth model.- 4 Global Asymptotic Stability and Existence of Optimal Trajectories for Infinite Horizon Autonomous Convex Systems.- 4.1 Introduction.- 4.2 The class of systems considered.- 4.3 Convergence toward the Von Neumann Set for weakly overtaking trajectories.- 4.4 The turnpike property.- 4.5 Global asymptotic stability for extremal trajectories.- 4.6 A Lyapunov function approach for GAS of optimal trajectories.- 4.7 Sufficient conditions for overtaking optimality.- 4.8 Existence of optimal trajectories.- 4.9 Overtaking optimality under relaxed assumptions.- 5 The Reduction to Finite Rewards.- 5.1 Introduction.- 5.2 The Property R.- 5.3 The connection between continuous and discrete time control systems.- 5.4 Existence of a reduction to finite rewards.- 5.5 A representation formula and turnpike properties of optimal controls.- 5.6 Systems with unbounded rewards and with discounting factors.- 5.7 Infinite horizon tracking of periodic signals.- 5.8 Optimal trajectories and turnpike properties of infinite horizon autonomous nonconvex systems.- 5.9 Two special cases: Scalar systems, and integrands in a separated form.- 6 Asymptotic Stability with a Discounted Criterion; Global and Local Analysis.- 6.1 Introduction.- 6.2 Modified Hamiltonian systems.- 6.3 Cass-Shell conditions for GAS of modified Hamiltonian systems.- 6.4 Brock-Sheinkman conditions for GAS of modified Hamiltonian systems.- 6.5 Another useful condition for GAS.- 6.6 Neighboring extremals, the second variation and analysis of local asymptotic stability of a stationary point, using the optimal linear quadratic regulator problem.- 6.7 The turnpike property for finite horizon optimal control problems with discounting.- 7 Turnpike Properties and Existence of Overtaking Optimal Solutions for Classes of Nonautonomous Nonconvex Control Problems.- 7.1 Introduction.- 7.2 G-supported trajectories.- 7.3 Carathéodory’s method for finite horizon optimal control problems.- 7.4 Carathéodory’s method for infinite horizon optimal control problems.- 7.5 The growth condition (?) and the compactness of the set of admissible trajectories.- 7.6 Upper closure and the existence of strongly optimal solutions.- 7.7 The existence of overtaking optimal solutions.- 8 Control of Systems with Integrodifferential Equations.- 8.1 Introduction.- 8.2 The basic model.- 8.3 Linear hereditary operators and an upper closure theorem.- 8.4 Existence of overtaking optimal solutions.- 8.4.1 Support property assumptions.- 8.4.2 A Turnpike theorem.- 8.4.3 A sufficient condition for overtaking optimality.- 8.4.4 Existence of overtaking solutions.- 8.5 Examples.- 9 Extensions to Distributed Parameter Systems.- 9.1 Introduction.- 9.2 Examples.- 9.3 Semigroups of operators and linear control systems.- 9.4 The optimal control problem.- 9.5 The turnpike properties.- 9.6 Existence of overtaking optimal solutions.- 9.7 More on the examples.- 9.8 The extension to systems with distributed parameters and boundary controls.- 10 Stochastic Control with the Overtaking Criterion.- 10.1 Introduction.- 10.2 The reduction to finite costs and the infinite-horizon Bellman equation.- 10.3 Infinite-horizon stochastic tracking.- 10.4 Optimal control of nonlinear diffusions in ?n.- 10.5 On almost-sure overtaking optimality.- 11 Maximum Principle and Turnpike Properties for Systems with Random Modal Jumps.- 11.1 Introduction.- 11.2 Optimal control under random stopping time.- 11.3 Turnpike properties.- 11.3.1 A global result.- 11.3.2 A local result.- 11.3.3 An example: Economics of innovation.- 11.4 Piecewise Deterministic Control Systems.- 11.4.1 PDCS dynamics.- 11.4.2 Reformulation as a Markov Renewal Decision Process.- 11.5 Global turnpike property for constant jump rates 3.