Further Topics on Discrete-Time Markov Control Processes

Hernandez-Lerma Onesimo Lasserre Jean B.

ISBN 10: 1461268184 / ISBN 13: 9781461268185

Editore: Springer, 2012

Riguardo questo articolo

PRINT ON DEMAND pp. 296.

Codice articolo 1897522564

Dati bibliografici

• Titolo: Further Topics on Discrete-Time Markov Control Processes
• Editore: Springer
• Data di pubblicazione: 2012
• Lingua: Inglese
• ISBN 10: 1461268184
• ISBN 13: 9781461268185
• Rilegatura: Brossura
• Condizione: New

Descrizione articolo

Riassunto

This book presents the second part of a two-volume series devoted to a systematic exposition of some recent developments in the theory of discrete-time Markov control processes. As in Volume I (AM 30 Discrete-Time Markov Control Processes: Basic Optimality Criteria), interest is mainly confined to MCPs with Borel state and control spaces. However, an important feature of the present volume is that is it essentially self-contained and can be read independently of Volume I, because the assumptions on the control models are usually different.

Contenuti

7 Ergodicity and Poisson’s Equation.- 7.1 Introduction.- 7.2 Weighted norms and signed kernels.- A. Weighted-norm spaces.- B. Signed kernels.- C. Contraction maps.- 7.3 Recurrence concepts.- A. Irreducibility and recurrence.- B. Invariant measures.- C. Conditions for irreducibility and recurrence.- D. w-Geometric ergodicity.- 7.4 Examples on w-geometric ergodicity.- 7.5 Poisson’s equation.- A. The multichain case.- B. The unichain P.E.- C. Examples.- 8 Discounted Dynamic Programming with Weighted Norms.- 8.1 Introduction.- 8.2 The control model and control policies.- 8.3 The optimality equation.- A. Assumptions.- B. The discounted-cost optimality equation.- C. The dynamic programming operator.- D. Proof of Theorem 8.3.6.- 8.4 Further analysis of value iteration.- A. Asymptotic discount optimality.- B. Estimates of VI convergence.- C. Rolling horizon procedures.- D. Forecast horizons and elimination of non-optimal actions.- 8.5 The weakly continuous case.- 8.6 Examples.- 8.7 Further remarks.- 9 The Expected Total Cost Criterion.- 9.1 Introduction.- 9.2 Preliminaries.- A. Extended real numbers.- B. Integrability.- 9.3 The expected total cost.- 9.4 Occupation measures.- A. Expected occupation measures.- B. The sufficiency problem.- 9.5 The optimality equation.- A. The optimality equation.- B. Optimality criteria.- C. Deterministic stationary policies.- 9.6 The transient case.- A. Transient models.- B. Optimality conditions.- C. Reduction to deterministic policies.- D. The policy iteration algorithm.- 10 Undiscounted Cost Criteria.- 10.1 Introduction.- A. Undiscounted criteria.- B. AC criteria.- C. Outline of the chapter.- 10.2 Preliminaries.- A. Assumptions.- B. Corollaries.- C. Discussion.- 10.3 From AC-optimality to undiscounted criteria.- A. The AC optimality inequality.- B. The AC optimality equation.- C. Uniqueness of the ACOE.- D. Bias-optimal policies.- E. Undiscounted criteria.- 10.4 Proof of Theorem 10.3.1.- A. Preliminary lemmas.- B. Completion of the proof.- 10.5 Proof of Theorem 10.3.6.- A. Proof of part (a).- B. Proof of part (b).- C. Policy iteration.- 10.6 Proof of Theorem 10.3.7.- 10.7 Proof of Theorem 10.3.10.- 10.8 Proof of Theorem 10.3.11.- 10.9 Examples.- 11 Sample Path Average Cost.- 11.1 Introduction.- A. Definitions.- B. Outline of the chapter.- 11.2 Preliminaries.- A. Positive Harris recurrence.- B. Limiting average variance.- 11.3 The w-geometrically ergodic case.- A. Optimality in IIDS.- B. Optimality in II.- C. Variance minimization.- D. Proof of Theorem 11.3.5.- E. Proof of Theorem 11.3.8.- 11.4 Strictly unbounded costs.- 11.5 Examples.- 12 The Linear Programming Approach.- 12.1 Introduction.- A. Outline of the chapter.- 12.2 Preliminaries.- A. Dual pairs of vector spaces.- B. Infinite linear programming.- C. Approximation of linear programs.- D. Tightness and invariant measures.- 12.3 Linear programs for the AC problem.- A. The linear programs.- B. Solvability of (P).- C. Absence of duality gap.- D. The Farkas alternative.- 12.4 Approximating sequences and strong duality.- A. Minimizing sequences for (P).- B. Maximizing sequences for (P*).- 12.5 Finite LP approximations.- A. Aggregation.- B. Aggregation-relaxation.- C. Aggregation-relaxion-inner approximations.- 12.6 Proof of Theorems 12.5.3, 12.5.5, 12.5.7.- References.- Abbreviations.- Glossary of notation.