This comprehensive introduction to the estimation and control of dynamic stochastic systems provides complete derivations of key results. The second edition includes improved and updated material, and a new presentation of polynomial control and new derivation of linear-quadratic-Gaussian control.
1. Introduction.- 1.1 What is a Stochastic System?.- Bibhography.- 2. Some Probability Theory.- 2.1 Introduction.- 2.2 Random Variables and Distributions.- 2.2.1 Basic Concepts.- 2.2.2 Gaussian Distributions.- 2.2.3 Correlation and Dependence.- 2.3 Conditional Distributions.- 2.4 The Conditional Mean for Gaussian Variables.- 2.5 Complex-Valued Gaussian Variables.- 2.5.1 The Scalar Case.- 2.5.2 The Multivariable Case.- 2.5.3 The Rayleigh Distribution.- Exercises.- 3. Models.- 3.1 Introduction.- 3.2 Stochastic Processes.- 3.3 Markov Processes and the Concept of State.- 3.4 Covariance Function and Spectrum.- 3.5 Bispectrum.- 3.A Appendix. Linear Complex-Valued Signals and Systems.- 3.A.1 Complex-Valued Model of a Narrow-Band Signal.- 3.A.2 Linear Complex-Valued Systems.- 3.B Appendix. Markov Chains.- Exercises.- 4. Analysis.- 4.1 Introduction.- 4.2 Linear Filtering.- 4.2.1 Transfer Function Models.- 4.2.2 State Space Models.- 4.2.3 Yule-Walker Equations.- 4.3 Spectral Factorization.- 4.3.1 Transfer Function Models.- 4.3.2 State Space Models.- 4.3.3 An Example.- 4.4 Continuous-time Models.- 4.4.1 Covariance Function and Spectra.- 4.4.2 Spectral Factorization.- 4.4.3 White Noise.- 4.4.4 Wiener Processes.- 4.4.5 State Space Models.- 4.5 Sampling Stochastic Models.- 4.5.1 Introduction.- 4.5.2 State Space Models.- 4.5.3 Aliasing.- 4.6 The Positive Real Part of the Spectrum.- 4.6.1 ARMA Processes.- 4.6.2 State Space Models.- 4.6.3 Continuous-time Processes.- 4.7 Effect of Linear Filtering on the Bispectrum.- 4.8 Algorithms for Covariance Calculations and Sampling.- 4.8.1 ARMA Covariance Function.- 4.8.2 ARMA Cross-Covariance Function.- 4.8.3 Continuous-Time Covariance Function.- 4.8.4 Sampling.- 4.8.5 Solving the Lyapunov Equation.- 4. A Appendix. Auxiliary Lemmas.- Exercises.- 5. Optimal Estimation.- 5.1 Introduction.- 5.2 The Conditional Mean.- 5.3 The Linear Least Mean Square Estimate.- 5.4 Propagation of the Conditional Probability Density Function.- 5.5 Relation to Maximum Likelihood Estimation.- 5.A Appendix. A Lemma for Optimality of the Conditional Mean.- Exercises.- 6. Optimal State Estimation for Linear Systems.- 6.1 Introduction.- 6.2 The Linear Least Mean Square One-Step Prediction and Filter Estimates.- 6.3 The Conditional Mean.- 6.4 Optimal Filtering and Prediction.- 6.5 Smoothing.- 6.5.1 Fixed Point Smoothing.- 6.5.2 Fixed Lag Smoothing.- 6.6 Maximum a posteriori Estimates.- 6.7 The Stationary Case.- 6.8 Algorithms for Solving the Algebraic Riccati Equation.- 6.8.1 Introduction.- 6.8.2 An Algorithm Based on the Euler Matrix.- 6.A Appendix. Proofs.- 6.A.1 The Matrix Inversion Lemma.- 6.A.2 Proof of Theorem 6.1.- 6.A.3 Two Determinant Results.- Exercises.- 7. Optimal Estimation for Linear Systems by Polynomial Methods.- 7.1 Introduction.- 7.2 Optimal Prediction.- 7.2.1 Introduction.- 7.2.2 Optimal Prediction of ARMA Processes.- 7.2.3 A General Case.- 7.2.4 Prediction of Nonstationary Processes.- 7.3 Wiener Filters.- 7.3.1 Statement of the Problem.- 7.3.2 The Unrealizable Wiener Filter.- 7.3.3 The Realizable Wiener Filter.- 7.3.4 Illustration.- 7.3.5 Algorithmic Aspects.- 7.3.6 The Causal Part of a Filter, Partial Fraction Decomposition and a Diophantine Equation.- 7.4 Minimum Variance Filters.- 7.4.1 Introduction.- 7.4.2 Solution.- 7.4.3 The Estimation Error.- 7.4.4 Extensions.- 7.4.5 Illustrations.- 7.5 Robustness Against Modelling Errors.- Exercises.- 8. Illustration of Optimal Linear Estimation.- 8.1 Introduction.- 8.2 Spectral Factorization.- 8.3 Optimal Prediction.- 8.4 Optimal Filtering.- 8.5 Optimal Smoothing.- 8.6 Estimation Error Variance.- 8.7 Weighting Pattern.- 8.8 Frequency Characteristics.- Exercises.- 9. Nonlinear Filtering.- 9.1 Introduction.- 9.2 Extended Kaiman Filters.- 9.2.1 The Basic Algorithm.- 9.2.2 An Iterated Extended Kalman Filter.- 9.2.3 A Second-order Extended Kalman Filter.- 9.2.4 An Example.- 9.3 Gaussian Sum Estimators.- 9.4 The Multiple Model Approach.- 9.4.1 Introduction.- 9.4.2 Fixed Models.- 9.4.3 Switching Models.- 9.4,4 Interacting Multiple Models Algorithm.- 9.5 Monte Carlo Methods for Propagating the Conditional Probability Density Functions.- 9.6 Quantized Measurements.- 9.7 Median Filters.- 9.7.1 Introduction.- 9.7.2 Step Response.- 9.7.3 Response to Sinusoids.- 9.7.4 Effect on Noise.- 9.A Appendix. Auxiliary results.- 9.A.1 Analysis of the Sheppard Correction.- 9.A.2 Some Probability Density Functions.- Exercises.- 10. Introduction to Optimal Stochastic Control.- 10.1 Introduction.- 10.2 Some Simple Examples.- 10.2.1 Introduction.- 10.2.2 Deterministic System.- 10 2 3 Random Time Constant.- 10.2.4 Noisy Observations.- 10 2 5 Process Noise.- 10.2.6 Unknown Time Constants and Measurement Noise.- 10 2 7 Unknown Gain.- 10.3 Mathematical Preliminaries.- 10.4 Dynamic Programming.- 10.4.1 Deterministic Systems.- 10.4.2 Stochastic Systems.- 10.5 Some Stochastic Controllers.- 10.5.1 Dual Control.- 10.5.2 Certainty Equivalence Control.- 10.5.3 Cautious Control.- Exercises.- 11. Linear Quadratic Gaussian Control.- 11.1 Introduction.- 11.2 The Optimal Controllers.- 11.2.1 Optimal Control of Deterministic Systems.- 11.2.2 Optimal Control with Complete State Information.- 11.2.3 Optimal Control with Incomplete State Information.- 11.3 Duality Between Estimation and Control.- 11.4 Closed Loop System Properties.- 114 1 Representations of the Regulator.- 11.4.2 Representations of the Closed Loop System.- 11.4.3 The Closed Loop Poles.- 11.5 Linear Quadratic Gaussian Design by Polynomial Methods.- 11.5.1 Problem Formulation.- 11.5.2 Minimum Variance Control.- 11.5.3 The General Case.- 11.6 Controller Design by Linear Quadratic Gaussian Theory.- 11.6.1 Introduction.- 11.6.2 Choice of Observer Poles.- 11. A Appendix. Derivation of the Optimal Linear Quadratic Gaussian Feedback and the Riccati Equation from the Bellman Equation.- Exercises.- Answers to Selected Exercises.
