Homogeneous Denumerable Markov Processes

Markov processes play an important role in the study of probability theory. Homogeneous denumerable Markov processes are among the main topics in the theory and have a wide range of application in various fields of science and technology (for example, in physics, cybernetics, queuing theory and dynamical programming). This book is a detailed presentation and summary of the research results obtained by the authors in recent years. Most of the results are published for the first time. Two new methods are given: one is the minimal nonnegative solution, the second the limit transition method. With the help of these two methods, the authors solve many important problems in the framework of denumerable Markov processes.

I Construction Theory of Sample Functions of Homogeneous Denumerable Markov Processes.- I The First Construction Theorem.- § 1.1 Introduction.- § 1.2 Definition of transformation gn.- § 1.3 Convergence of the sequence X(n)(?) (n?1).- § 1.4 Further properties of X(n)(?) (n?1).- § 1.5 The first construction theorem.- II The Second Construction Theorem.- § 2.1 Introduction.- § 2.2 The mapping Tmn.- § 2.3 The mapping Wn.- § 2.4 Constructing auxiliary functions.- § 2.5 The second construction theorem.- § 2.6 Summary.- § 2.7 Two notes.- II Theory of Minimal Nonnegative Solutions for Systems of Nonnegative Linear Equations.- III General Theory.- § 3.1 Introduction.- § 3.2 Definition of a system of nonnegative linear equations and definition, existence and uniqueness of its minimal nonnegative solution.- § 3.3 Comparison theorem and linear combination theorem.- § 3.4 Localization theorem.- § 3.5 Connecting property of the minimal nonnegative solution.- § 3.6 Limit theorem.- § 3.7 Matrix representation.- § 3.8 Dual theorem.- IV Calculation.- § 4.1 Some lemmas.- § 4.2 Reduction of the problems.- § 4.3 Ordinary systems of strictly nonhomogeneous equations with dimension n.- V Systems of 1-Bounded Equations.- § 5.1 Introduction.- § 5.2 First-type leading-outside systems of equations.- § 5.3 First-type consistent systems of equations.- § 5.4 Tailed random systems of strictly nonhomogeneous equations.- § 5.5 Regular systems of equations.- § 5.6 Pseudo-normal systems of equations.- § 5.7 Pseudo-normal systems of equations of finite dimension.- § 5.8 Second-type regular systems of equations.- III Homogeneous Denumerable Markov Chains.- VI General Theory.- § 6.1 Introduction.- § 6.2 Transition probabilities.- § 6.3 Distribution and moments of the first passage time.- § 6.4 Distribution and moments of the first passage time of a homogeneous finite Markov chain.- § 6.5 Distribution and moments of the times of passage.- § 6.6 Criteria for recurrence.- § 6.7 Distribution and moments of additive functionals.- § 6.8 Derived Markov chains and criteria for atomic almost closed sets.- VII Martin Exit Boundary Theory.- § 7.1 Introduction.- § 7.2 Decomposition for Markov chains.- § 7.3 Limit behaviour of excessive functions.- § 7.4 Green functions and Martin kernels.- § 7.5 h-chains.- § 7.6 Limit theorem for Martin kernels.- § 7.7 Martin boundaries.- § 7.8 Distribution of x?.- § 7.9 Martin expressions of excessive functions.- § 7.10 Exit space.- § 7.11 Uniqueness theorem.- § 7.12 Minimal excessive functions.- § 7.13 Terminal random variables.- § 7.14 Criteria for potentials and excessive functions, Riesz decomposition.- § 7.15 Criteria for minimal harmonic functions, minimal potentials and minimal excessive functions.- § 7.16 Atomic exit spaces and nonatomic exit spaces.- § 7.17 Blackwell decomposition of the state space.- VIII Martin Entrance Boundary Theory.- § 8.1 Introduction.- § 8.2 The first group of lemmas.- § 8.3 Properties of finite excessive measures.- § 8.4 The second group of lemmas.- § 8.5 Entrance boundary.- § 8.6 Entrance space and the expressions of excessive measures.- IV Homogeneous Denumerable Markov Processes.- IX Minimal Q-Processes.- § 9.1 Introduction.- § 9.2 Transition probabilities.- § 9.3 Distribution and moments of the first passage time.- § 9.4 Criterion for the positive recurrence.- § 9.5 Distribution and moments of integral-type functionals.- § 9.6 Distribution and moments of integral-type functionals on pseudo-translatable sets.- § 9.7 Extensions of the results in § 9.3.- X Q-Processes of Order One.- § 10.1 Introduction.- § 10.2 Transition probabilities.- § 10.3 Distribution and moments of the first passage time.- XI Arbitrary Q-Processes.- § 11.1 Strengthening of the first construction theorem.- § 11.2 Transition probability.- § 11.3 Decomposition theorems for excessive measures and excessive functions.- V Construction Theory of Homogeneous Denumerable Markov Processes.- XII Criteria for the Uniqueness of Q-Processes.- § 12.1 Introduction.- § 12.2 Lemmas.- § 12.3 Proof of the main theorem.- § 12.4 The case of diagonal type.- § 12.5 The bounded case.- § 12.6 The case when E is finite.- § 12.7 The case of a branch Q-matrix.- § 12.8 Another criterion and the finite and nonconservative case.- § 12.9 Independence of the two conditions in Theorem 12.1.1.- § 12.10 Probability interpretation of Condition (i) in Theorem 12.1.1.- XIII Construction of Q-Processes.- § 13.1 Construction theorem.- § 13.2 Specifications of all the Q-processes.- § 13.3 Expression of $$\left\{ {Q,\,{\Pi _{{{\left( {\partial X} \right)}_{e,}}\,x\,E}}} \right\}$$-processes.- § 13.4 Discussion.- XIV Qualitative Theory.- § 14.1 Introduction.- § 14.2 Statement of results.- § 14.3 Reduction of the construction problem of B-type Q-processes, Doob processes.- § 14.4 Reduction of the construction problem of B?F-type Q-processes.- § 14.5 Proofs of Theorems 14.2.1–14.2.3.- § 14.6 Proof and examples of applications of Theorem 14.2.4.- § 14.7 Proofs of Theorems 14.2.5–14.2.10.