An Introduction to Probabilistic Modeling

Introduction to the basic concepts of probability theory: independence, expectation, convergence in law and almost-sure convergence. Short expositions of more advanced topics such as Markov Chains, Stochastic Processes, Bayesian Decision Theory and Information Theory.

1 Basic Concepts and Elementary Models.- 1. The Vocabulary of Probability Theory.- 2. Events and Probability.- 2.1. Probability Space.- 2.2. Two Elementary Probabilistic Models.- 3. Random Variables and Their Distributions.- 3.1. Random Variables.- 3.2. Cumulative Distribution Function.- 4. Conditional Probability and Independence.- 4.1. Independence of Events.- 4.2. Independence of Random Variables.- 5. Solving Elementary Problems.- 5.1. More Formulas.- 5.2. A Small Bestiary of Exercises.- 6. Counting and Probability.- 7. Concrete Probability Spaces.- Illustration 1. A Simple Model in Genetics: Mendel’s Law and Hardy—Weinberg’s Theorem.- Illustration 2. The Art of Counting: The Ballot Problem and the Reflection Principle.- Illustration 3. Bertrand’s Paradox.- 2 Discrete Probability.- 1. Discrete Random Elements.- 1.1. Discrete Probability Distributions.- 1.2. Expectation.- 1.3. Independence.- 2. Variance and Chebyshev’s Inequality.- 2.1. Mean and Variance.- 2.2. Chebyshev’s Inequality.- 3. Generating Functions.- 3.1. Definition and Basic Properties.- 3.2. Independence and Product of Generating Functions.- Illustration 4. An Introduction to Population Theory: Galton—Watson’s Branching Process.- Illustration 5. Shannon’s Source Coding Theorem: An Introduction to Information Theory.- 3 Probability Densities.- I. Expectation of Random Variables with a Density.- 1.1. Univariate Probability Densities.- 1.2. Mean and Variance.- 1.3. Chebyshev’s Inequality.- 1.4. Characteristic Function of a Random Variable.- 2. Expectation of Functionals of Random Vectors.- 2.1. Multivariate Probability Densities.- 2.2. Covariance, Cross-Covariance, and Correlation.- 2.3. Characteristic Function of a Random Vector.- 3. Independence.- 3.1. Independent Random Variables.- 3.2. Independent Random Vectors.- 4. Random Variables That Are Not Discrete and Do Not Have a pd.- 4.1. The Abstract Definition of Expectation.- 4.2. Lebesgue’s Theorems and Applications.- Illustration 6. Buffon’s Needle: A Problem in Random Geometry.- 4 Gauss and Poisson.- 1. Smooth Change of Variables.- 1.1. The Method of the Dummy Function.- 1.2. Examples.- 2. Gaussian Vectors.- 2.1. Characteristic Function of Gaussian Vectors.- 2.2. Probability Density of a Nondegenerate Gaussian Vector.- 2.3. Moments of a Centered Gaussian Vector.- 2.4. Random Variables Related to Gaussian Vectors.- 3. Poisson Processes.- 3.1. Homogeneous Poisson Processes Over the Positive Half Line.- 3.2. Nonhomogeneous Poisson Processes Over the Positive Half Line.- 3.3. Homogeneous Poisson Processes on the Plane.- 4. Gaussian Stochastic Processes.- 4.1. Stochastic Processes and Their Law.- 4.2. Gaussian Stochastic Processes.- Illustration 7. An Introduction to Bayesian Decision Theory: Tests of Gaussian Hypotheses.- 5 Convergences.- 1. Almost-Sure Convergence.- 1.1. The Borel—Cantelli Lemma.- 1.2. A Criterion for Almost-Sure Convergence.- 1.3. The Strong Law of Large Numbers.- 2. Convergence in Law.- 2.1. Criterion of the Characteristic Function.- 2.2. The Central Limit Theorem.- 3. The Hierarchy of Convergences.- 3.1. Almost-Sure Convergence Versus Convergence in Probability.- 3.2. Convergence in the Quadratic Mean.- 3.3. Convergence in Law in the Hierarchy of Convergences.- 3.4. The Hierarchical Tableau.- Illustration 8. A Statistical Procedure: The Chi-Square Test.- Illustration 9. Introduction to Signal Theory: Filtering.- Additional Exercises.- Solutions to Additional Exercises.