Asymptotic Methods in Statistical Decision Theory

This book grew out of lectures delivered at the University of California, Berkeley, over many years. The subject is a part of asymptotics in statistics, organized around a few central ideas. The presentation proceeds from the general to the particular since this seemed the best way to emphasize the basic concepts. The reader is expected to have been exposed to statistical thinking and methodology, as expounded for instance in the book by H. Cramer [1946] or the more recent text by P. Bickel and K. Doksum [1977]. Another pos­ sibility, closer to the present in spirit, is Ferguson [1967]. Otherwise the reader is expected to possess some mathematical maturity, but not really a great deal of detailed mathematical knowledge. Very few mathematical objects are used; their assumed properties are simple; the results are almost always immediate consequences of the definitions. Some objects, such as vector lattices, may not have been included in the standard background of a student of statistics. For these we have provided a summary of relevant facts in the Appendix. The basic structures in the whole affair are systems that Blackwell called "experiments" and "transitions" between them. An "experiment" is a mathe­ matical abstraction intended to describe the basic features of an observational process if that process is contemplated in advance of its implementation. Typically, an experiment consists of a set E> of theories about what may happen in the observational process.

1 Experiments―Decision Spaces.- 1 Introduction.- 2 Vector Lattices―L-Spaces―Transitions.- 3 Experiments―Decision Procedures.- 4 A Basic Density Theorem.- 5 Building Experiments from Other Ones.- 6 Representations―Markov Kernels.- 2 Some Results from Decision Theory: Deficiencies.- 1 Introduction.- 2 Characterization of the Spaces of Risk Functions: Minimax Theorem.- 3 Deficiencies; Distances.- 4 The Form of Bayes Risks―Choquet Lattices.- 3 Likelihood Ratios and Conical Measures.- 1 Introduction.- 2 Homogeneous Functions of Measures.- 3 Deficiencies for Binary Experiments: Isometries.- 4 Weak Convergence of Experiments.- 5 Boundedly Complete Experiments.- 6 Convolutions: Hellinger Transforms.- 7 The Blackwell-Sherman-Stein Theorem.- 4 Some Basic Inequalities.- 1 Introduction.- 2 Hellinger Distances: L1-Norm.- 3 Approximation Properties for Likelihood Ratios.- 4 Inequalities for Conditional Distributions.- 5 Sufficiency and Insufficiency.- 1 Introduction.- 2 Projections and Conditional Expectations.- 3 Equivalent Definitions for Sufficiency.- 4 Insufficiency.- 5 Estimating Conditional Distributions.- 6 Domination, Compactness, Contiguity.- 1 Introduction.- 2 Definitions and Elementary Relations.- 3 Contiguity.- 4 Strong Compactness and a Result of D. Lindae.- 7 Some Limit Theorems.- 1 Introduction.- 2 Convergence in Distribution or in Probability.- 3 Distinguished Sequences of Statistics.- 4 Lower-Semicontinuity for Spaces of Risk Functions.- 5 A Result on Asymptotic Admissibility.- 8 Invariance Properties.- 1 Introduction.- 2 The Markov―Kakutani Fixed Point Theorem.- 3 A Lifting Theorem and Some Applications.- 4 Automatic Invariance of Limits.- 5 Invariant Exponential Families.- 6 The Hunt-Stein Theorem and Related Results.- 9 Infinitely Divisible, Gaussian, and Poisson Experiments.- 1 Introduction.- 2 Infinite Divisibility.- 3 Gaussian Experiments.- 4 Poisson Experiments.- 5 A Central Limit Theorem.- 10 Asymptotically Gaussian Experiments: Local Theory.- 1 Introduction.- 2 Convergence to a Gaussian Shift Experiment.- 3 A Framework which Arises in Many Applications.- 4 Weak Convergence of Distributions.- 5 An Application of a Martingale Limit Theorem.- 6 Asymptotic Admissibility and Minimaxity.- 11 Asymptotic Normality―Global.- 1 Introduction.- 2 Preliminary Explanations.- 3 Construction of Centering Variables.- 4 Definitions Relative to Quadratic Approximations.- 5 Asymptotic Properties of the Centerings $$\hat{Z}$$.- 6 The Asymptotically Gaussian Case.- 7 Some Particular Cases.- 8 Reduction to the Gaussian Case by Small Distortions.- 9 The Standard Tests and Confidence Sets.- 10 Minimum ?2 and Relatives.- 12 Posterior Distributions and Bayes Solutions.- 1 Introduction.- 2 Inequalities on Conditional Distributions.- 3 Asymptotic behavior of Bayes Procedures.- 4 Approximately Gaussian Posterior Distributions.- 13 An Approximation Theorem for Certain Sequential Experiments.- 1 Introduction.- 2 Notations and Assumptions.- 3 Basic Auxiliary Lemmas.- 4 Reduction Theorems.- 5 Remarks on Possible Applications.- 14 Approximation by Exponential Families.- 1 Introduction.- 2 A Lemma on Approximate Sufficiency.- 3 Homogeneous Experiments of Finite Rank.- 4 Approximation by Experiments of Finite Rank.- 5 Construction of Distinguished Sequences of Estimates.- 15 Sums of Independent Random Variables.- 1 Introduction.- 2 Concentration Inequalities.- 3 Compactness and Shift-Compactness.- 4 Poisson Exponentials and Approximation Theorems.- 5 Limit Theorems and Related Results.- 6 Sums of Independent Stochastic Processes.- 16 Independent Observations.- 1 Introduction.- 2 Limiting Distributions for Likelihood Ratios.- 3 Conditions for Asymptotic Normality.- 4 Tests and Distances.- 5 Estimates for Finite Dimensional Parameter Spaces.- 6 The Risk of Formal Bayes Procedures.- 7 Empirical Measures and Cumulatives.- 8 Empirical Measures on Vapnik-?ervonenkis Classes.- 17 Independent Identically Distributed Observations.- 1 Introduction.- 2 Hilbert Spaces Around a Point.- 3 A Special Role for $$\sqrt{n}$$: Differentiability in Quadratic Mean.- 4 Asymptotic Normality for Rates Other than $$\sqrt{n}$$.- 5 Existence of Consistent Estimates.- 6 Estimates Converging at the $$\sqrt{n}$$-Rate.- 7 The Behavior of Posterior Distributions.- 8 Maximum Likelihood.- 9 Some Cases where the Number of Observations Is Random.- Appendix: Results from Classical Analysis.- 1 The Language of Set Theory.- 2 Topological Spaces.- 3 Uniform Spaces.- 4 Metric Spaces.- 5 Spaces of Functions.- 6 Vector Spaces.- 7 Vector Lattices.- 8 Vector Lattices Arising from Experiments.- 9 Lattices of Numerical Functions.- 10 Extensions of Positive Linear Functions.- 11 Smooth Linear Functionals.- 12 Derivatives and Tangents.