Introduction to Shannon Sampling and Interpolation Theory

Much of that which is ordinal is modeled as analog. Most computational engines on the other hand are dig- ital. Transforming from analog to digital is straightforward: we simply sample. Regaining the original signal from these samples or assessing the information lost in the sampling process are the fundamental questions addressed by sampling and interpolation theory. This book deals with understanding, generalizing, and extending the cardinal series of Shannon sampling theory. The fundamental form of this series states, remarkably, that a bandlimited signal is uniquely specified by its sufficiently close equally spaced samples. The contents of this book evolved from a set of lecture notes prepared for a graduate survey course on Shannon sampling and interpolation theory. The course was taught at the Department of Electrical Engineering at the University of Washington, Seattle. Each of the seven chapters in this book includes a list of references specific to that chapter. A sequel to this book will contain an extensive bibliography on the subject. The author has also opted to include solutions to selected exercises in the Appendix.

1 Introduction.- 1.1 The Cardinal Series.- 1.2 History.- 2 Fundamentals of Fourier Analysis and Stochastic Processes.- 2.1 Signal Classes.- 2.2 The Fourier Transform.- 2.2.1 The Fourier Series.- 2.2.1.1 Convergence.- 2.2.1.2 Orthogonal Basis Functions.- 2.2.2 Some Elementary Functions.- 2.2.3 Some Transforms of Elementary Functions.- 2.2.4 Other Properties.- 2.3 Stochastic Processes.- 2.3.1 First and Second Order Statistics.- 2.3.2 Stationary Processes.- 2.3.2.1 Power Spectral Density.- 2.3.2.2 Some Stationary Noise Models.- 2.3.2.3 Linear Systems with Stationary Stochastic Inputs.- 2.4 Exercises.- 3 The Cardinal Series.- 3.1 Interpretation.- 3.2 Proofs.- 3.2.1 Using Comb Functions.- 3.2.2 Fourier Series Proof.- 3.2.3 Papoulis’ Proof.- 3.3 Properties.- 3.3.1 Convergence.- 3.3.1.1 For Finite Energy Signals.- 3.3.1.2 For Bandlimited Functions with Finite Area Spectra.- 3.3.2 Trapezoidal Integration.- 3.3.2.1 Of Bandlimited Functions.- 3.3.2.2 Of Linear Integral Transforms.- 3.3.2.3 Parseval’s Theorem for the Cardinal Series.- 3.3.3 The Time-Bandwidth Product.- 3.4 Application to Spectra Containing Distributions.- 3.5 Application to Bandlimited Stochastic Processes.- 3.6 Exercises.- 4 Generalizations of the Sampling Theorem.- 4.1 Generalized Interpolation Functions.- 4.1.1 Oversampling.- 4.1.1.1 Sample Dependency.- 4.1.1.2 Relaxed Interpolation Formulae.- 4.1.2 Criteria for Generalized Interpolation Functions.- 4.1.2.1 Interpolation Functions.- 4.1.2.2 Reconstruction from a Filtered Signal’s Samples.- 4.2 Papoulis’ Generalization.- 4.2.1 Derivation.- 4.2.2 Interpolation Function Computation.- 4.2.3 Example Applications.- 4.2.3.1 Recurrent Nonuniform Sampling.- 4.2.3.2 Interlaced Signal-Derivative Sampling.- 4.2.3.3 Higher Order Derivative Sampling.- 4.2.3.4 Effects of Oversampling.- 4.3 Derivative Interpolation.- 4.3.1 Properties of the Derivative Kernel.- 4.4 A Relation Between the Taylor and Cardinal Series.- 4.5 Sampling Trigonometric Polynomials.- 4.6 Sampling Theory for Bandpass Functions.- 4.6.1 Heterodyned Sampling.- 4.6.2 Direct Bandpass Sampling.- 4.7 A Summary of Sampling Theorems for Directly Sampled Signals.- 4.8 Lagrangian Interpolation.- 4.9 Kramer’s Generalization.- 4.10 Exercises.- 5 Sources of Error.- 5.1 Effects of Additive Data Noise.- 5.1.1 On Cardinal Series Interpolation.- 5.1.1.1 Interpolation Noise Level.- 5.1.1.2 Effects of Oversampling and Filtering.- 5.1.2 Interpolation Noise Variance for Directly Sampled Signals.- 5.1.2.1 Interpolation with Lost Samples.- 5.1.2.2 Bandpass Functions.- 5.1.3 On Papoulis’ Generalization.- 5.1.3.1 Examples.- 5.1.3.2 Notes.- 5.1.4 On Derivative Interpolation.- 5.1.4.1 A Lower Bound on the NINV.- 5.1.4.2 Examples.- 5.2 Jitter.- 5.2.1 Filtered Cardinal Series Interpolation.- 5.2.2 Unbiased Interpolation from Jittered Samples.- 5.2.3 In Stochastic Bandlimited Signal Interpolation.- 5.2.3.1 NINV of Unbiased Restoration.- 5.2.3.2 Examples.- 5.3 Truncation Error.- 5.3.1 An Error Bound.- 5.3.2 Noisy Stochastic Signals.- 5.4 Exercises.- 6 The Sampling Theorem in Higher Dimensions.- 6.1 Multidimensional Fourier Analysis.- 6.1.1 Properties.- 6.1.1.1 Separability.- 6.1.1.2 Rotation, Scale and Transposition.- 6.1.1.3 Polar Representation.- 6.1.2 Fourier Series.- 6.1.2.1 Multidimensional Periodicity.- 6.1.2.2 The Fourier Series Expansion.- 6.2 The Multidimensional Sampling Theorem.- 6.2.1 The Nyquist Density.- 6.2.2 Generalized Interpolation Functions.- 6.2.2.1 Tightening the Integration Region.- 6.2.2.2 Allowing Slower Roll Off.- 6.3 Restoring Lost Samples.- 6.3.1 Restoration Formulae.- 6.3.2 Noise Sensitivity.- 6.3.2.1 Filtering.- 6.3.2.2 Deleting Samples from Optical Images.- 6.4 Periodic Sample Decimation and Restoration.- 6.4.1 Preliminaries.- 6.4.2 First Order Decimated Sample Restoration.- 6.4.3 Sampling Below the Nyquist Density.- 6.4.4 Higher Order Decimation.- 6.5 Raster Sampling.- 6.6 Exercises.- 7 Continuous Sampling.- 7.1 Interpolation From Periodic Continuous Samples.- 7.1.1 The Restoration Algorithm.- 7.1.2 Noise Sensitivity.- 7.1.2.1 White Noise.- 7.1.2.2 Colored Noise.- 7.1.3 Observations.- 7.1.3.1 Comparison with the NINV of the Cardinal Series.- 7.1.3.2 In the Limit as an Extrapolation Algorithm.- 7.1.4 Application to Interval Interpolation.- 7.2 Prolate Spheroidal Wave Functions.- 7.2.1 Properties.- 7.2.2 Application to Extrapolation.- 7.2.3 Application to Interval Interpolation.- 7.3 The Papoulis-Gerchberg Algorithm.- 7.3.1 The Basic Algorithm.- 7.3.2 Proof of the PGA using PSWF’s.- 7.3.3 Geometrical Interpretation in a Hilbert Space.- 7.3.4 Remarks.- 7.4 Exercises.