Computer Processing of Electron Microscope Images

Towards the end of the 1960s, a number of quite different circumstances combined to launch a period of intense activity in the digital processing of electron micro­ graphs. First, many years of work on correcting the resolution-limiting aberrations of electron microscope objectives had shown that these optical impediments to very high resolution could indeed be overcome, but only at the cost of immense exper­ imental difficulty; thanks largely to the theoretical work of K. -J. Hanszen and his colleagues and to the experimental work of F. Thon, the notions of transfer func­ tions were beginning to supplant or complement the concepts of geometrical optics in electron optical thinking; and finally, large fast computers, capable of manipu­ lating big image matrices in a reasonable time, were widely accessible. Thus the idea that recorded electron microscope images could be improved in some way or rendered more informative by subsequent computer processing gradually gained ground. At first, most effort was concentrated on three-dimensional reconstruction, particu­ larly of specimens with natural symmetry that could be exploited, and on linear operations on weakly scattering specimens (Chap. l). In 1973, however, R. W. Gerchberg and W. O. Saxton described an iterative algorithm that in principle yielded the phase and amplitude of the electron wave emerging from a strongly scattering speci­ men.

1. Image Processing Based on the Linear Theory of Image Formation.- 1.1 Transfer Functions.- 1.2 Transfer Functions with Partially Coherent Illumination.- 1.3 Practical Exploitation of the Linear Relationship.- 1.3.1 Measurement of the Microscope Operating Characteristics.- 1.3.2 On-Line Processing.- 1.3.3 Filtering and Reconstruction.- References.- 2. Recovery of Specimen Information for Strongly Scattering Objects.- 2.1 Image Formation and Interpretation.- 2.1.1 Recapitulation of Coherent Image Formation.- 2.1.2 Interpreting the Specimen Wave.- 2.1.3 Rendering Images Discrete.- 2.2 Methods Iterating the Linear Theory Solution.- 2.3 Methods Requiring No Special Apertures.- 2.3.1 The Data Used.- 2.3.2 The Iterative Transform Algorithm.- 2.3.3 Examples and Practical Applications.- 2.3.4 Uniqueness.- 2.3.5 Periodic Images and Complex Zeros.- 2.3.6 Other Methods of Analysis.- 2.3.7 Conclusions.- 2.4 Methods Using Half-Plane Apertures.- 2.4.1 Hilbert Transforms.- 2.4.2 Logarithmic Hilbert Transforms.- 2.4.3 Real Aperture Shapes.- 2.4.4 Dark-Field Conditions.- 2.5 Analytic Wave Functions and Complex Zeros.- 2.5.1 Zero-Distributions and Zero Flipping.- 2.5.2 An Example.- 2.5.3 Immediate Applications.- 2.5.4 Reformulation of Zero Flipping.- 2.5.5 Two-Dimensional Extensions.- 2.6 Holography.- 2.6.1 The Linear Case.- 2.6.2 The General Case.- 2.6.3 Some Particular Cases.- 2.6.4 Nonplanar Reference Waves.- 2.7 Ptychography and Related Methods.- 2.8 Bright-Field/Dark-Field Subtraction.- 2.9 Other Perspectives.- 2.9.1 Coherence.- 2.9.2 Inelastic Scattering.- 2.9.3 Recording Noise and Radiation Damage.- 2.9.4 Practical Details of Computer Processing.- 2.9.5 Other Constraints.- 2.10 Conclusions.- References.- 3. Computer Reconstruction of Regular Biological Objects.- 3.1 The Biological Object.- 3.1.1 General Remarks.- 3.1.2 Regular Biological Objects.- 3.1.3 Chemical and Physical Processing of the Object.- 3.1.4 Contrast in Bright-Field Images.- 3.1.5 Radiation Damage.- 3.2 Fourier Processing of Electron Micrographs.- 3.2.1 Quantization and Preprocessing.- 3.2.2 The Whittaker-Shannon Sampling Theorem.- 3.2.3 Fourier Transforms of Regular Objects.- 3.2.4 Processing of Two-Dimensional Structures with Translational Symmetry.- 3.2.5 Rotational Filtering.- 3.2.6 Three-Dimensional Reconstruction of Objects with Helical Symmetry.- 3.2.7 Three-Dimensional Reconstruction of Particles with Icosahedral Symmetry.- 3.3 Recent Applications to Image Processing of Regular Biological Structure.- 3.3.1 One-Dimensional Filtering: Tropomyosin Paracrystal Structure.- 3.3.2 The Structure of Polyheads.- 3.3.3 The Structure of Ribosomes.- 3.3.4 The Structure of the Purple Membrane.- 3.3.5 A Correction for Distorted Images.- 3.3.6 Rotational Filtering of Base Plates.- 3.3.7 The Structure of the Contractile Sheath from Bacteriophage Mu.- 3.3.8 The Three-Dimensional Structure of an Icosahedral Virus Particle.- 3.4. Outlook.- References.- 4. Three-Dimensional Structure Determination by Electron Microscopy (Nonperiodic Specimens).- 4.1 History and General Discussion of the Subject.- 4.2 The Fundamental Theoretical Background.- 4.2.1 The Use of a CTEM as a Diffractometer.- 4.2.2 The Description of Structures in Three-Dimensional Electron Microscopy.- 4.2.3 Two-Dimensional Reconstruction (Image Filtering).- 4.2.4 The Projection Theorem.- 4.3 The Problem of Reconstruction.- 4.3.1 The Whittaker-Shannon-Type Interpolation.- 4.3.2 An Alternative Way of Incorporating the Finite Body Concept.- 4.3.3 Back-Projection and Filtered Back-Projection.- a) Simple Back-Projection.- b) Filtered Back-Projection.- c) The Influence of the Reconstruction Body.- d) The Influence of Restricted Tilting Angle.- 4.3.4 Conical Tilting.- 4.3.5 Reconstruction by Series Expansion.- a) The Cormack Method.- b) Aliasing.- 4.3.6 Algebraic Reconstruction in Direct Space.- 4.3.7 Reconstruction of an “Infinite” Platelet with Restricted Tilting Angle.- a) One-Dimensional Whittaker-Shannon Treatment of Single-Axis Tilting.- b) Reconstruction by Interpolation in Projection Space.- c) The Partially Defined “Unit Cell”.- 4.3.8 Determination of a Common Origin of the Projections.- 4.4 Aspects for the Future.- 4.4.1 The “Atom” Constraint.- 4.4.2 The Use of a STEM as a Diffractometer.- References.- 5. The Role of Correlation Techniques in Computer Image Processing.- 5.1 Correlation Functions.- 5.1.1 The Cross-Correlation Function.- 5.1.2 The Autocorrelation Function.- 5.1.3 Correlation and Similarity.- 5.2 Computation.- 5.3 Some Important Theorems.- 5.3.1 CCFs of Images Containing Signal and Noise.- 5.3.2 The CCF of Blurred Signals.- 5.3.3 Some Thoughts on Signal, Noise, and Correlation.- 5.4 Determination of Relative Positions.- 5.4.1 Translation.- 5.4.2 Alignment of Projections.- 5.4.3 Centering of a Centrosymmetric Particle.- 5.4.4 Determination of Relative Orientation.- 5.5 Matched Filtering.- 5.6 Characterization of Instrument Conditions.- 5.7 Signal-to-Noise Ratio Measurement.- 5.7.1 Theory.- 5.7.2 Measurement.- 5.7.3 Consequence for Phase Contrast Microscopy.- 5.7.4 Generalized Signal-to-Noise Ratio Measurement.- 5.8 Conclusions.- References.- 6. Holographic Methods in Electron Microscopy.- 6.1 Historical Background.- 6.2 Holographic Schemes.- 6.2.1 The Generalized Hologram.- 6.2.2 In-Line Fresnel and Fraunhofer Holograms.- 6.2.3 Sideband Fresnel Holograms.- 6.2.4 Fourier Transform Holograms.- 6.2.5 Single-Sideband Holograms.- 6.2.6 Zone Plate Interpretation.- 6.3 Experimental Electron Holography.- 6.4 Contrast Transfer and Holography.- 6.4.1 In-Line Fresnel Hologram.- 6.4.2 Fresnel Sideband Hologram.- 6.4.3 Single-Sideband Hologram.- 6.4.4 The Effect of Partial Coherence on Resolution.- 6.5 Additional Reading.- 6.6 Conclusions.- References.- 7. Analog Computer Processing of Scanning Transmission Electron Microscope Images.- 7.1 Organization.- 7.2 Characteristics of Analog Processing.- 7.2.1 Grey Scale Modification.- 7.2.2 Filters.- 7.2.3 Signal Mixing.- 7.3 Types of Signals Available in the STEM.- 7.3.1 Basic Signals.- 7.3.2 Detected Signals.- 7.3.3 Extraction of Basic Signals.- 7.3.4 Normalization.- 7.4 Instrumental Characteristics.- 7.4.1 The Analog Processor.- 7.4.2 Display System.- 7.5 Applications.- 7.5.1 Basic Operations.- 7.5.2 Color Conversion Techniques.- 7.6 Conclusion.- References.- Appendix: Publication Details of International and European Congresses on Electron Microscopy.- Additional References with Titles.