Models of Neural Networks

Since the appearance of Vol. 1 of Models of Neural Networks in 1991, the theory of neural nets has focused on two paradigms: information coding through coherent firing of the neurons and functional feedback. Information coding through coherent neuronal firing exploits time as a cardinal degree of freedom. This capacity of a neural network rests on the fact that the neuronal action potential is a short, say 1 ms, spike, localized in space and time. Spatial as well as temporal correlations of activity may represent different states of a network. In particular, temporal correlations of activity may express that neurons process the same "object" of, for example, a visual scene by spiking at the very same time. The traditional description of a neural network through a firing rate, the famous S-shaped curve, presupposes a wide time window of, say, at least 100 ms. It thus fails to exploit the capacity to "bind" sets of coherently firing neurons for the purpose of both scene segmentation and figure-ground segregation. Feedback is a dominant feature of the structural organization of the brain. Recurrent neural networks have been studied extensively in the physical literature, starting with the ground breaking work of John Hop­ field (1982).

1. Coding and Information Processing in Neural Networks.- 1.1 Description of Neural Activity.- 1.1.1 Spikes, Rates, and Neural Assemblies.- 1.2 Oscillator Models.- 1.2.1 The Kuramoto Model.- 1.2.2 Oscillator Models in Action.- 1.3 Spiking Neurons.- 1.3.1 Hodgkin-Huxley Model.- 1.3.2 Integrate-and-Fire Model.- 1.3.3 Spike Response Model.- 1.4 A Network of Spiking Neurons.- 1.4.1 Stationary Solutions ― Incoherent Firing.- 1.4.2 Coherent Firing ― Oscillatory Solutions.- 1.5 Hebbian Learning of Spatio-Temporal Spike Patterns.- 1.5.1 Synaptic Organization of a Model Network.- 1.5.2 Hebbian Learning.- 1.5.3 Low-Activity Random Patterns.- 1.5.4 Discussion.- 1.6 Summary and Conclusions.- References.- 2. The Correlation Theory of Brain Function.- Foreword.- 2.1 Introduction.- 2.2 Conventional Brain Theory.- 2.2.1 Localization Theory.- 2.2.2 The Problem of Nervous Integration.- 2.2.3 Proposed Solutions.- 2.3 The Correlation Theory of Brain Function.- 2.3.1 Modifications of Conventional Theory.- 2.3.2 Elementary Discussion.- 2.3.3 Network Structures.- 2.3.4 Applications of Correlation Theory.- 2.4 Discussion.- 2.4.1 The Text Analogy.- 2.4.2 The Bandwidth Problem.- 2.4.3 Facing Experiment.- 2.4.4 Conclusions.- References.- 3. Firing Rates and Well-Timed Events in the Cerebral Cortex.- 3.1 Measuring the Activity of Nerve Cells.- 3.2 Rate Functions and Stationary Point Processes.- 3.3 Rate Functions for Nonstationary Point Processes.- 3.4 Rate Functions and Singular Events.- References.- 4. The Role of Synchrony in Neocortical Processing and Synaptic Plasticity.- 4.1 Introduction.- 4.2 Pattern Processing and the Binding Problem.- 4.3 Evidence for Dynamic Interactions Between Spatially Distributed Neurons.- 4.4 Stimulus-Dependent Changes of Synchronization Probability.- 4.5 Synchronization Between Areas.- 4.6 The Synchronizing Connections.- 4.7 Experience-Dependent Modifications of Synchronizing Connections and Synchronization Probabilities.- 4.8 Correlation Between Perceptual Deficits and Response Synchronization in Strabismic Amblyopia.- 4.9 The Relation Between Synchrony and Oscillations.- 4.10 Rhythm Generating Mechanisms.- 4.11 The Duration of Coherent States.- 4.12 Synchronization and Attention.- 4.13 The Role of Synchrony in Synaptic Plasticity.- 4.14 The Role of Oscillations in Synaptic Plasticity.- 4.15 Outlook.- 4.16 Concluding Remarks.- References.- 5. Associative Binding and Segregation in a Network of Spiking Neurons.- 5.1 Introduction.- 5.2 Spike Response Model.- 5.2.1 Starting Point.- 5.2.2 Neurons and Synapses.- 5.3 Theory of Locking.- 5.3.1 Equation of Motion.- 5.3.2 Stationary States.- 5.3.3 Oscillatory States.- 5.3.4 A Locking Condition.- 5.3.5 Weak Locking.- 5.4 Simulation Results.- 5.4.1 Three Scenarios.- 5.4.2 Interpretation.- 5.4.3 Spike Raster.- 5.4.4 Synchronization Between Two Hemispheres.- 5.5 Application to Binding and Segmentation.- 5.5.1 Feature Linking 201.- 5.5.2 Pattern Segmentation.- 5.5.3 Switching Between Patterns.- 5.6 Context Sensitive Binding in a Layered Network with Feedback.- 5.7 Discussion.- 5.8 Conclusions.- References.- 6. Modeling the Sensory Computations of the Olfactory Bulb.- 6.1 Introduction.- 6.2 Anatomical and Physiological Background.- 6.3 Modeling the Neural Oscillations in the Olfactory Bulb.- 6.3.1 General Model Structure.- 6.3.2 The Olfactory Bulb as a Group of Coupled Nonlinear Oscillators.- 6.3.3 Explanation of Bulbar Activities.- 6.4 A Model of Odor Recognition and Segmentation in the Olfactory Bulb.- 6.4.1 Emergence of Oscillations Detects Odors: Patterns of Oscillations Code the Odor Identity and Strength.- 6.4.2 Odor Segmentation in the Olfactory Bulb ― Olfactory Adaptation.- 6.4.3 Olfactory Psychophysics ― Cross-Adaptation, Sensitivity Enhancement, and Cross-Enhancement.- 6.5 A Model of Odor Segmentation Through Odor Fluctuation Analysis.- 6.5.1 A Different Olfactory Environment and a Different Task.- 6.5.2 Odor Segmentation in an Adaptive Network.- 6.6 Discussion.- References.- 7. Detecting Coherence in Neuronal Data.- 7.1 Introduction.- 7.1.1 Correlations and Statistical Dependency.- 7.2 Time Resolved Detection of Coherence.- 7.2.1 Statistical Dependency of Continuous Signals.- 7.2.2 Detection of Coherent Patterns of Activities.- 7.3 Memory and Switching in Local Field Potentials from Cat Visual Cortex.- 7.3.1 Temporal Coherence of Local Field Potentials.- 7.3.2 Spatial Coherence of Local Field Potentials.- 7.4 A Model-Dependent Approach.- 7.4.1 Renewal Dynamics.- 7.4.2 Switching in the Assembly Dynamics.- 7.4.3 Network States are Hidden.- 7.4.4 Model Identification from Multiunit Activities.- 7.5 Memory and Switching in Multiunit Activities from Cat Visual Cortex.- 7.5.1 Identifying the Local Pool Dynamics.- 7.5.2 Testing the Assembly Hypothesis.- 7.6 Reconstruction of Synchronous Network States.- 7.7 Summary.- References.- 8. Hebbian Synaptic Plasticity: Evolution of the Contemporary Concept.- 8.1 Concept of a Hebbian Synapse.- 8.1.1 Contemporary Concept of a Hebbian Synaptic Modification.- 8.2 Experimental Evidence for Hebbian Synaptic Mechanisms.- 8.2.1 Induction of Hippocampal LTP.- 8.3 Biophysical Models of LTP Induction.- 8.3.1 Second-Generation Spine Model.- 8.4 Bidirectional Regulation of Synaptic Strength.- 8.4.1 Theoretical Representations of Generalized Hebbian Modifications.- 8.4.2 Experimental Evidence for Long-Term Synaptic Depression.- 8.5 Interaction Between Dendritic Signaling and Hebbian.- Learning.- References.- 9. Reentry and Dynamical Interactions of Cortical Networks.- 9.1 Introduction.- 9.2 Models of Cortical Integration.- 9.2.1 Dynamic Behavior of Single Neuronal Groups.- 9.2.2 Coupled Neuronal Groups.- 9.2.3 Figure-Ground Segregation.- 9.2.4 Cooperative Interactions Among Multiple Cortical Areas.- 9.3 Summary and Conclusion.- 9.3.1 Cortical Integration at Multiple Levels.- 9.3.2 Temporal Dynamics.- 9.3.3 Correlations and Behavior.- 9.3.4 Functions of Reentry.- References.