Networks of Learning Automata

Networks of Learning Automata: Techniques for Online Stochastic Optimization is a comprehensive account of learning automata models with emphasis on multiautomata systems. It considers synthesis of complex learning structures from simple building blocks and uses stochastic algorithms for refining probabilities of selecting actions. Mathematical analysis of the behavior of games and feedforward networks is provided. Algorithms considered here can be used for online optimization of systems based on noisy measurements of performance index. Also, algorithms that assure convergence to the global optimum are presented. Parallel operation of automata systems for improving speed of convergence is described. The authors also include extensive discussion of how learning automata solutions can be constructed in a variety of applications.

1. Introduction.- 1.1 Machine Intelligence and Learning.- 1.2 Learning Automata.- 1.3 The Finite Action Learning Automaton (FALA).- 1.3.1 The Automaton.- 1.3.2 The Random Environment.- 1.3.3 Operation of FALA.- 1.4 Some Classical Learning Algorithms.- 1.4.1 Linear Reward-Inaction (LR?I) Algorithm.- 1.4.2 Other Linear Algorithms.- 1.4.3 Estimator Algorithms.- 1.4.4 Simulation Results.- 1.5 The Discretized Probability FALA.- 1.5.1 DLR?I Algorithm.- 1.5.2 Discretized Pursuit Algorithm.- 1.6 The Continuous Action Learning Automaton (CALA).- 1.6.1 Analysis of the Algorithm.- 1.6.2 Simulation Results.- 1.6.3 Another Continuous Action Automaton.- 1.7 The Generalized Learning Automaton (GLA).- 1.7.1 Learning Algorithm.- 1.7.2 An Example.- 1.8 The Parameterized Learning Automaton (PLA).- 1.8.1 Learning Algorithm.- 1.9 Multiautomata Systems.- 1.10 Supplementary Remarks.- 2. Games of Learning Automata.- 2.1 Introduction.- 2.2 A Multiple Payoff Stochastic Game of Automata.- 2.2.1 The Learning Algorithm.- 2.3 Analysis of the Automata Game Algorithm.- 2.3.1 Analysis of the Approximating ODE.- 2.4 Game with Common Payoff.- 2.5 Games of FALA.- 2.5.1 Common Payoff Games of FALA.- 2.5.2 Pursuit Algorithm for a Team of FALA.- 2.5.3 Other Types of Games.- 2.6 Common Payoff Games of CALA.- 2.6.1 Stochastic Approximation Algorithms and CALA.- 2.7 Applications.- 2.7.1 System Identification.- 2.7.2 Learning Conjunctive Concepts.- 2.8 Discussion.- 2.9 Supplementary Remarks.- 3. Feedforward Networks.- 3.1 Introduction.- 3.2 Networks of FALA.- 3.3 The Learning Model.- 3.3.1 G-Environment.- 3.3.2 The Network.- 3.3.3 Network Operation.- 3.4 The Learning Algorithm.- 3.5 Analysis.- 3.6 Extensions.- 3.6.1 Other Network Structures.- 3.6.2 Other Learning Algorithms.- 3.7 Convergence to the Global Maximum.- 3.7.1 The Network.- 3.7.2 The Global Learning Algorithm.- 3.7.3 Analysis of the Global Algorithm.- 3.8 Networks of GLA.- 3.9 Discussion.- 3.10 Supplementary Remarks.- 4. Learning Automata for Pattern Classification.- 4.1 Introduction.- 4.2 Pattern Recognition.- 4.3 Common Payoff Game of Automata for PR.- 4.3.1 Pattern Classification with FALA.- 4.3.2 Pattern Classification with CALA.- 4.3.3 Simulations.- 4.4 Automata Network for Pattern Recognition.- 4.4.1 Simulations.- 4.4.2 Network of Automata for Learning Global Maximum.- 4.5 Decision Tree Classifiers.- 4.5.1 Learning Decision Trees using GLA and CALA.- 4.5.2 Learning Piece-wise Linear Functions.- 4.6 Discussion.- 4.7 Supplementary Remarks.- 5. Parallel Operation of Learning Automata.- 5.1 Introduction.- 5.2 Parallel Operation of FALA.- 5.2.1 Analysis.- 5.2.2 ?-optimality.- 5.2.3 Speed of Convergence and Module Size.- 5.2.4 Simulation Studies.- 5.3 Parallel Operation of CALA.- 5.4 Parallel Pursuit Algorithm.- 5.4.1 Simulation Studies.- 5.5 General Procedure.- 5.6 Parallel Operation of Games of FALA.- 5.6.1 Analysis.- 5.6.2 Common Payoff Game.- 5.7 Parallel Operation of Networks of FALA.- 5.7.1 Analysis.- 5.7.2 Modules of Parameterized Learning Automata (PLA).- 5.7.3 Modules of Generalized Learning Automata (GLA).- 5.7.4 Pattern Classification Example.- 5.8 Discussion.- 5.9 Supplementary Remarks.- 6. Some Recent Applications.- 6.1 Introduction.- 6.2 Supervised Learning of Perceptual Organization in Computer Vision.- 6.3 Distributed Control of Broadcast Communication Networks.- 6.4O ther Applications.- 6.5 Discussion.- Epilogue.- Appendices.- A The ODE Approach to Analysis of Learning Algorithms.- A.I Introduction.- A.2 Derivation of the ODE Approximation.- A.2.1 Assumptions.- A.2.2 Analysis.- A.3 Approximating ODEs for Some Automata Algorithms.- A.3.2 The CALA Algorithm.- A.3.3 Automata Team Algorithms.- A.4 Relaxing the Assumptions.- B Proofs of Convergence for Pursuit Algorithm.- B.1 Proof of Theorem 1.1.- B.2 Proof of Theorem 5.7.- C Weak Convergence and SDE Approximations.- C.I Introduction.- C.2 Weak Convergence.- C.3 Convergence to SDE.- C.3.1 Application to Global Algorithms.- C.4 Convergence to ODE.- References.