H. M. Schwartz

Multi-Agent Machine Learning

A Reinforcement Approach

• Gebundenes Buch

Produktbeschreibung

A branch of artificial intelligence, machine learning pertains to algorithm design and development that evolves computer behavior. Multi-Agent Machine Learning applies these principles to multi-differential games and offers applications in game theory and mobile robotics. Touching upon value functions, Markov games, Nash equilibrium, infinitesimal gradient methods, and friend or foe Q-learning, among many other topics, this book provides thorough coverage of multi-differential games for researchers and students.

---

The book begins with a chapter on traditional methods of supervised learning, covering recursive least squares learning, mean square error methods, and stochastic approximation. Chapter 2 covers single agent reinforcement learning. Topics include learning value functions, Markov games, and TD learning with eligibility traces. Chapter 3 discusses two player games including two player matrix games with both pure and mixed strategies. Numerous algorithms and examples are presented. Chapter 4 covers learning in multi-player games, stochastic games, and Markov games, focusing on learning multi-player grid games-two player grid games, Q-learning, and Nash Q-learning. Chapter 5 discusses differential games, including multi player differential games, actor critique structure, adaptive fuzzy control and fuzzy interference systems, the evader pursuit game, and the defending a territory games. Chapter 6 discusses new ideas on learning within robotic swarms and the innovative idea of the evolution of personality traits.

- Framework for understanding a variety of methods and approaches in multi-agent machine learning.

- Discusses methods of reinforcement learning such as a number of forms of multi-agent Q-learning

- Applicable to research professors and graduate students studying electrical and computer engineering, computer science, and mechanical and aerospace engineering

Produktdetails

• Verlag: Wiley & Sons
• 1. Auflage
• Seitenzahl: 256
• Erscheinungstermin: September 2014
• Englisch
• Abmessung: 240mm x 161mm x 18mm
• Gewicht: 480g
• ISBN-13: 9781118362082
• ISBN-10: 111836208X
• Artikelnr.: 40055467

Autorenporträt

Howard M. Schwartz, PhD, received his B.Eng. Degree from McGill University, Montreal, Canada in une 1981 and his MS Degree and PhD Degree from MIT, Cambridge, USA in 1982 and 1987 respectively. He is currently a professor in systems and computer engineering at Carleton University, Canada. His research interests include adaptive and intelligent control systems, robotic, artificial intelligence, system modelling, system identification, and state estimation.

Inhaltsangabe

Preface ix Chapter 1 A Brief Review of Supervised Learning 1 1.1 Least
Squares Estimates 1 1.2 Recursive Least Squares 5 1.3 Least Mean Squares 6
1.4 Stochastic Approximation 10 References 11 Chapter 2 Single-Agent
Reinforcement Learning 12 2.1 Introduction 12 2.2 n-Armed Bandit Problem 13
2.3 The Learning Structure 15 2.4 The Value Function 17 2.5 The Optimal
Value Functions 18 2.5.1 The Grid World Example 20 2.6 Markov Decision
Processes 23 2.7 Learning Value Functions 25 2.8 Policy Iteration 26 2.9
Temporal Difference Learning 28 2.10 TD Learning of the State-Action
Function 30 2.11 Q-Learning 32 2.12 Eligibility Traces 33 References 37
Chapter 3 Learning in Two-Player Matrix Games 38 3.1 Matrix Games 38 3.2
Nash Equilibria in Two-Player Matrix Games 42 3.3 Linear Programming in
Two-Player Zero-Sum Matrix Games 43 3.4 The Learning Algorithms 47 3.5
Gradient Ascent Algorithm 47 3.6 WoLF-IGA Algorithm 51 3.7 Policy Hill
Climbing (PHC) 52 3.8 WoLF-PHC Algorithm 54 3.9 Decentralized Learning in
Matrix Games 57 3.10 Learning Automata 59 3.11 Linear Reward-Inaction
Algorithm 59 3.12 Linear Reward-Penalty Algorithm 60 3.13 The Lagging
Anchor Algorithm 60 3.14 LR.I Lagging Anchor Algorithm 62 3.14.1 Simulation
68 References 70 Chapter 4 Learning in Multiplayer Stochastic Games 73 4.1
Introduction 73 4.2 Multiplayer Stochastic Games 75 4.3 Minimax-Q Algorithm
79 4.3.1 2 ×2 Grid Game 80 4.4 Nash Q-Learning 87 4.4.1 The Learning
Process 95 4.5 The Simplex Algorithm 96 4.6 The Lemke-Howson Algorithm 100
4.7 Nash-Q Implementation 107 4.8 Friend-or-Foe Q-Learning 111 4.9 Infinite
Gradient Ascent 112 4.10 Policy Hill Climbing 114 4.11 WoLF-PHC Algorithm
114 4.12 Guarding a Territory Problem in a Grid World 117 4.12.1 Simulation
and Results 119 4.13 Extension of LR.I Lagging Anchor Algorithm to
Stochastic Games 125 4.14 The Exponential Moving-Average Q-Learning (EMA
Q-Learning) Algorithm 128 4.15 Simulation and Results Comparing EMA
Q-Learning to Other Methods 131 4.15.1 Matrix Games 131 4.15.2 Stochastic
Games 134 References 141 Chapter 5 Differential Games 144 5.1 Introduction
144 5.2 A Brief Tutorial on Fuzzy Systems 146 5.2.1 Fuzzy Sets and Fuzzy
Rules 146 5.2.2 Fuzzy Inference Engine 148 5.2.3 Fuzzifier and Defuzzifier
151 5.2.4 Fuzzy Systems and Examples 152 5.3 Fuzzy Q-Learning 155 5.4 Fuzzy
Actor-Critic Learning 159 5.5 Homicidal Chauffeur Differential Game 162 5.6
Fuzzy Controller Structure 165 5.7 Q(lambda)-Learning Fuzzy Inference
System 166 5.8 Simulation Results for the Homicidal Chauffeur 171 5.9
Learning in the Evader-Pursuer Game with Two Cars 174 5.10 Simulation of
the Game of Two Cars 177 5.11 Differential Game of Guarding a Territory 180
5.12 Reward Shaping in the Differential Game of Guarding a Territory 184
5.13 Simulation Results 185 5.13.1 One Defender Versus One Invader 185
5.13.2 Two Defenders Versus One Invader 191 References 197 Chapter 6 Swarm
Intelligence and the Evolution of Personality Traits 200 6.1 Introduction
200 6.2 The Evolution of Swarm Intelligence 200 6.3 Representation of the
Environment 201 6.4 Swarm-Based Robotics in Terms of Personalities 203 6.5
Evolution of Personality Traits 206 6.6 Simulation Framework 207 6.7 A
Zero-Sum Game Example 208 6.7.1 Convergence 208 6.7.2 Simulation Results
214 6.8 Implementation for Next Sections 216 6.9 Robots Leaving a Room 218
6.10 Tracking a Target 221 6.11 Conclusion 232 References 233 Index 237