Distributed Adaptive Systems: Feedback Control with Evolutionary Games

点击翻译按钮获取中文摘要

The proposed research is to develop feedback control theoretic methods for learning and adaptation inmultiagent systems and to investigate their potential as analysis and design tools for distributed adaptivesystems. The proposed work stems from recent innovations at UCLA on the role of feedback control in thegame theoretic framework of distributed learning. This recent work has overcome long standing perceivedobstacles in game theoretic learning and opens new opportunities for distributed system design.The proposed research will emphasize underlying mathematical models of multiagent learning. Theresearch directions to be explored are:Advanced analysis of feedback control based learning in games: The introduction of feedback controlmethods in learning in games is very recent, and there are many important unresolved issues. Proposedtopics of interest include strategic advantage of feedback control based methods, analysis of systems withheterogeneous learning algorithms, dynamic network interconnections, and supervised switching for learning.Continuum action spaces: The learning in games framework currently applies to learning among a finiteset of choices. This direction involves learning with decisions over a continuum, such as dynamic resourceallocation problems. Issues include new feedback control based approaches to distributed optimization,severely limited information structures, and complementary sharing of information structures.Multiagent games with state evolution: The learning in games framework uses a static game setup.The state variable is the state of learning, but not inherent states of subsystems or the environment. Thisdirection considers learning for games with internal states, also known as Markov or stochastic games. Issuesinclude changes of time-scale for expert selection, feedback control based Q-learning, and integrator action.Application to Evolvable Hardware: Evolvable hardware is an emerging area that uses notions inspiredby biological evolution for the design of reconfigurable, self-organizing hardware. The proposed concepts onmultiagent may be viewed as an engineered evolution with its blending of feedback control and multiagentlearning. This project will use the evolvable hardware paradigm to illustrate and motivate the proposedresearch.Intellectual Merit: There is an extensive body of research in the area of multiple player games. In the caseof non-zero-sum games, the concept of mixed strategy (i.e, randomized) Nash equilibrium, despite its centralrole, has received considerable scrutiny as to how players, through repeated interactions, would ever convergeto a Nash equilibrium. Indeed there are long standing examples to the contrary. The innovative basis ofthe proposed research establishes that simple notions from feedback control can enable such convergence.Consequentially, this research opens many new possibilities into how to design distributed adaptive systemsfrom a game theoretic viewpoint.Broader Impact: The proposed research will have a broader impact on societal applications and undergraduatestudent eduction.Societal: The distributed systems concept is relevant in a multitude of domains, both engineered andsocial. These include data networks, distributed robotics, traffic networks, distributed design, power gridinfrastructure, and distributed computation, as well as economic exchange, social exchange, and politicalcoalition dynamics. The proposed work addresses a fundamental component of the mathematical models ofsuch systems and has potential implications in a variety of areas.Educational: Key activities to be undertaken are 1) Introducing 1-unit freshman courses on the conceptof distributed systems and feedback control and their applications. This is through UCLA's Fiat Luxfreshman seminar series, and 2) Developing an research test-bed based on evolvable hardware as a venue toincorporate undergraduate research-level participation through directed study electives. Of course, these activitiesare in addition to the usual education and training of graduate student researchers and disseminationof results to the research community.