Distributed Adaptive Systems: Feedback Control with Evolutionary Games

分布式自适应系统：进化博弈的反馈控制

• 批准号: 0747783
• 负责人: Jeff Shamma
• 金额: --
• 依托单位: Georgia Tech Research Corporation
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2007
• 资助国家: 美国
• 起止时间: 2007-07-01 至 2009-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=0747783&HistoricalAwards=false
• 关键词: Distributed; Adaptive; Systems; Feedback; Control

The proposed research is to develop feedback control theoretic methods for learning and adaptation inmultiagent systems and to investigate their potential as analysis and design tools for distributed adaptivesystems. The proposed work stems from recent innovations at UCLA on the role of feedback control in thegame theoretic framework of distributed learning. This recent work has overcome long standing perceivedobstacles in game theoretic learning and opens new opportunities for distributed system design.The proposed research will emphasize underlying mathematical models of multiagent learning. Theresearch directions to be explored are:Advanced analysis of feedback control based learning in games: The introduction of feedback controlmethods in learning in games is very recent, and there are many important unresolved issues. Proposedtopics of interest include strategic advantage of feedback control based methods, analysis of systems withheterogeneous learning algorithms, dynamic network interconnections, and supervised switching for learning.Continuum action spaces: The learning in games framework currently applies to learning among a finiteset of choices. This direction involves learning with decisions over a continuum, such as dynamic resourceallocation problems. Issues include new feedback control based approaches to distributed optimization,severely limited information structures, and complementary sharing of information structures.Multiagent games with state evolution: The learning in games framework uses a static game setup.The state variable is the state of learning, but not inherent states of subsystems or the environment. Thisdirection considers learning for games with internal states, also known as Markov or stochastic games. Issuesinclude changes of time-scale for expert selection, feedback control based Q-learning, and integrator action.Application to Evolvable Hardware: Evolvable hardware is an emerging area that uses notions inspiredby biological evolution for the design of reconfigurable, self-organizing hardware. The proposed concepts onmultiagent may be viewed as an engineered evolution with its blending of feedback control and multiagentlearning. This project will use the evolvable hardware paradigm to illustrate and motivate the proposedresearch.Intellectual Merit: There is an extensive body of research in the area of multiple player games. In the caseof non-zero-sum games, the concept of mixed strategy (i.e, randomized) Nash equilibrium, despite its centralrole, has received considerable scrutiny as to how players, through repeated interactions, would ever convergeto a Nash equilibrium. Indeed there are long standing examples to the contrary. The innovative basis ofthe proposed research establishes that simple notions from feedback control can enable such convergence.Consequentially, this research opens many new possibilities into how to design distributed adaptive systemsfrom a game theoretic viewpoint.Broader Impact: The proposed research will have a broader impact on societal applications and undergraduatestudent eduction.Societal: The distributed systems concept is relevant in a multitude of domains, both engineered andsocial. These include data networks, distributed robotics, traffic networks, distributed design, power gridinfrastructure, and distributed computation, as well as economic exchange, social exchange, and politicalcoalition dynamics. The proposed work addresses a fundamental component of the mathematical models ofsuch systems and has potential implications in a variety of areas.Educational: Key activities to be undertaken are 1) Introducing 1-unit freshman courses on the conceptof distributed systems and feedback control and their applications. This is through UCLA's Fiat Luxfreshman seminar series, and 2) Developing an research test-bed based on evolvable hardware as a venue toincorporate undergraduate research-level participation through directed study electives. Of course, these activitiesare in addition to the usual education and training of graduate student researchers and disseminationof results to the research community.

建议的研究是发展反馈控制理论方法的学习和适应多智能体系统，并探讨其潜在的分析和设计工具，为分布式adaptivesystems。拟议的工作源于最近的创新在加州大学洛杉矶分校的作用，反馈控制在thegame理论框架的分布式学习。最近的这项工作已经克服了长期存在的感知障碍，在博弈论学习和分布式系统design.The建议的研究将强调多智能体学习的基础数学模型打开了新的机会。需要探索的研究方向有：基于游戏学习的反馈控制的高级分析：游戏学习中的反馈控制方法的引入是最近的事情，还有许多重要的问题没有解决。建议的主题感兴趣的反馈控制为基础的方法的战略优势，分析系统与异构学习算法，动态网络互连，和监督切换learning.Continuum行动空间：在游戏框架中的学习目前适用于学习之间的一个有限的选择。这个方向涉及学习决策的连续性，如动态资源分配问题。问题包括新的基于反馈控制的分布式优化方法，严重限制的信息结构，以及信息结构的互补共享。带状态进化的多智能体游戏：游戏中的学习框架使用静态游戏设置。状态变量是学习的状态，而不是子系统或环境的固有状态。这个方向考虑了具有内部状态的游戏的学习，也称为马尔可夫或随机游戏。Issuesincludes专家选择的时间尺度的变化，反馈控制为基础的Q-学习，和积分action.Application到进化硬件：进化硬件是一个新兴的领域，它使用的概念inspiredby生物进化的可重构，自组织硬件的设计。多智能体概念的提出可以看作是反馈控制和多智能体学习相结合的工程进化。这个项目将使用可进化的硬件范例来说明和激励所提出的研究。智力优点：在多人游戏领域有广泛的研究。在非零和博弈的情况下，混合策略（即随机化）纳什均衡的概念，尽管它的核心作用，已经收到了相当多的审查，球员如何，通过反复的相互作用，将永远收敛到纳什均衡。事实上，长期以来一直有相反的例子。该研究的创新基础是建立了反馈控制的简单概念可以实现这种收敛。因此，该研究为如何从博弈论的角度设计分布式自适应系统开辟了许多新的可能性。更广泛的影响：该研究将对社会应用和本科生教育产生更广泛的影响。分布式系统的概念在许多领域都是相关的，包括工程领域和社会领域。这些包括数据网络，分布式机器人，交通网络，分布式设计，电网基础设施和分布式计算，以及经济交换，社会交换和政治联盟动态。所提出的工作解决了一个基本组成部分的数学模型ofsuch系统，并在各种area.Educational潜在的影响：要进行的主要活动是1）介绍一个单元的新生课程的概念，分布式系统和反馈控制及其应用。这是通过加州大学洛杉矶分校的菲亚特Luxfreshman系列研讨会，和2）开发一个研究试验台的基础上进化硬件作为一个场地，通过定向学习选修纳入本科生研究水平的参与。当然，这些活动是在研究生研究人员的通常教育和培训以及向研究界传播成果之外的。