Learning, Innovation, and Explanation in Self-Organising Multi-Agent Systems

自组织多智能体系统的学习、创新和解释

• 批准号: 2127915
• 金额: --
• 依托单位: Imperial College London
• 依托单位国家: 英国
• 项目类别: Studentship
• 财政年份: 2018
• 资助国家: 英国
• 起止时间: 2018 至 无数据
• 项目状态: 已结题
• 来源: https://gtr.ukri.org/projects?ref=studentship-2127915
• 关键词: Learning; Innovation; Explanation; Self; Organising

The initial research proposal focuses on learning, innovation, and explanation in multi-agent systems. Three levels have been identified concerning the design of multi-agent systems: the agent (individual) level, the inter-agent (social) level, and the system level. Previous work has proposed socially-inspired mechanisms to promote collective action and self-organisation at the inter-agent level and GP-based techniques for adaptation and approximate optimisation at the agent and system levels. We would like to explore learning and innovation at all three levels as ways to handle potentially dynamic changes in the environment and ensure sustainability.We note that the methods which can be applied to the agent level, such as reinforcement learning for policy optimisation, are the same sort of methods which can be applied to the system level if we consider the system entity as a control agent. So far, our research has focused on constructing a single system-wide policy by which all agents must abide. However, in a polycentric perspective, the agents should be able to formulate their own policies and plans of action and to participate in the formulation of system policies. This would require mechanisms for combining different sensory experiences and opinions, using communication and knowledge transfer protocols in a collective learning fashion in order to achieve social learning and social innovation.The research question we propose at this stage is: how should individual and collective system policies be formulated and explained so as to sustainably improve and maintain system performance and agent satisfaction, even in dynamic environments? Formulation refers both to the distributed learning algorithms and protocols which enable the agents to construct individual and collective policies and to the way the policies are represented. Explainability refers to the intelligibility of both the learning algorithms which result in the creation of new policies and the policies themselves. Understanding why a policy has been selected and why it is better than other candidate policies which could also have been selected is important for system designers and users to be able to trust its recommendations and gather valuable knowledge regarding the problem domain. Some learning models are inherently intelligible while others are not and it is possible that some unintelligible models can be made understandable, explainable, and interpretable. Intelligible models could shed a new light upon problem domains which are intrinsically hard for humans to understand and lead to the adoption of new and better strategies for searching the theoretically infinite space of sets of rules upon which policies are built. At this stage we have identified the following high-level tasks:- Review previous work on the design of multi-agent systems, focusing on the three levels identified above: the agent, the social, and the system levels- Identify knowledge gaps in previous work, namely regarding polycentric construction of policies and explainability- Write survey paper based on both the literature review and the conclusions about knowledge gaps- Identify possible applications and systems to use as running examples for policy construction and selection- Iteratively propose solutions to three levels of policy construction: single system policy (current work); multiple agent policies; multiple agent policies and a collectively formulated system policyThis project aims at proposing a new paradigm for the design of collective adaptive systems in which system policies are not handcrafted and designers do not hardcode immutable rule sets, but rather the result of automatic construction with the explicit goal of improving system performance and agent satisfaction.EPSRC Research Areas: Control Engineering, Engineering design, Artificial intelligence technologies, Software engineering

最初的研究建议集中在多智能体系统中的学习，创新和解释。关于多智能体系统的设计，已经确定了三个层次：智能体（个体）层次、智能体间（社会）层次和系统层次。以前的工作提出了社会启发的机制，以促进集体行动和自组织在代理间的水平和基于GP的技术，在代理和系统的水平，适应和近似优化。我们希望在这三个层次上探索学习和创新，以应对环境中潜在的动态变化并确保可持续性。我们注意到，如果我们将系统实体视为控制代理，则可以应用于代理层的方法（例如用于政策优化的强化学习）与可以应用于系统层的方法是相同的。到目前为止，我们的研究集中在构建一个单一的系统范围内的政策，所有代理人都必须遵守。然而，从多中心的角度来看，代理人应该能够制定自己的政策和行动计划，并参与制定系统政策。这就需要有一种机制来结合不同的感官体验和意见，以集体学习的方式使用沟通和知识转移协议，以实现社会学习和社会创新。我们在这一阶段提出的研究问题是：应如何制定和解释个人和集体系统政策，以持续改善和保持系统绩效和代理人满意度，即使在动态环境中？制定是指分布式学习算法和协议，使代理构建个人和集体的政策和政策的方式表示。可解释性是指导致创建新策略的学习算法和策略本身的可理解性。理解为什么选择了一个策略，为什么它比其他可能被选择的候选策略更好，对于系统设计者和用户能够信任它的建议并收集关于问题域的有价值的知识是很重要的。一些学习模型本质上是可理解的，而另一些则不是，并且一些不可理解的模型可以变得可理解，可解释和可解释。可理解的模型可以为人类理解本质上难以理解的问题领域提供新的启示，并导致采用新的更好的策略来搜索理论上无限的规则空间，这些规则是建立在政策基础上的。在这个阶段，我们已经确定了以下高层次的任务：-审查以前的工作设计的多代理系统，重点是上述三个层次：代理人，社会和系统层面-确定以前工作中的知识差距，即关于政策的多中心结构和可解释性-根据文献综述和关于知识差距的结论撰写调查报告-确定可能的应用程序和系统，用作策略构建和选择的运行示例-迭代地提出三个级别的策略构建解决方案：单个系统策略（当前工作）;多代理策略;多代理策略和集体制定的系统策略本项目旨在提出一个新的范式，设计集体适应系统其中系统策略不是手工制作的，设计人员也不硬编码不可变的规则集，而是自动构建的结果，其明确目标是提高系统性能和代理满意度。EPSRC研究领域：控制工程，工程设计，人工智能技术，软件工程