Human-AI Composite Systems

研究背景和潜在影响我们如何设计能够与人类用户无缝协作的人工智能系统？这是激励本文提出的研究的中心问题。更好地回答这个问题将有助于开发能够有效地与人类协作完成任务的人工智能系统，提高使用人工智能系统的生产率并赋予人类用户权力。目的和目标通过提高人-人工智能复合系统的整体性能，一个明确的目标是开发能够进行特别协调的代理的算法，最好是在合理的设置范围内。另一个目标是开发更接近于集体人类文化并在其中更好地运行的人工智能系统。一个相关的目标是探索如何利用文化进化来创建开放的、持续的学习系统，这些系统能够世代改进和在线适应其他主体引起的分布变化。研究方法论的创新在方法论上，本研究将重点使用多智能体强化学习(MAIL)和大型语言模型(LLMS)来解决这些问题。MARL算法已被用于训练能够进行临时协调的代理，尽管在有限的环境中，并且在某些合作博弈中表现出色。同时，根据他们训练的性质，LLM有效地模拟了人类文化。此外，他们对语言的理解使他们成为专家沟通者，并提供了一种简化协调问题的机制。最后，他们的情境学习能力，以及他们规划和充当代理的能力，可以解决与上述目标相关的关键挑战。在开发符合我们目标的代理时，综合使用Marl和LLMS所提供的灵活性为新的研究提供了很大的空间。此外，在将这些方法应用于相关问题的背景下出现显著限制的情况下，我们将进行分析，以说明潜在方法如何以及为什么面临这些限制。与ESPRC的战略和研究区域保持一致本研究与ESPRC的人工智能技术主题保持一致。

Research Context & Potential ImpactHow can we design AI systems that are able to seamlessly coordinate with human users?This is the central question motivating the research proposed in this document. Better answering this question will facilitate the development of AI systems that can effectively collaborate with humans to complete tasks, boosting the productivity gains of working with an AI system and empowering human users.Aims and ObjectivesBy aiming to improve the overall performance of human-AI composite systems, one clearobjective is to develop algorithms for training agents that are capable of ad-hoc coordination, ideally across a reasonable range of settings.Another aim is to develop AI systems that more closely resemble, and better operate within,collective human culture. An associated objective is to explore how cultural evolution can beleveraged to create open-ended, continual learning systems that are capable of generationalimprovement and online adaptation to distribution shifts induced by other agents.Novelty of Research MethodologyMethodologically, this research will focus on the use of Multi-Agent Reinforcement Learning (MARL) and Large Language Models (LLMs) in addressing these problems. MARLalgorithms have been used to train agents capable of ad-hoc coordination, though in limitedsettings, and that excel in certain cooperative games. Meanwhile, by the nature of theirtraining LLMs effectively model human culture. Moreover, their understanding of languagemakes them expert communicators and provides a mechanism for simplifying coordinationproblems. Finally, their in-context learning abilities, along with their capacity for planning andacting as agents, could solve key challenges associated with the aforementioned objectives.The flexibility afforded by a comprehensive use of both MARL and LLMs in developing agentsthat meet our objectives provides a lot of room for novel research. Additionally, where notable limitations arise in the context of applying these methods to relevant problems, we willconduct analysis to illuminate how and why the underlying methods face these limitations.Alignment to ESPRC's Strategies and Research AreasThis research is aligned to ESPRC's Artificial Intelligence Technologies theme.