Human-AI Composite Systems

研究背景和潜在影响我们如何设计能够与人类用户无缝协调的人工智能系统？这是激励本文件中提出的研究的中心问题。更好地回答这个问题将促进人工智能系统的发展，这些系统可以有效地与人类合作完成任务，提高与人工智能系统合作的生产力，并赋予人类用户权力。目标和目的通过旨在提高人类-人工智能复合系统的整体性能，一个明确的目标是开发能够进行ad-hoc协调的训练代理的算法，另一个目标是开发更接近人类集体文化并能更好地在其中运行的人工智能系统。一个相关的目标是探索如何文化进化可以beleveraged创造开放式的，持续的学习系统，能够generationalimprovement和在线适应其他agents.新奇的研究方法论的分布变化，本研究将集中在使用多智能体强化学习（MARL）和大型语言模型（LLM）在解决这些问题。MARL算法已被用来训练代理能够ad-hoc协调，虽然在有限的设置，并在某些合作游戏中表现出色。同时，由于其培训的性质，LLM有效地塑造了人类文化。此外，他们对语言的理解使他们成为专家沟通者，并提供了一种简化协调问题的机制。最后，他们的上下文学习能力，沿着他们的规划和代理能力，可以解决与上述目标相关的关键挑战。在开发符合我们目标的代理时，综合使用MARL和LLM所提供的灵活性为新的研究提供了很大的空间。此外，在将这些方法应用于相关问题的背景下出现显着的局限性，我们将进行分析，以阐明如何以及为什么基本的方法面临这些局限性。对齐ESPRC的战略和研究领域这项研究是对齐ESPRC的人工智能技术的主题。

Research Context & Potential ImpactHow can we design AI systems that are able to seamlessly coordinate with human users?This is the central question motivating the research proposed in this document. Better answering this question will facilitate the development of AI systems that can effectively collaborate with humans to complete tasks, boosting the productivity gains of working with an AI system and empowering human users.Aims and ObjectivesBy aiming to improve the overall performance of human-AI composite systems, one clearobjective is to develop algorithms for training agents that are capable of ad-hoc coordination, ideally across a reasonable range of settings.Another aim is to develop AI systems that more closely resemble, and better operate within,collective human culture. An associated objective is to explore how cultural evolution can beleveraged to create open-ended, continual learning systems that are capable of generationalimprovement and online adaptation to distribution shifts induced by other agents.Novelty of Research MethodologyMethodologically, this research will focus on the use of Multi-Agent Reinforcement Learning (MARL) and Large Language Models (LLMs) in addressing these problems. MARLalgorithms have been used to train agents capable of ad-hoc coordination, though in limitedsettings, and that excel in certain cooperative games. Meanwhile, by the nature of theirtraining LLMs effectively model human culture. Moreover, their understanding of languagemakes them expert communicators and provides a mechanism for simplifying coordinationproblems. Finally, their in-context learning abilities, along with their capacity for planning andacting as agents, could solve key challenges associated with the aforementioned objectives.The flexibility afforded by a comprehensive use of both MARL and LLMs in developing agentsthat meet our objectives provides a lot of room for novel research. Additionally, where notable limitations arise in the context of applying these methods to relevant problems, we willconduct analysis to illuminate how and why the underlying methods face these limitations.Alignment to ESPRC's Strategies and Research AreasThis research is aligned to ESPRC's Artificial Intelligence Technologies theme.