RI: Small: Designing Preferences, Beliefs, and Identities for Artificial Intelligence

从历史上看，人工智能研究人员主要专注于开发能够很好地实现预先指定的目标的技术，这些技术提供了一种衡量这些技术工作情况的有用指标。在技术还没有准备好走出实验室并走向世界的情况下，这种方法是完全明智的。然而，随着人工智能现在世界上被广泛部署，需要在设计人工智能系统的目标的方法上投入更多的思考。这是因为我们的目标不再仅仅是评估我们的其他技术是否能够很好地追求给定的目标，而是真正让它们在世界上做好事。除了人工智能系统的目标外，我们还必须指定系统的一个部分在哪里结束，另一个部分在哪里开始，以及它如何对世界进行建模。一般来说，简单地将系统交给客户(广义上)，然后客户必须以某种方式填充这些空白，这是不可能或不可取的。人工智能研究人员需要参与这一过程，因为他们了解系统是如何工作的，并能够为这些决策提供算法支持。但这些过程缺乏严格的计算框架，这正是本研究旨在提供的。具体地说，人工智能、经济学和其他相关领域的镜像框架的现有研究都是建立在人工智能系统作为主体的概念之上的。它通常基于这样的前提，即随着时间的推移，每个这样的代理人都有一个明确的身份，对事情可能进行的不同方式有明确的偏好，以及对世界的现状和它将如何随着时间的发展有明确的信念。然后，典型的研究关注在所有这些方面都已经被指定的假设下的算法设计(除了仍然需要对环境做一些学习的常见例外)。然而，当我们设计真实的人工智能系统时，我们实际上需要指定系统中一个代理和另一个代理之间的边界在哪里，这些代理旨在最大化哪些目标函数，甚至在某种程度上，他们使用什么信念形成过程。这项研究的前提是，随着人工智能在世界上的广泛部署，我们需要有充分基础的理论，以及如何设计偏好、身份和信念的方法和算法。以负责任的方式做到这一点，将需要开发和严格评估新技术。该项目将从决策理论、博弈论、社会选择理论、机制设计理论以及这些领域的算法和计算方面的严格基础上解决这些问题。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Historically, AI researchers have primarily focused on developing techniques that work well for pre-specified objectives that provide a useful measure of how well the techniques are working. This approach is perfectly sensible in a situation where the techniques are not yet ready to make their way out of the lab and into the world. However, as AI is now being broadly deployed in the world, more thought needs to be put into the methodologies for designing the objectives of AI systems. This is because our aim is no longer just to evaluate whether our other techniques are able to pursue a given objective well, but rather to actually have them do good in the world. Besides the AI system's objectives, we must also specify where one part of the system ends and another begins, as well as how it models the world. Generally, it is not possible or desirable to simply hand off the system to a customer (in the broad sense of the word) who then must somehow fill in these blanks. AI researchers need to be involved in this process because they understand how the system works and are able to provide algorithmic support for these decisions. But rigorous computational frameworks for these processes are lacking, and they are what this research aims to provide.Specifically, existing research in artificial intelligence, mirroring frameworks in economics and other related fields, is built on a conception of AI systems as agents. It generally proceeds from the premise that each such agent has a well-defined identity over time, well-defined preferences over the different ways in which things may proceed, and well-defined beliefs about the world as it is and how it will develop over time. Typical research then concerns the design of algorithms under the assumption that all these aspects have already been specified (with the common exception of still needing to do some learning about the environment). However, as we design real AI systems, we in fact need to specify where the boundaries between one agent and another in the system lie, what objective functions these agents aim to maximize, and to some extent even what belief formation processes they use. The premise of this research is that as AI is being broadly deployed in the world, we need well-founded theories of, and methodologies and algorithms for, how to design preferences, identities, and beliefs. Doing so in a responsible fashion will require the development and rigorous evaluation of new techniques. The project will address these questions from a rigorous foundation in decision theory, game theory, social choice theory, mechanism design theory, and the algorithmic and computational aspects of these fields.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.