Safety, robustness, and economic properties of machine learning

该项目属于EPSRC信息和通信技术（ICT）研究领域。它也与数字经济和工程领域的接口。在Yarin Gal教授和Allan Dafoe教授的指导下，我将从经验和理论上研究机器学习方法的安全性和鲁棒性，以及该技术的经济特性。随着机器学习算法变得越来越强大，部署它们的安全关键环境的数量也在增加。例如，股票交易算法必须符合某些规则，这些规则不容易作为训练信号硬编码。因此，强化学习算法可以找到执行获胜但非法的策略的方法，绕过任何旨在阻止此类行为的特设目标。在高层次上，我们对鲁棒（深度）学习方法的基础研究感兴趣，这些方法可以在没有昂贵监督的情况下代表人类采取行动。在Atari等控制良好的领域中，我们可以看到当前的ML技术可以扩展到很远的地方，这表明它们可以在更现实的领域中扩展到很远的地方。我们对变化分布的鲁棒性特别感兴趣，这是一个很难理解的问题，并且与人类偏好保持一致，这是迈向安全和有用的人工智能系统的关键一步。一个不健壮的系统，部署在一个新的分布上，可能会严重误解它的情况，因此可能会自信地做出有害的决定。一个不完全对齐的系统可能会被敌对输入所利用，或者可能会优化其既定目标和预期目标之间的相关性。我的项目将涉及贝叶斯神经网络在机器学习中的安全问题的应用。盖尔教授极大地推动了这些网络的发展，它们可以帮助检测和适应分布变化（鲁棒性），或者主动征求有关人类偏好的数据（一致性）。此外，我们将研究机器学习作为一种技术的经济特性。在经济学中，一项技术是由它的生产函数来定义的，它将有价值的投入与产出联系起来。在机器学习中，这些输入包括数据、计算和人工。输出取决于具体的任务。它们可能是诸如测试错误之类的度量，也可能是使用学习模型（例如应用程序或股票交易算法）的结果产品的价值。估计这样的生产函数是经济学中最古老的经验问题之一，但还没有明确地用于机器学习。该方法涉及微观经济模型，而机器学习研究人员迄今尚未使用这种模型。

This project falls within the EPSRC Information and communication technologies (ICT) research area. It also interfaces with the Digital Economy and Engineering areas. Under the supervision of Prof Yarin Gal and Prof Allan Dafoe, I will empirically and theoretically study safety and robustness of machine learning methods as well as the economic properties of the technology. As machine learning algorithms are becoming more capable, the number of safety-critical environments in which they will be deployed increases. For example, a stock-trading algorithm has to conform to certain rules which are not easy to hard-code as a training signal. Therefore, a reinforcement learning algorithm could find ways to execute winning but illegal strategies that circumvent any ad-hoc objective that is meant to discourage such behavior. On a high level, we are interested in fundamental research on robust (deep) learning methods that can be used to act on behalf of humans without costly supervision. Within well-controlled domains such as Atari, we can see that current ML techniques can scale far, indicating that they could scale quite far in more realistic domains. We are particularly interested in robustness to changing distributions, a poorly understood problem, and in alignment with human preferences, a critical step towards safe and useful AI systems. A non-robust system, deployed on a novel distribution, may badly misunderstand its situation, and thus may make harmful decisions confidently. An imperfectly aligned system may be exploited with adversarial inputs or may optimize away the correlation between its stated and intended objective. My project will involve the application or Bayesian neural networks to such safety problems in machine learning. Prof Gal has significantly advanced the development of these networks and they can help to detect and adapt to distribution shift (robustness) or actively solicit data about human preferences (alignment). Furthermore, we will research economic properties of machine learning as a technology. In economics, a technology is defined by its production function which relates valuable inputs to outputs. In machine learning, those inputs include data, compute, and labor. The outputs depend on the specific task. They might be measures such as test errors, but also the value of the resulting product that uses a learned model (e.g. an application or a stock trading algorithm). Estimating such a production function is one of the oldest empirical problems in economics, but has not been explicitly done for machine learning. The methodology involves microeconomic modeling which machine learning researchers have so far not used.