Modeling the real world for safe and responsible AI systems

我的目标是以与现实世界中负责任的部署相关的方式促进基础科学和对人工智能系统的理解。人工智能(AI)系统越来越多地部署在现实世界的环境中，但我们缺乏一门严谨的科学来理解或预测这些环境中的行为。在我们将人工智能系统的问题形式化的方式与实际情况以及我们从人工智能系统中“真正想要”什么方面存在“规范差距”。即使我们可以将以非常清楚的统计术语(例如，固定数据集上的监督学习)解决的问题形式化，对于深层网络如何以及为什么能够像它们那样进行泛化，为什么它们在这样做时失败，以及它们将如何对非分布数据执行性能，我们仍然有很多不理解的地方。在这些问题上取得进展对于负责任地使用人工智能在社会、环境和科学上的重要应用是必要的。人工智能的工业应用在很大程度上是以利润为动机的，这种影响渗透到问题设置和方法研究最多的领域。建立在有偏见的数据基础上的预测算法的意外副作用给已经被边缘化的群体造成了实质性的伤害。缺乏对泛化、隐私和稳健性的理解或保证，可能会在应用人工智能来应对当前的气候、生态和流行病危机方面构成关键障碍。我的研究计划通过对人工智能系统进行建模来解决这些问题-不仅是数据，而且是更广泛的学习环境，包括任务设计/规范、损失函数和正规化，以及部署的更广泛的社会背景，包括隐私考虑、趋势和激励、规范和人类偏见。在相对较短的时间内(5年)，我的技术研究的计划结果是(1)理论和实验结果，有助于在更现实的环境中更好地理解学习和泛化行为，(1)安全和负责任地开发人工智能的实用方法的“工具箱”(例如，安全和稳健的学习算法、代表性和一致性的衡量标准、在部署人工智能系统之前测试人工智能系统的沙盒、不良行为的单元测试)、(3)普及新的问题设置，并提供基线结果，用于人工智能系统解决高影响的社会和环境问题(例如，估计个人水平的风险和气候变化、污染或流行病等负面外部性的影响)；内容推荐中的极化和失真)。加拿大在人工智能研究人员的学术培训方面已经处于领先地位。我相信，我们在全球处于有利地位，能够领导和影响这种交叉技术的发展，朝着更加安全和负责任的方向发展。我的研究计划寻求将这一努力建立在强大和可行的科学基础上。

My objective is to advance fundamental science and understanding of AI systems in ways relevant to their responsible deployment in the real world. Artificial intelligence (AI) systems are increasingly deployed in real-world settings, but we lack a rigorous science to understand or predict behavior in these settings. There is a 'specification gap' in the way we formalize problems for AI systems vs. practical reality and what we 'really want' from AI systems.  Even when we can formalize the problem addressed in quite clear statistical terms (e.g. supervised learning on a fixed dataset), there is much we still do not understand about how and why deep networks are able to generalize as well as they do, why they fail when they do, and how they will perform on out-of-distribution data. Progress on these questions is necessary for the responsible use of AI in socially, environmentally, and scientifically important applications. Industrial applications of AI are largely profit-motivated, and this influence permeates the field in which problem settings and methods are most studied. Substantial harm is caused to already-marginalized groups by unintended side effects of predictive algorithms built on biased data. And lack of understanding or guarantees about generalization, privacy, and robustness can present critical barriers in applying AI to address current climate, ecological, and epidemiological crises.  My research programme addresses these questions by modelling AI systems and `what goes into' them - not only data, but the broader learning environment including task design/specification, loss function, and regularization, as well as the broader societal context of deployment, including privacy considerations, trends and incentives, norms, and human biases. In the relatively short (5yr) term, planned outcomes of my technical research are (1) theoretical & experimental results which help better understand learning and generalization behaviour in more realistic settings, particularly for underspecified problems and out-of-distribution data, (1) a `toolbox' of practical methods for safe and responsible development of AI (e.g. safe and robust learning algorithms, metrics for representativeness and alignment, sandboxes for testing AI systems prior to deployment, unit tests for undesirable behaviours), (3) popularization of novel problem settings, with baseline results, for AI systems addressing high-impact social and environmental problems (e.g. estimating individual-level risk and impact from negative externalities such as climate change, pollution, or epidemic disease; polarization and distortion in content recommendation). Canada is already a leader in academic training of AI researchers. I believe we are well-positioned globally to lead and influence the development of this cross-cutting technology in a more safe and responsible direction. My research programme seeks to place this effort on a strong and actionable scientific foundation.