Modeling the real world for safe and responsible AI systems

我的目标是推进对人工智能系统的基础科学和理解，以与它们在现实世界中的负责任部署相关的方式。人工智能（AI）系统越来越多地应用于现实世界，但我们缺乏一门严谨的科学来理解或预测这些环境中的行为。我们将AI系统的问题形式化的方式与实际现实以及我们“真正想要”的AI系统存在“规范差距”。即使我们可以用非常明确的统计术语来形式化解决问题（例如，在固定数据集上进行监督学习），我们仍然不了解深度网络如何以及为什么能够泛化，为什么它们在泛化时失败，以及它们在非分布数据上的表现如何。在这些问题上取得进展对于在社会、环境和科学重要应用中负责任地使用人工智能是必要的。人工智能的工业应用在很大程度上是受利润驱动的，这种影响渗透到问题设置和方法研究最多的领域。基于偏见数据的预测算法产生的意想不到的副作用，对已经被边缘化的群体造成了实质性伤害。缺乏对泛化、隐私和鲁棒性的理解或保证，可能会成为应用人工智能解决当前气候、生态和流行病学危机的关键障碍。我的研究计划通过对人工智能系统建模和“进入”它们的内容来解决这些问题——不仅是数据，还有更广泛的学习环境，包括任务设计/规范、损失函数和正则化，以及更广泛的部署社会背景，包括隐私考虑、趋势和激励、规范和人类偏见。在相对较短的（5年）期限内，我的技术研究的计划成果是：(1)理论和实验结果，有助于更好地理解更现实环境中的学习和泛化行为，特别是对于未明确规定的问题和分布外的数据；(1)安全负责任的人工智能开发的实用方法的“工具箱”(例如安全和稳健的学习算法，代表性和一致性的指标；(3)普及新问题设置，并提供基线结果，用于AI系统解决高影响的社会和环境问题（例如，估计气候变化、污染或流行病等负面外部性的个人层面风险和影响；内容推荐中的两极分化和扭曲）。加拿大在人工智能研究人员的学术培训方面已经处于领先地位。我相信我们有能力在全球范围内引领和影响这种跨领域技术的发展，使其朝着更安全、更负责任的方向发展。我的研究计划旨在将这一努力建立在一个强有力的、可操作的科学基础上。

My objective is to advance fundamental science and understanding of AI systems in ways relevant to their responsible deployment in the real world. Artificial intelligence (AI) systems are increasingly deployed in real-world settings, but we lack a rigorous science to understand or predict behavior in these settings. There is a 'specification gap' in the way we formalize problems for AI systems vs. practical reality and what we 'really want' from AI systems.  Even when we can formalize the problem addressed in quite clear statistical terms (e.g. supervised learning on a fixed dataset), there is much we still do not understand about how and why deep networks are able to generalize as well as they do, why they fail when they do, and how they will perform on out-of-distribution data. Progress on these questions is necessary for the responsible use of AI in socially, environmentally, and scientifically important applications. Industrial applications of AI are largely profit-motivated, and this influence permeates the field in which problem settings and methods are most studied. Substantial harm is caused to already-marginalized groups by unintended side effects of predictive algorithms built on biased data. And lack of understanding or guarantees about generalization, privacy, and robustness can present critical barriers in applying AI to address current climate, ecological, and epidemiological crises.  My research programme addresses these questions by modelling AI systems and `what goes into' them - not only data, but the broader learning environment including task design/specification, loss function, and regularization, as well as the broader societal context of deployment, including privacy considerations, trends and incentives, norms, and human biases. In the relatively short (5yr) term, planned outcomes of my technical research are (1) theoretical & experimental results which help better understand learning and generalization behaviour in more realistic settings, particularly for underspecified problems and out-of-distribution data, (1) a `toolbox' of practical methods for safe and responsible development of AI (e.g. safe and robust learning algorithms, metrics for representativeness and alignment, sandboxes for testing AI systems prior to deployment, unit tests for undesirable behaviours), (3) popularization of novel problem settings, with baseline results, for AI systems addressing high-impact social and environmental problems (e.g. estimating individual-level risk and impact from negative externalities such as climate change, pollution, or epidemic disease; polarization and distortion in content recommendation). Canada is already a leader in academic training of AI researchers. I believe we are well-positioned globally to lead and influence the development of this cross-cutting technology in a more safe and responsible direction. My research programme seeks to place this effort on a strong and actionable scientific foundation.