Modeling the real world for safe and responsible AI systems

为安全且负责任的人工智能系统对现实世界进行建模

• 批准号: RGPIN-2022-05079
• 负责人: Maharaj, Tegan
• 金额: $ 1.82万
• 依托单位: University of Toronto
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2022
• 资助国家: 加拿大
• 起止时间: 2022-01-01 至 2023-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=757266
• 关键词: Modeling; real; world; safe; responsible

My objective is to advance fundamental science and understanding of AI systems in ways relevant to their responsible deployment in the real world. Artificial intelligence (AI) systems are increasingly deployed in real-world settings, but we lack a rigorous science to understand or predict behavior in these settings. There is a 'specification gap' in the way we formalize problems for AI systems vs. practical reality and what we 'really want' from AI systems.  Even when we can formalize the problem addressed in quite clear statistical terms (e.g. supervised learning on a fixed dataset), there is much we still do not understand about how and why deep networks are able to generalize as well as they do, why they fail when they do, and how they will perform on out-of-distribution data. Progress on these questions is necessary for the responsible use of AI in socially, environmentally, and scientifically important applications. Industrial applications of AI are largely profit-motivated, and this influence permeates the field in which problem settings and methods are most studied. Substantial harm is caused to already-marginalized groups by unintended side effects of predictive algorithms built on biased data. And lack of understanding or guarantees about generalization, privacy, and robustness can present critical barriers in applying AI to address current climate, ecological, and epidemiological crises.  My research programme addresses these questions by modelling AI systems and `what goes into' them - not only data, but the broader learning environment including task design/specification, loss function, and regularization, as well as the broader societal context of deployment, including privacy considerations, trends and incentives, norms, and human biases. In the relatively short (5yr) term, planned outcomes of my technical research are (1) theoretical & experimental results which help better understand learning and generalization behaviour in more realistic settings, particularly for underspecified problems and out-of-distribution data, (1) a `toolbox' of practical methods for safe and responsible development of AI (e.g. safe and robust learning algorithms, metrics for representativeness and alignment, sandboxes for testing AI systems prior to deployment, unit tests for undesirable behaviours), (3) popularization of novel problem settings, with baseline results, for AI systems addressing high-impact social and environmental problems (e.g. estimating individual-level risk and impact from negative externalities such as climate change, pollution, or epidemic disease; polarization and distortion in content recommendation). Canada is already a leader in academic training of AI researchers. I believe we are well-positioned globally to lead and influence the development of this cross-cutting technology in a more safe and responsible direction. My research programme seeks to place this effort on a strong and actionable scientific foundation.

我的目标是推进基础科学和对人工智能系统的理解，使其在真实的世界中负责任地部署。人工智能（AI）系统越来越多地部署在现实世界中，但我们缺乏严格的科学来理解或预测这些环境中的行为。在我们形式化人工智能系统问题的方式与实际现实以及我们“真正想要”人工智能系统的方式之间存在“规范差距”。即使我们可以用非常明确的统计术语形式化解决问题（例如，在固定数据集上的监督学习），我们仍然不了解深度网络如何以及为什么能够像它们那样泛化，为什么它们会失败，以及它们将如何处理分发外的数据。在这些问题上取得进展对于在社会，环境和科学重要应用中负责任地使用AI是必要的。人工智能的工业应用在很大程度上是以利润为动机的，这种影响渗透到问题设置和方法研究最多的领域。建立在有偏见的数据上的预测算法的意外副作用对已经被边缘化的群体造成了实质性的伤害。缺乏对泛化、隐私和鲁棒性的理解或保证，可能会成为应用人工智能解决当前气候、生态和流行病危机的关键障碍。我的研究项目通过对人工智能系统和“什么进入”它们的建模来解决这些问题-不仅是数据，还有更广泛的学习环境，包括任务设计/规范、损失函数和正则化，以及部署的更广泛的社会背景，包括隐私考虑，趋势和激励措施，规范和人类偏见。在相对较短的（5年）期限内，我的技术研究的计划成果是（1）理论和实验结果，有助于更好地理解更现实环境中的学习和泛化行为，特别是对于未指定的问题和未分发的数据，（1）安全和负责任地开发人工智能的实用方法的“工具箱”（例如，安全和强大的学习算法，代表性和一致性的指标，部署前测试AI系统的沙箱，针对不良行为的单元测试），（3）普及新的问题设置，并提供基线结果，用于人工智能系统解决高影响力的社会和环境问题（例如，估计气候变化、污染或流行病等负面外部性的个人风险和影响;内容推荐中的两极分化和扭曲）。加拿大已经是人工智能研究人员学术培训的领导者。我相信，我们在全球范围内处于有利地位，能够引领和影响这种跨领域技术的发展，使其朝着更安全和负责任的方向发展。我的研究计划旨在将这一努力建立在一个强大的和可操作的科学基础上。