SaTC: CORE: Medium: Collaborative: Towards Trustworthy Deep Neural Network Based AI: A Systems Approach

SaTC：核心：媒介：协作：迈向基于可信深度神经网络的人工智能：一种系统方法

• 批准号: 1801426
• 负责人: Suman Jana
• 金额: $ 30万
• 依托单位: Columbia University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-08-01 至 2022-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1801426&HistoricalAwards=false
• 关键词: SaTC; CORE; Medium; Collaborative; Towards

Artificial intelligence (AI) is poised to revolutionize the world in fields ranging from technology to medicine, physics and the social sciences. Yet as AI is deployed in these domains, recent work has shown that systems may be vulnerable to different types of attacks that cause them to misbehave; for instance, attacks that cause an AI system to recognize a stop sign as a speed-limit sign. The project seeks to develop methodologies for testing, verifying and debugging AI systems, with a specific focus on deep neural network (DNN)-based AI systems, to ensure their safety and security. The intellectual merits of the proposed research are encompassed in four new software tools that will be developed: (1) DeepXplore, a tool for automated and systematic testing of DNNs that discovers erroneous behavior that might be either inadvertently or maliciously introduced; (2) BadNets, a framework that automatically generated DNNs with known and stealthy misbehaviours in order to stress-test DeepXplore; (3) SafetyNets; a low-overhead scheme for safe and verifiable execution of DNNs in the cloud; and (4) VisualBackProp; a visual debugging tool for DNNs. The synergistic use of these tools for secure deployment of an AI system for autonomous driving will be demonstrated.The project outcomes will significantly improve the security and safety of AI systems and increase their deployment in safety- and security-critical settings, resulting in broad societal impact. The results of the project will be widely disseminated via publications, talks, open access code, and competitions hosted on sites such as Kaggle and NYU's annual Cyber-Security Awareness Week (CSAW). Furthermore, students from under-represented minority groups in science, technology, engineering and mathematics (STEM) will be actively recruited and mentored to be leaders in this critical area. The code for this project will be made publicly available via github.com. Preliminary code for the tools that will be developed is already hosted on this website, including DeepXplore (https://github.com/peikexin9/deepxplore) and BadNets (https://github.com/Kooscii/BadNets/). These repositories will be linked to from a homepage that describes the entire project. The project homepage will be hosted on wp.nyu.edu/mlsecproject.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

人工智能(AI)正准备在从技术到医学、物理和社会科学的各个领域给世界带来革命性的变化。然而，随着人工智能部署在这些领域，最近的研究表明，系统可能容易受到不同类型的攻击，这些攻击会导致它们行为不当；例如，导致人工智能系统将停车标志识别为限速标志的攻击。该项目寻求开发测试、验证和调试人工智能系统的方法，重点是基于深度神经网络(DNN)的人工智能系统，以确保其安全和保障。拟议研究的智力优势体现在将开发的四个新软件工具中：(1)DeepXplore，一个用于自动和系统地测试DNN的工具，它可以发现可能无意或恶意引入的错误行为；(2)BadNets，一个框架，它自动生成具有已知和隐蔽行为的DNN，以便对DeepXplore进行压力测试；(3)SafetyNets，一个用于在云中安全和可验证地执行DNN的低开销方案；以及(4)VisualBackProp，一个DNN的可视化调试工具。将展示这些工具在自动驾驶人工智能系统安全部署方面的协同使用。项目成果将显著提高人工智能系统的安全性和安全性，并增加其在安全和安保关键环境中的部署，产生广泛的社会影响。该项目的结果将通过出版物、讲座、开放获取代码和在Kaggle和纽约大学年度网络安全意识周(CSAW)等网站上举办的比赛来广泛传播。此外，将积极招募来自科学、技术、工程和数学(STEM)中代表性不足的少数群体的学生，并指导他们成为这一关键领域的领导者。该项目的代码将通过githorb.com公开提供。将开发的工具的初步代码已经托管在该网站上，包括DeepXplore(https://github.com/peikexin9/deepxplore)和BadNets(https://github.com/Kooscii/BadNets/).这些储存库将从描述整个项目的主页链接到。该项目的主页将托管在wp.nyu.edu/mlsecproject上。这一奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。