SaTC: CORE: Medium: Collaborative: Towards Trustworthy Deep Neural Network Based AI: A Systems Approach

SaTC：核心：媒介：协作：迈向基于可信深度神经网络的人工智能：一种系统方法

• 批准号: 1801495
• 负责人: Siddharth Garg
• 金额: $ 90万
• 依托单位: New York University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-08-01 至 2022-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1801495&HistoricalAwards=false
• 关键词: SaTC; CORE; Medium; Collaborative; Towards

Artificial intelligence (AI) is poised to revolutionize the world in fields ranging from technology to medicine, physics and the social sciences. Yet as AI is deployed in these domains, recent work has shown that systems may be vulnerable to different types of attacks that cause them to misbehave; for instance, attacks that cause an AI system to recognize a stop sign as a speed-limit sign. The project seeks to develop methodologies for testing, verifying and debugging AI systems, with a specific focus on deep neural network (DNN)-based AI systems, to ensure their safety and security. The intellectual merits of the proposed research are encompassed in four new software tools that will be developed: (1) DeepXplore, a tool for automated and systematic testing of DNNs that discovers erroneous behavior that might be either inadvertently or maliciously introduced; (2) BadNets, a framework that automatically generated DNNs with known and stealthy misbehaviours in order to stress-test DeepXplore; (3) SafetyNets; a low-overhead scheme for safe and verifiable execution of DNNs in the cloud; and (4) VisualBackProp; a visual debugging tool for DNNs. The synergistic use of these tools for secure deployment of an AI system for autonomous driving will be demonstrated.The project outcomes will significantly improve the security and safety of AI systems and increase their deployment in safety- and security-critical settings, resulting in broad societal impact. The results of the project will be widely disseminated via publications, talks, open access code, and competitions hosted on sites such as Kaggle and NYU's annual Cyber-Security Awareness Week (CSAW). Furthermore, students from under-represented minority groups in science, technology, engineering and mathematics (STEM) will be actively recruited and mentored to be leaders in this critical area. The code for this project will be made publicly available via github.com. Preliminary code for the tools that will be developed is already hosted on this website, including DeepXplore (https://github.com/peikexin9/deepxplore) and BadNets (https://github.com/Kooscii/BadNets/). These repositories will be linked to from a homepage that describes the entire project. The project homepage will be hosted on wp.nyu.edu/mlsecproject.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

人工智能 (AI) 有望在技术、医学、物理学和社会科学等领域彻底改变世界。然而，随着人工智能在这些领域的部署，最近的研究表明，系统可能容易受到不同类型的攻击，从而导致其行为不当；例如，导致人工智能系统将停车标志识别为限速标志的攻击。该项目旨在开发用于测试、验证和调试人工智能系统的方法，特别关注基于深度神经网络（DNN）的人工智能系统，以确保其安全性。拟议研究的智力优点包含在将开发的四个新软件工具中：（1）DeepXplore，一种用于自动和系统测试 DNN 的工具，可发现可能无意或恶意引入的错误行为； (2) BadNets，一个框架，可以自动生成具有已知和隐秘的不当行为的 DNN，以便对 DeepXplore 进行压力测试； (3) 安全网；用于在云中安全且可验证地执行 DNN 的低开销方案； (4) VisualBackProp； DNN 的可视化调试工具。将展示这些工具的协同使用，以安全部署自动驾驶人工智能系统。该项目成果将显着提高人工智能系统的安全性和安全性，并增加其在安全和安全关键环境中的部署，从而产生广泛的社会影响。该项目的结果将通过出版物、演讲、开放获取代码以及在 Kaggle 和纽约大学年度网络安全意识周 (CSAW) 等网站上举办的竞赛来广泛传播。此外，来自科学、技术、工程和数学（STEM）领域代表性不足的少数群体的学生将被积极招募和指导，成为这一关键领域的领导者。 该项目的代码将通过 github.com 公开发布。将要开发的工具的初步代码已经托管在该网站上，包括 DeepXplore (https://github.com/peikexin9/deepxplore) 和 BadNets (https://github.com/Kooscii/BadNets/)。这些存储库将从描述整个项目的主页链接到。该项目主页将托管在 wp.nyu.edu/mlsecproject 上。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Bias Busters: Robustifying DL-Based Lithographic Hotspot Detectors Against Backdooring Attacks

• DOI: 10.1109/tcad.2020.3033749
• 发表时间: 2020-04
• 期刊: IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems
• 影响因子: 2.9
• 作者: Kang Liu;Benjamin Tan;Gaurav Rajavendra Reddy;S. Garg;Y. Makris;R. Karri

Poisoning the (Data) Well in ML-Based CAD: A Case Study of Hiding Lithographic Hotspots

• DOI: 10.23919/date48585.2020.9116489
• 发表时间: 2020-03
• 期刊: 2020 Design, Automation & Test in Europe Conference & Exhibition (DATE)
• 作者: Kang Liu;Benjamin Tan;R. Karri;S. Garg

Training Data Poisoning in ML-CAD: Backdooring DL-Based Lithographic Hotspot Detectors

• DOI: 10.1109/tcad.2020.3024780
• 发表时间: 2020-09
• 期刊: IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems
• 影响因子: 2.9
• 作者: Kang Liu;Benjamin Tan;R. Karri;S. Garg

BadNets: Evaluating Backdooring Attacks on Deep Neural Networks

• DOI: 10.1109/access.2019.2909068
• 发表时间: 2019-01-01
• 期刊: IEEE ACCESS
• 影响因子: 3.9
• 作者: Gu, Tianyu;Liu, Kang;Garg, Siddharth
• 通讯作者: Garg, Siddharth

Adaptive Adversarial Videos on Roadside Billboards: Dynamically Modifying Trajectories of Autonomous Vehicles

• DOI: 10.1109/iros40897.2019.8968267
• 发表时间: 2019-11
• 期刊: 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
• 作者: Naman Patel;P. Krishnamurthy;S. Garg;F. Khorrami