Collaborative Research: SHF: Medium: Analog EDA-Inspired Methods for Efficient and Robust Neural Network Design
合作研究:SHF:媒介:用于高效、鲁棒神经网络设计的模拟 EDA 启发方法
基本信息
- 批准号:2107189
- 负责人:
- 金额:$ 40.29万
- 依托单位:
- 依托单位国家:美国
- 项目类别:Continuing Grant
- 财政年份:2021
- 资助国家:美国
- 起止时间:2021-07-15 至 2025-06-30
- 项目状态:未结题
- 来源:
- 关键词:
项目摘要
Deep neural networks have achieved great success in many engineering fields including, but not limited to, image classification, speech recognition, recommendation systems and autonomous driving. However, they suffer from two major challenges. Firstly, many neural network models are not robust, i.e. a neural network could produce inaccurate results when the input data experiences a very small amount of perturbation. Secondly, the huge cost of generating and deploying large-size neural networks limits their applications in resource-constrained platforms (e.g. mobile devices and robots). The research team notices that there is a strong mathematical connection between certain types of neural networks and analog integrated circuits. It is also known that the EDA (electronic design automation) field has 50 years of successful history of modeling, simulating, verifying and optimizing analog integrated circuits. Therefore, this project aims to substantially enrich the algorithms and theoretical understanding of neural networks by leveraging the principled approaches in the EDA community. This research will support the cross-disciplinary development of a diverse cohort of graduate and undergraduate students at the University of California at Santa Barbara, the University of California at San Diego, and the Massachusetts Institute of Technology. Several graduate-level courses on computational methods, data science and artificial intelligence are being created or enriched. The research team willis also collaborating with industry to ensure effective technology transfers.This project focuses on certain types of deep neural networks (e.g., residual neural networks, recurrent neural networks and normalizing flows) that can be described as ordinary differential equations. The technical aims of the project are divided into three thrusts. The first thrust investigates the training and compression algorithms of deep neural networks from circuit simulation and modeling perspectives. Specifically, parallel training algorithms are being developed for neural networks by borrowing the idea from parallel circuit simulation. Hardware-friendly neural-network compression algorithms are being developed from the perspective of circuit model order reduction, thereby enabling energy-efficient and real-time inference of deep neural networks. The second thrust investigates probabilistic and accurate verification techniques for the robustness of deep neural networks from circuit uncertainty quantification perspectives. Specifically, high-confidence and tighter verification bounds are being developed to describe the reachable set of a deep neural network by leveraging the hierarchical and non-Monte-Carlo techniques in analog circuit uncertainty quantification. The third thrust aims to improve the robustness of a deep neural network from the perspective of analog circuit yield optimization. In this final thrust, two ideas are being explored: (1) pre-silicon yield optimization techniques for robust neural network training, and (2) post-silicon self-healing techniques for robustness improvement of a trained neural network.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
深度神经网络在许多工程领域都取得了巨大的成功,包括但不限于图像分类、语音识别、推荐系统和自动驾驶。然而,他们面临着两大挑战。首先,许多神经网络模型是不稳健的,即当输入数据受到很小的扰动时,神经网络可能会产生不准确的结果。其次,生成和部署大型神经网络的巨大成本限制了它们在资源受限的平台(如移动设备和机器人)中的应用。研究小组注意到,某些类型的神经网络和模拟集成电路之间存在着很强的数学联系。众所周知,EDA(电子设计自动化)领域在模拟集成电路建模、仿真、验证和优化方面已有50年的成功历史。因此,本项目旨在通过利用EDA社区中的原则性方法来极大地丰富神经网络的算法和理论理解。这项研究将支持加州大学圣巴巴拉分校、加州大学圣地亚哥分校和麻省理工学院的不同研究生和本科生的跨学科发展。关于计算方法、数据科学和人工智能的几门研究生级别的课程正在创建或丰富中。研究小组Willis还与业界合作,确保有效的技术转移。该项目专注于某些类型的深度神经网络(例如,残差神经网络、递归神经网络和归一化流量),这些网络可以描述为常微分方程式。该项目的技术目标分为三个方面。第一个推力从电路仿真和建模的角度研究了深度神经网络的训练和压缩算法。具体地说,借鉴并行电路仿真的思想,正在开发神经网络的并行训练算法。从电路模型降阶的角度出发,正在开发硬件友好的神经网络压缩算法,从而实现对深层神经网络的节能和实时推理。第二个重点从电路不确定性量化的角度研究了深度神经网络健壮性的概率和精确验证技术。具体地说,通过利用模拟电路不确定性量化中的分层和非蒙特卡罗技术,正在开发高置信度和更严格的验证边界来描述深度神经网络的可达集合。第三个推力旨在从模拟电路成品率优化的角度提高深度神经网络的稳健性。在这个最后的推力中,探索了两个想法:(1)用于稳健神经网络训练的硅前成品率优化技术,以及(2)用于改善训练神经网络的稳健性的后硅自修复技术。该奖项反映了NSF的法定使命,并通过使用基金会的智力优势和更广泛的影响审查标准进行评估,被认为值得支持。
项目成果
期刊论文数量(2)
专著数量(0)
科研奖励数量(0)
会议论文数量(0)
专利数量(0)
Robust Deep Reinforcement Learning through Adversarial Loss
- DOI:
- 发表时间:2020-08
- 期刊:
- 影响因子:0
- 作者:Tuomas P. Oikarinen;Tsui-Wei Weng;L. Daniel
- 通讯作者:Tuomas P. Oikarinen;Tsui-Wei Weng;L. Daniel
On the Equivalence between Neural Network and Support Vector Machine
- DOI:
- 发表时间:2021-11
- 期刊:
- 影响因子:0
- 作者:Yilan Chen;Wei Huang-;Lam M. Nguyen;Tsui-Wei Weng
- 通讯作者:Yilan Chen;Wei Huang-;Lam M. Nguyen;Tsui-Wei Weng
{{
item.title }}
{{ item.translation_title }}
- DOI:
{{ item.doi }} - 发表时间:
{{ item.publish_year }} - 期刊:
- 影响因子:{{ item.factor }}
- 作者:
{{ item.authors }} - 通讯作者:
{{ item.author }}
数据更新时间:{{ journalArticles.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ monograph.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ sciAawards.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ conferencePapers.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ patent.updateTime }}
Tsui-Wei Weng其他文献
Tsui-Wei Weng的其他文献
{{
item.title }}
{{ item.translation_title }}
- DOI:
{{ item.doi }} - 发表时间:
{{ item.publish_year }} - 期刊:
- 影响因子:{{ item.factor }}
- 作者:
{{ item.authors }} - 通讯作者:
{{ item.author }}
{{ truncateString('Tsui-Wei Weng', 18)}}的其他基金
RI: Medium: Foundations of Recourse Verification in Machine Learning
RI:媒介:机器学习资源验证的基础
- 批准号:
2313105 - 财政年份:2023
- 资助金额:
$ 40.29万 - 项目类别:
Standard Grant
相似国自然基金
Research on Quantum Field Theory without a Lagrangian Description
- 批准号:24ZR1403900
- 批准年份:2024
- 资助金额:0.0 万元
- 项目类别:省市级项目
Cell Research
- 批准号:31224802
- 批准年份:2012
- 资助金额:24.0 万元
- 项目类别:专项基金项目
Cell Research
- 批准号:31024804
- 批准年份:2010
- 资助金额:24.0 万元
- 项目类别:专项基金项目
Cell Research (细胞研究)
- 批准号:30824808
- 批准年份:2008
- 资助金额:24.0 万元
- 项目类别:专项基金项目
Research on the Rapid Growth Mechanism of KDP Crystal
- 批准号:10774081
- 批准年份:2007
- 资助金额:45.0 万元
- 项目类别:面上项目
相似海外基金
Collaborative Research: SHF: Medium: Differentiable Hardware Synthesis
合作研究:SHF:媒介:可微分硬件合成
- 批准号:
2403134 - 财政年份:2024
- 资助金额:
$ 40.29万 - 项目类别:
Standard Grant
Collaborative Research: SHF: Small: LEGAS: Learning Evolving Graphs At Scale
协作研究:SHF:小型:LEGAS:大规模学习演化图
- 批准号:
2331302 - 财政年份:2024
- 资助金额:
$ 40.29万 - 项目类别:
Standard Grant
Collaborative Research: SHF: Small: LEGAS: Learning Evolving Graphs At Scale
协作研究:SHF:小型:LEGAS:大规模学习演化图
- 批准号:
2331301 - 财政年份:2024
- 资助金额:
$ 40.29万 - 项目类别:
Standard Grant
Collaborative Research: SHF: Small: Efficient and Scalable Privacy-Preserving Neural Network Inference based on Ciphertext-Ciphertext Fully Homomorphic Encryption
合作研究:SHF:小型:基于密文-密文全同态加密的高效、可扩展的隐私保护神经网络推理
- 批准号:
2412357 - 财政年份:2024
- 资助金额:
$ 40.29万 - 项目类别:
Standard Grant
Collaborative Research: SHF: Medium: Enabling Graphics Processing Unit Performance Simulation for Large-Scale Workloads with Lightweight Simulation Methods
合作研究:SHF:中:通过轻量级仿真方法实现大规模工作负载的图形处理单元性能仿真
- 批准号:
2402804 - 财政年份:2024
- 资助金额:
$ 40.29万 - 项目类别:
Standard Grant
Collaborative Research: SHF: Medium: Tiny Chiplets for Big AI: A Reconfigurable-On-Package System
合作研究:SHF:中:用于大人工智能的微型芯片:可重新配置的封装系统
- 批准号:
2403408 - 财政年份:2024
- 资助金额:
$ 40.29万 - 项目类别:
Standard Grant
Collaborative Research: SHF: Medium: Toward Understandability and Interpretability for Neural Language Models of Source Code
合作研究:SHF:媒介:实现源代码神经语言模型的可理解性和可解释性
- 批准号:
2423813 - 财政年份:2024
- 资助金额:
$ 40.29万 - 项目类别:
Standard Grant
Collaborative Research: SHF: Medium: Enabling GPU Performance Simulation for Large-Scale Workloads with Lightweight Simulation Methods
合作研究:SHF:中:通过轻量级仿真方法实现大规模工作负载的 GPU 性能仿真
- 批准号:
2402806 - 财政年份:2024
- 资助金额:
$ 40.29万 - 项目类别:
Standard Grant
Collaborative Research: SHF: Medium: Differentiable Hardware Synthesis
合作研究:SHF:媒介:可微分硬件合成
- 批准号:
2403135 - 财政年份:2024
- 资助金额:
$ 40.29万 - 项目类别:
Standard Grant
Collaborative Research: SHF: Medium: Tiny Chiplets for Big AI: A Reconfigurable-On-Package System
合作研究:SHF:中:用于大人工智能的微型芯片:可重新配置的封装系统
- 批准号:
2403409 - 财政年份:2024
- 资助金额:
$ 40.29万 - 项目类别:
Standard Grant