Computational and Mathematical Studies of Complexity Reduction Methods for Deep Neural Networks and Applications

深度神经网络复杂度降低方法的计算和数学研究及应用

基本信息

  • 批准号:
    1854434
  • 负责人:
  • 金额:
    $ 30万
  • 依托单位:
  • 依托单位国家:
    美国
  • 项目类别:
    Standard Grant
  • 财政年份:
    2019
  • 资助国家:
    美国
  • 起止时间:
    2019-07-15 至 2023-06-30
  • 项目状态:
    已结题

项目摘要

Deep neural networks (DNN) have become the state-of-the-art computing technology driving the recent advances in artificial intelligence, surpassing human performance on image and speech recognition tasks, and in playing complex games such as Go. However, deep networks typically consume billions of flops in computation and gigabytes of storage for model and data, rendering their deployment a challenge on mobile and energy limited platforms such as cellular phones and battery powered cars. The project aims to develop theory and algorithms for complexity reduction methods so as to maintain DNN's performance on low computational budget, achieving speed up and saving memory space. The project also studies light weight deep networks through automated architecture search and selection to reduce complexity at a higher design level. A broad range of applications include mobile computer vision, disease diagnosis and detection, face verification, as well as monitor and rescue missions by the drone. The project will actively involve graduate students and enrich their career development through both education and research activities. The approaches to be studied include (1) training of deep networks with low-precision weights and activation functions (so called quantization), (2) hand-crafted and automated lightweight deep networks, their training via variable splitting and their quantization. The training of quantized networks concerns with minimizing high dimensional discontinuous non-convex objectives under discrete constraints, for which novel coarse gradients and an accelerated technique (so called blending) will be analyzed to guide the descent and reach convergence. A differentiable treatment of discrete constraints, and of non-smooth and combinatorial structures will be fully developed. The methodologies and resulting algorithms from the project will contribute to information technology, optimization of civil infrastructure, smart and efficient mobile computing.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
深度神经网络(DNN)已经成为最先进的计算技术,推动了人工智能的最新进展,在图像和语音识别任务以及围棋等复杂游戏中超越了人类的表现。然而,深度网络通常在计算中消耗数十亿次失败以及用于模型和数据的千兆字节的存储,使得它们的部署在移动的和能量有限的平台(诸如蜂窝电话和电池供电的汽车)上成为挑战。该项目旨在开发降低复杂性方法的理论和算法,以便在低计算预算下保持DNN的性能,实现加速和节省内存空间。 该项目还通过自动架构搜索和选择来研究轻量级深度网络,以降低更高设计级别的复杂性。广泛的应用包括移动的计算机视觉、疾病诊断和检测、人脸验证以及无人机的监控和救援任务。该项目将通过教育和研究活动积极吸引研究生参与,丰富他们的职业发展。要研究的方法包括(1)使用低精度权重和激活函数(所谓的量化)训练深度网络,(2)手工制作和自动化的轻量级深度网络,通过变量拆分和量化进行训练。量化网络的训练涉及在离散约束下最小化高维不连续非凸目标,为此将分析新的粗梯度和加速技术(所谓的混合)以指导下降并达到收敛。一个可微处理离散约束,非光滑和组合结构将得到充分发展。该项目的方法和算法将有助于信息技术、民用基础设施的优化、智能和高效的移动的计算。该奖项反映了NSF的法定使命,并通过使用基金会的知识价值和更广泛的影响审查标准进行评估,被认为值得支持。

项目成果

期刊论文数量(23)
专著数量(0)
科研奖励数量(0)
会议论文数量(0)
专利数量(0)
Two-Grid Based Adaptive Proper Orthogonal Decomposition Method for Time Dependent Partial Differential Equations
基于二网格的时变偏微分方程自适应原正交分解方法
  • DOI:
    10.1007/s10915-020-01288-9
  • 发表时间:
    2020-08
  • 期刊:
  • 影响因子:
    2.5
  • 作者:
    Dai Xiaoying;Kuang Xiong;Xin Jack;Zhou Aihui
  • 通讯作者:
    Zhou Aihui
Searching Intrinsic Dimensions of Vision Transformers
寻找视觉变压器的内在维度
A Channel-Pruned and Weight-Binarized Convolutional Neural Network for Keyword Spotting
  • DOI:
    10.1007/978-3-030-38364-0_22
  • 发表时间:
    2019-09
  • 期刊:
  • 影响因子:
    0
  • 作者:
    J. Lyu;S. Sheen
  • 通讯作者:
    J. Lyu;S. Sheen
Global convergence and geometric characterization of slow to fast weight evolution in neural network training for classifying linearly non-separable data
用于分类线性不可分离数据的神经网络训练中从慢到快权重演化的全局收敛和几何表征
  • DOI:
    10.3934/ipi.2020077
  • 发表时间:
    2021
  • 期刊:
  • 影响因子:
    1.3
  • 作者:
    Long, Ziang;Yin, Penghang;Xin, Jack
  • 通讯作者:
    Xin, Jack
Synchronized Front Propagation and Delayed Flame Quenching in Strain G-Equation and Time-Periodic Cellular Flows
应变 G 方程和时间周期细胞流中的同步前沿传播和延迟火焰淬火
{{ item.title }}
{{ item.translation_title }}
  • DOI:
    {{ item.doi }}
  • 发表时间:
    {{ item.publish_year }}
  • 期刊:
  • 影响因子:
    {{ item.factor }}
  • 作者:
    {{ item.authors }}
  • 通讯作者:
    {{ item.author }}

数据更新时间:{{ journalArticles.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ monograph.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ sciAawards.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ conferencePapers.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ patent.updateTime }}

Jack Xin其他文献

A structure-preserving scheme for computing effective diffusivity and anomalous diffusion phenomena of random flows
计算随机流的有效扩散率和反常扩散现象的结构保持方案
  • DOI:
    10.48550/arxiv.2405.19003
  • 发表时间:
    2024
  • 期刊:
  • 影响因子:
    0
  • 作者:
    Tan Zhang;Zhongjian Wang;Jack Xin;Zhiwen Zhang
  • 通讯作者:
    Zhiwen Zhang
Finite Element Computation of KPP Front Speeds in Cellular and Cat#39;s Eye Flows
Cellular 和 Cat 中 KPP 前沿速度的有限元计算
Learning Sparse Neural Networks via \ell _0 and T \ell _1 by a Relaxed Variable Splitting Method with Application to Multi-scale Curve Classification
通过松弛变量分裂方法通过 ell _0 和 T ell _1 学习稀疏神经网络并应用于多尺度曲线分类
Design projects motivated and informed by the needs of severely disabled autistic children
设计项目以严重残疾自闭症儿童的需求为动力和信息
Three $$l_1$$ Based Nonconvex Methods in Constructing Sparse Mean Reverting Portfolios
  • DOI:
    10.1007/s10915-017-0578-5
  • 发表时间:
    2017-10-20
  • 期刊:
  • 影响因子:
    3.300
  • 作者:
    Xiaolong Long;Knut Solna;Jack Xin
  • 通讯作者:
    Jack Xin

Jack Xin的其他文献

{{ item.title }}
{{ item.translation_title }}
  • DOI:
    {{ item.doi }}
  • 发表时间:
    {{ item.publish_year }}
  • 期刊:
  • 影响因子:
    {{ item.factor }}
  • 作者:
    {{ item.authors }}
  • 通讯作者:
    {{ item.author }}

{{ truncateString('Jack Xin', 18)}}的其他基金

Deep Particle Algorithms and Advection-Reaction-Diffusion Transport Problems
深层粒子算法与平流反应扩散传输问题
  • 批准号:
    2309520
  • 财政年份:
    2023
  • 资助金额:
    $ 30万
  • 项目类别:
    Standard Grant
Collaborative Research: ATD: Fast Algorithms and Novel Continuous-depth Graph Neural Networks for Threat Detection
合作研究:ATD:用于威胁检测的快速算法和新颖的连续深度图神经网络
  • 批准号:
    2219904
  • 财政年份:
    2023
  • 资助金额:
    $ 30万
  • 项目类别:
    Standard Grant
Computational and Mathematical Studies of Compression and Distillation Methods for Deep Neural Networks and Applications
深度神经网络压缩和蒸馏方法的计算和数学研究及应用
  • 批准号:
    2151235
  • 财政年份:
    2022
  • 资助金额:
    $ 30万
  • 项目类别:
    Continuing Grant
FRG: Collaborative Research: Robust, Efficient, and Private Deep Learning Algorithms
FRG:协作研究:稳健、高效、私密的深度学习算法
  • 批准号:
    1952644
  • 财政年份:
    2020
  • 资助金额:
    $ 30万
  • 项目类别:
    Standard Grant
Collaborative Research: ATD: Robust, Accurate and Efficient Graph-Structured RNN for Spatio-Temporal Forecasting and Anomaly Detection
合作研究:ATD:用于时空预测和异常检测的鲁棒、准确和高效的图结构 RNN
  • 批准号:
    1924548
  • 财政年份:
    2019
  • 资助金额:
    $ 30万
  • 项目类别:
    Standard Grant
BIGDATA: Collaborative Research: F: Foundations of Nonconvex Problems in BigData Science and Engineering: Models, Algorithms, and Analysis
BIGDATA:协作研究:F:大数据科学与工程中非凸问题的基础:模型、算法和分析
  • 批准号:
    1632935
  • 财政年份:
    2016
  • 资助金额:
    $ 30万
  • 项目类别:
    Standard Grant
Theory and Algorithms of Transformed L1 Minimization with Applications in Data Science
变换 L1 最小化的理论和算法及其在数据科学中的应用
  • 批准号:
    1522383
  • 财政年份:
    2015
  • 资助金额:
    $ 30万
  • 项目类别:
    Standard Grant
Reaction-Diffusion Front Speeds in Chaotic and Stochastic Flows
混沌和随机流中的反应扩散前沿速度
  • 批准号:
    1211179
  • 财政年份:
    2012
  • 资助金额:
    $ 30万
  • 项目类别:
    Continuing Grant
ATD: Blind and Template Assisted Source Separation Algorithms with Applications to Spectroscopic Data
ATD:盲和模板辅助源分离算法及其在光谱数据中的应用
  • 批准号:
    1222507
  • 财政年份:
    2012
  • 资助金额:
    $ 30万
  • 项目类别:
    Continuing Grant
ADT: Sparse Blind Separation Algorithms of Spectral Mixtures and Applications
ADT:混合光谱的稀疏盲分离算法及应用
  • 批准号:
    0911277
  • 财政年份:
    2009
  • 资助金额:
    $ 30万
  • 项目类别:
    Continuing Grant

相似海外基金

Computational and Mathematical Studies of Compression and Distillation Methods for Deep Neural Networks and Applications
深度神经网络压缩和蒸馏方法的计算和数学研究及应用
  • 批准号:
    2151235
  • 财政年份:
    2022
  • 资助金额:
    $ 30万
  • 项目类别:
    Continuing Grant
Mathematical and Computational Studies on Bose-Einstein Superfluid
玻色-爱因斯坦超流体的数学和计算研究
  • 批准号:
    1913293
  • 财政年份:
    2019
  • 资助金额:
    $ 30万
  • 项目类别:
    Standard Grant
Mathematical Modelling and Computational Simulation Studies of Flagellate Motility
鞭毛运动的数学建模和计算模拟研究
  • 批准号:
    1941986
  • 财政年份:
    2017
  • 资助金额:
    $ 30万
  • 项目类别:
    Studentship
Mathematical and Computational Studies in Poroelasticity
多孔弹性的数学和计算研究
  • 批准号:
    1619969
  • 财政年份:
    2016
  • 资助金额:
    $ 30万
  • 项目类别:
    Standard Grant
CAREER: Mathematical Modeling and Computational Studies of Human Seizure Initiation and Spread
职业:人类癫痫发作和传播的数学建模和计算研究
  • 批准号:
    1451384
  • 财政年份:
    2015
  • 资助金额:
    $ 30万
  • 项目类别:
    Continuing Grant
Mathematical modeling and computational studies for proliferation kinetics of tumor growth with spatial-temporal dynamics
时空动力学肿瘤生长增殖动力学的数学建模和计算研究
  • 批准号:
    1308948
  • 财政年份:
    2013
  • 资助金额:
    $ 30万
  • 项目类别:
    Continuing Grant
Mathematical and Computational Studies of Interfaces and Defects
界面和缺陷的数学和计算研究
  • 批准号:
    1016073
  • 财政年份:
    2010
  • 资助金额:
    $ 30万
  • 项目类别:
    Standard Grant
Mathematical and Computational Studies of Fuel Cell Dynamics
燃料电池动力学的数学和计算研究
  • 批准号:
    0308946
  • 财政年份:
    2005
  • 资助金额:
    $ 30万
  • 项目类别:
    Standard Grant
Engineering Research Equipment: Workstations for Computational Studies in Large-Scale Simulation and Mathematical Programming Research
工程研究设备:大规模仿真和数学规划研究中的计算研究工作站
  • 批准号:
    9423929
  • 财政年份:
    1995
  • 资助金额:
    $ 30万
  • 项目类别:
    Standard Grant
Mathematical Sciences: Studies in Commutative Algebra and Its Applications to Computational Algebra
数学科学:交换代数研究及其在计算代数中的应用
  • 批准号:
    9500786
  • 财政年份:
    1995
  • 资助金额:
    $ 30万
  • 项目类别:
    Continuing Grant
{{ showInfoDetail.title }}

作者:{{ showInfoDetail.author }}

知道了