Physics Based Architectures for Deep Neural Networks
基于物理的深度神经网络架构
基本信息
- 批准号:RGPIN-2019-04052
- 负责人:
- 金额:$ 2.7万
- 依托单位:
- 依托单位国家:加拿大
- 项目类别:Discovery Grants Program - Individual
- 财政年份:2022
- 资助国家:加拿大
- 起止时间:2022-01-01 至 2023-12-31
- 项目状态:已结题
- 来源:
- 关键词:
项目摘要
Deep Neural Networks (DNNs) have been the engine behind the recent deep learning revolution. Such networks have demonstrated remarkable capabilities in solving image classification problems, image segmentation problems and other computer vision tasks. Such tasks are common in applications such as face recognition, autonomous cars and medical image understanding. In recent years, modifications to these networks have been proposed, and in particular, residual networks have shown to improve over existing techniques, and yield even better results. More specifically, residual networks are easier to train, have been used with hundreds and even thousands of layers and can be made memory efficient. Encouraged from these results, a few researchers have attempted to explain the success of such networks by analyzing them as discrete dynamical systems, using theory from ordinary differential equations and optimal control. While initial results are interesting, there is still a large gap between the current technology of training and understanding DNNs, to other fields that use similar methodologies, such as time dependent optimal control, inverse problems and partial differential equations based optimization. Furthermore, DNNs do not have an over-arching continuous analog which makes them somewhat add-hoc and problem/data dependent. Understanding how to move networks between scales, changing the network parameters and understanding its properties are all difficult and in many cases, impossible. The goal of this work is to suggest and develop new ideas for the understanding of DNNs. In particular, we exploit the interpretation of neural networks as dynamical systems in order to propose physics based, or physically motivated networks. These networks represent physical processes such as nonlinear diffusion and nonlinear wave propagation similar to the partial differential equations that have been used for compressed sensing. Such networks can have favorable properties in term of their overall character and may be more amendable to fast optimization algorithms that utilize this structure such as multigrid methods. I will be training 3 Ph.D. students as well as 3-5 undergraduate research assistants in the fundamentals of machine learning and its application to problems in geoscience imaging.Overall, this research program will increase scientific knowledge in the computing of machine learning and will contribute to the growing number of companies working on AI in Canada. We expect students trained via the proposed research will pursue promising professional careers in either academia or industry.
深度神经网络(DNN)是最近深度学习革命背后的引擎。这种网络在解决图像分类问题、图像分割问题和其他计算机视觉任务方面表现出了非凡的能力。这类任务在人脸识别、自动驾驶汽车和医学图像理解等应用中很常见。近年来,已经提出了对这些网络的修改,特别是残差网络已显示出比现有技术更好的改进,并产生更好的结果。更具体地说,残差网络更容易训练,已被用于数百甚至数千层,并可以提高内存效率。受到这些结果的鼓舞,一些研究人员试图通过将它们分析为离散动态系统,使用常微分方程组和最优控制的理论来解释此类网络的成功。虽然初步结果很有趣,但当前的DNN训练和理解技术与其他使用类似方法的领域仍有很大差距,例如依赖时间的最优控制、反问题和基于偏微分方程组的优化。此外,这些DNN没有全面的连续模拟,这使得它们有点附加和问题/数据依赖。了解如何在不同比例之间移动网络、更改网络参数以及了解其属性都是困难的,在许多情况下,甚至是不可能的。这项工作的目标是为理解DNN提供建议和开发新的想法。特别是,我们利用神经网络作为动态系统的解释来提出基于物理的或由物理驱动的网络。这些网络表示物理过程,如非线性扩散和非线性波传播,类似于用于压缩传感的偏微分方程组。这种网络在整体特性方面可以具有良好的性质,并且可能更适合于使用这种结构的快速优化算法,例如多重网格法。我将培训3名博士生以及3-5名本科生研究助理,学习机器学习的基础知识及其在地学成像问题中的应用。总体而言,这个研究计划将增加机器学习计算方面的科学知识,并将为加拿大越来越多的致力于人工智能的公司做出贡献。我们预计,通过拟议的研究培训的学生将在学术界或行业追求有前途的职业生涯。
项目成果
期刊论文数量(0)
专著数量(0)
科研奖励数量(0)
会议论文数量(0)
专利数量(0)
数据更新时间:{{ journalArticles.updateTime }}
{{
item.title }}
{{ item.translation_title }}
- DOI:
{{ item.doi }} - 发表时间:
{{ item.publish_year }} - 期刊:
- 影响因子:{{ item.factor }}
- 作者:
{{ item.authors }} - 通讯作者:
{{ item.author }}
数据更新时间:{{ journalArticles.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ monograph.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ sciAawards.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ conferencePapers.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ patent.updateTime }}
Haber, Eldad其他文献
A computational framework for image-based constrained registration
- DOI:
10.1016/j.laa.2009.03.020 - 发表时间:
2009-07-15 - 期刊:
- 影响因子:1.1
- 作者:
Haber, Eldad;Heldmann, Stefan;Modersitzki, Jan - 通讯作者:
Modersitzki, Jan
EXPERIMENTAL DESIGN FOR BIOLOGICAL SYSTEMS
- DOI:
10.1137/100791063 - 发表时间:
2012-01-01 - 期刊:
- 影响因子:2.2
- 作者:
Chung, Matthias;Haber, Eldad - 通讯作者:
Haber, Eldad
An octree multigrid method for quasi-static Maxwell's equations with highly discontinuous coefficients
- DOI:
10.1016/j.jcp.2006.10.012 - 发表时间:
2007-05-01 - 期刊:
- 影响因子:4.1
- 作者:
Haber, Eldad;Heldmann, Stefan - 通讯作者:
Heldmann, Stefan
MULTILEVEL ALGORITHMS FOR LARGE-SCALE INTERIOR POINT METHODS
- DOI:
10.1137/060650799 - 发表时间:
2009-01-01 - 期刊:
- 影响因子:3.1
- 作者:
Benzi, Michele;Haber, Eldad;Taralli, Lauren - 通讯作者:
Taralli, Lauren
Stable architectures for deep neural networks
- DOI:
10.1088/1361-6420/aa9a90 - 发表时间:
2018-01-01 - 期刊:
- 影响因子:2.1
- 作者:
Haber, Eldad;Ruthotto, Lars - 通讯作者:
Ruthotto, Lars
Haber, Eldad的其他文献
{{
item.title }}
{{ item.translation_title }}
- DOI:
{{ item.doi }} - 发表时间:
{{ item.publish_year }} - 期刊:
- 影响因子:{{ item.factor }}
- 作者:
{{ item.authors }} - 通讯作者:
{{ item.author }}
{{ truncateString('Haber, Eldad', 18)}}的其他基金
Physics Based Architectures for Deep Neural Networks
基于物理的深度神经网络架构
- 批准号:
RGPIN-2019-04052 - 财政年份:2021
- 资助金额:
$ 2.7万 - 项目类别:
Discovery Grants Program - Individual
Physics Based Architectures for Deep Neural Networks
基于物理的深度神经网络架构
- 批准号:
RGPIN-2019-04052 - 财政年份:2020
- 资助金额:
$ 2.7万 - 项目类别:
Discovery Grants Program - Individual
Physics Based Architectures for Deep Neural Networks
基于物理的深度神经网络架构
- 批准号:
RGPAS-2019-00088 - 财政年份:2020
- 资助金额:
$ 2.7万 - 项目类别:
Discovery Grants Program - Accelerator Supplements
Physics Based Architectures for Deep Neural Networks
基于物理的深度神经网络架构
- 批准号:
RGPIN-2019-04052 - 财政年份:2019
- 资助金额:
$ 2.7万 - 项目类别:
Discovery Grants Program - Individual
Physics Based Architectures for Deep Neural Networks
基于物理的深度神经网络架构
- 批准号:
RGPAS-2019-00088 - 财政年份:2019
- 资助金额:
$ 2.7万 - 项目类别:
Discovery Grants Program - Accelerator Supplements
NSERC Industrial Research Chair in Advanced Computational Methods for Geophysical Electromagnetics Modeling, Inversion and Integration
NSERC 地球物理电磁学建模、反演和积分高级计算方法工业研究主席
- 批准号:
395175-2014 - 财政年份:2018
- 资助金额:
$ 2.7万 - 项目类别:
Industrial Research Chairs
NSERC Industrial Research Chair in Advanced Computational Methods for Geophysical Electromagnetics Modeling, Inversion and Integration
NSERC 地球物理电磁学建模、反演和积分高级计算方法工业研究主席
- 批准号:
395175-2014 - 财政年份:2017
- 资助金额:
$ 2.7万 - 项目类别:
Industrial Research Chairs
NSERC Industrial Research Chair in Advanced Computational Methods for Geophysical Electromagnetics Modeling, Inversion and Integration
NSERC 地球物理电磁学建模、反演和积分高级计算方法工业研究主席
- 批准号:
395175-2014 - 财政年份:2014
- 资助金额:
$ 2.7万 - 项目类别:
Industrial Research Chairs
NSERC/ Barrick/Xstrata/TeckCominco/Newmont/Vale Industrial Research Chair in Computational Geoscience
NSERC/ Barrick/Xstrata/TeckCominco/Newmont/Vale 计算地球科学工业研究主席
- 批准号:
395175-2008 - 财政年份:2013
- 资助金额:
$ 2.7万 - 项目类别:
Industrial Research Chairs
NSERC/ Barrick/Xstrata/TeckCominco/Newmont/Vale Industrial Research Chair in Computational Geoscience
NSERC/ Barrick/Xstrata/TeckCominco/Newmont/Vale 计算地球科学工业研究主席
- 批准号:
395175-2008 - 财政年份:2012
- 资助金额:
$ 2.7万 - 项目类别:
Industrial Research Chairs
相似国自然基金
Data-driven Recommendation System Construction of an Online Medical Platform Based on the Fusion of Information
- 批准号:
- 批准年份:2024
- 资助金额:万元
- 项目类别:外国青年学者研究基金项目
Exploring the Intrinsic Mechanisms of CEO Turnover and Market Reaction: An Explanation Based on Information Asymmetry
- 批准号:W2433169
- 批准年份:2024
- 资助金额:万元
- 项目类别:外国学者研究基金项目
Incentive and governance schenism study of corporate green washing behavior in China: Based on an integiated view of econfiguration of environmental authority and decoupling logic
- 批准号:
- 批准年份:2024
- 资助金额:万元
- 项目类别:外国学者研究基金项目
A study on prototype flexible multifunctional graphene foam-based sensing grid (柔性多功能石墨烯泡沫传感网格原型研究)
- 批准号:
- 批准年份:2020
- 资助金额:20 万元
- 项目类别:
基于tag-based单细胞转录组测序解析造血干细胞发育的可变剪接
- 批准号:81900115
- 批准年份:2019
- 资助金额:21.0 万元
- 项目类别:青年科学基金项目
应用Agent-Based-Model研究围术期单剂量地塞米松对手术切口愈合的影响及机制
- 批准号:81771933
- 批准年份:2017
- 资助金额:50.0 万元
- 项目类别:面上项目
Reality-based Interaction用户界面模型和评估方法研究
- 批准号:61170182
- 批准年份:2011
- 资助金额:57.0 万元
- 项目类别:面上项目
Multistage,haplotype and functional tests-based FCAR 基因和IgA肾病相关关系研究
- 批准号:30771013
- 批准年份:2007
- 资助金额:30.0 万元
- 项目类别:面上项目
差异蛋白质组技术结合Array-based CGH 寻找骨肉瘤分子标志物
- 批准号:30470665
- 批准年份:2004
- 资助金额:8.0 万元
- 项目类别:面上项目
GaN-based稀磁半导体材料与自旋电子共振隧穿器件的研究
- 批准号:60376005
- 批准年份:2003
- 资助金额:20.0 万元
- 项目类别:面上项目
相似海外基金
Scalable and Automated Tuning of Spin-based Quantum Computer Architectures
基于自旋的量子计算机架构的可扩展和自动调整
- 批准号:
2887634 - 财政年份:2024
- 资助金额:
$ 2.7万 - 项目类别:
Studentship
Collaborative Research: SHF: Medium: EPIC: Exploiting Photonic Interconnects for Resilient Data Communication and Acceleration in Energy-Efficient Chiplet-based Architectures
合作研究:SHF:中:EPIC:利用光子互连实现基于节能 Chiplet 的架构中的弹性数据通信和加速
- 批准号:
2311543 - 财政年份:2023
- 资助金额:
$ 2.7万 - 项目类别:
Continuing Grant
CAREER: Addressing Scalability Challenges in Designing Next-generation GPU-Based Heterogeneous Architectures
职业:解决设计下一代基于 GPU 的异构架构时的可扩展性挑战
- 批准号:
2316694 - 财政年份:2023
- 资助金额:
$ 2.7万 - 项目类别:
Continuing Grant
Collaborative Research: SHF: Medium: EPIC: Exploiting Photonic Interconnects for Resilient Data Communication and Acceleration in Energy-Efficient Chiplet-based Architectures
合作研究:SHF:中:EPIC:利用光子互连实现基于节能 Chiplet 的架构中的弹性数据通信和加速
- 批准号:
2311544 - 财政年份:2023
- 资助金额:
$ 2.7万 - 项目类别:
Continuing Grant
HVDC-WISE (HVDC-based grid architectures for reliable and resilient WIdeSprEad hybrid AC/DC transmission systems)
HVDC-WISE(基于 HVDC 的电网架构,用于可靠且有弹性的 WIdeSprEad 混合交流/直流输电系统)
- 批准号:
10041877 - 财政年份:2022
- 资助金额:
$ 2.7万 - 项目类别:
EU-Funded
HVDC-WISE: HVDC-based grid architectures for reliable and resilient WIdeSprEad hybrid AC/DC transmission systems
HVDC-WISE:基于 HVDC 的电网架构,用于可靠且有弹性的 WIdeSprEad 混合交流/直流输电系统
- 批准号:
10051113 - 财政年份:2022
- 资助金额:
$ 2.7万 - 项目类别:
EU-Funded
Memristor-based Architectures for Neuromorphic Computing
用于神经形态计算的基于忆阻器的架构
- 批准号:
RGPIN-2020-06613 - 财政年份:2022
- 资助金额:
$ 2.7万 - 项目类别:
Discovery Grants Program - Individual
Next-generation audio and speech processing architectures based on deep learning
基于深度学习的下一代音频和语音处理架构
- 批准号:
22H03614 - 财政年份:2022
- 资助金额:
$ 2.7万 - 项目类别:
Grant-in-Aid for Scientific Research (B)
New Deep Neural Network Architectures for Blind Image Watermarking Based on the Information-Theoretic Principles
基于信息论原理的新型图像盲水印深度神经网络架构
- 批准号:
547243-2020 - 财政年份:2022
- 资助金额:
$ 2.7万 - 项目类别:
Alexander Graham Bell Canada Graduate Scholarships - Doctoral
New Deep Neural Network Architectures for Blind Image Watermarking Based on the Information-Theoretic Principles
基于信息论原理的新型图像盲水印深度神经网络架构
- 批准号:
547243-2020 - 财政年份:2021
- 资助金额:
$ 2.7万 - 项目类别:
Alexander Graham Bell Canada Graduate Scholarships - Doctoral