Information-theoretic analysis and synthesis in deep learning

深度学习中的信息论分析与综合

基本信息

  • 批准号:
    RGPIN-2020-06285
  • 负责人:
  • 金额:
    $ 2.4万
  • 依托单位:
  • 依托单位国家:
    加拿大
  • 项目类别:
    Discovery Grants Program - Individual
  • 财政年份:
    2020
  • 资助国家:
    加拿大
  • 起止时间:
    2020-01-01 至 2021-12-31
  • 项目状态:
    已结题

项目摘要

The methodology of deep neural networks, or deep learning, has revolutionized the field of machine learning and now prevails in numerous fields of applied sciences. Despite its great successes, the working mechanism of deep learning is still poorly understood to date. As a consequence, practical applications of this methodology primarily rely on intuitions and heuristics. In this project, we aim at developing new theoretical understanding of deep learning and inventing novel, more effective, and principled techniques for modelling with deep neural networks and training of such models. A key hypothesis of this project is that information theory, a mathematical theory originally developed for the study of data compression and data transmission, can be used as an effective tool for the analysis and synthesis in deep learning. Built on the concepts of entropy and mutual information, which characterize notions of information, information theory possesses a powerful set of analytic techniques that have allowed the determination of the fundamental limits of data compression and data communication. In the deep learning era, information-theoretic concepts and techniques have also been applied to the construction and training of neural network models. More recently information theory has also been demonstrated as a promising tool to theorize deep learning and analyze learning algorithms. In this project, we propose to use information theory, jointly with other mathematical techniques and computer simulations, to study deep learning. Specifically, this research aims to achieve the following objectives. 1) Developing better understanding of the generalization behaviour of deep neural networks. 2) Developing more effective network architectures and training methods for deep learning. 3) Developing principled and more effective data-dependent regularization schemes. This project will provide new scientific knowledge to the research in deep learning. It will result in innovative and more effective learning techniques and offer fundamental insights and important guidelines to applying deep learning in practice. The success of this project will further strengthen Canada's leading position in AI research. The techniques developed in this research may benefit Canadian industry in developing innovative AI technologies.
深度神经网络或深度学习的方法论已经彻底改变了机器学习领域,现在盛行于应用科学的许多领域。尽管深度学习取得了巨大的成功,但人们对深度学习的工作机制仍然知之甚少。因此,这种方法的实际应用主要依赖于直觉和启发式方法。在这个项目中,我们的目标是发展对深度学习的新的理论理解,并发明新的、更有效的和原则性的技术来使用深度神经网络建模和训练此类模型。 这个项目的一个关键假设是,信息论是一种最初为研究数据压缩和数据传输而发展起来的数学理论,可以作为深度学习中分析和综合的有效工具。信息论建立在表征信息概念的熵和互信息概念的基础上,拥有一套强大的分析技术,可以确定数据压缩和数据通信的基本界限。在深度学习时代,信息论的概念和技术也被应用于神经网络模型的构建和训练。最近,信息论也被证明是一种很有前途的工具,可以用来对深度学习进行理论分析和学习算法分析。 在这个项目中,我们建议使用信息论,结合其他数学技术和计算机模拟,来研究深度学习。具体地说,本研究旨在实现以下目标。 1)更好地理解深度神经网络的泛化行为。 2)为深度学习开发更有效的网络体系结构和训练方法。 3)开发原则性和更有效的数据依赖正则化方案。 该项目将为深度学习的研究提供新的科学知识。它将产生创新和更有效的学习技术,并为在实践中应用深度学习提供基本的见解和重要的指导方针。该项目的成功将进一步巩固加拿大在人工智能研究领域的领先地位。这项研究开发的技术可能有助于加拿大工业开发创新的人工智能技术。

项目成果

期刊论文数量(0)
专著数量(0)
科研奖励数量(0)
会议论文数量(0)
专利数量(0)

数据更新时间:{{ journalArticles.updateTime }}

{{ item.title }}
{{ item.translation_title }}
  • DOI:
    {{ item.doi }}
  • 发表时间:
    {{ item.publish_year }}
  • 期刊:
  • 影响因子:
    {{ item.factor }}
  • 作者:
    {{ item.authors }}
  • 通讯作者:
    {{ item.author }}

数据更新时间:{{ journalArticles.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ monograph.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ sciAawards.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ conferencePapers.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ patent.updateTime }}

Mao, Yongyi其他文献

Trust Prediction via Belief Propagation
通过置信传播进行信任预测
Rateless coding for wireless relay channels
Recurrent Neural Networks With Finite Memory Length
  • DOI:
    10.1109/access.2018.2890297
  • 发表时间:
    2019-01-01
  • 期刊:
  • 影响因子:
    3.9
  • 作者:
    Long, Dingkun;Zhang, Richong;Mao, Yongyi
  • 通讯作者:
    Mao, Yongyi

Mao, Yongyi的其他文献

{{ item.title }}
{{ item.translation_title }}
  • DOI:
    {{ item.doi }}
  • 发表时间:
    {{ item.publish_year }}
  • 期刊:
  • 影响因子:
    {{ item.factor }}
  • 作者:
    {{ item.authors }}
  • 通讯作者:
    {{ item.author }}

{{ truncateString('Mao, Yongyi', 18)}}的其他基金

Information-theoretic analysis and synthesis in deep learning
深度学习中的信息论分析与综合
  • 批准号:
    RGPIN-2020-06285
  • 财政年份:
    2022
  • 资助金额:
    $ 2.4万
  • 项目类别:
    Discovery Grants Program - Individual
Information-theoretic analysis and synthesis in deep learning
深度学习中的信息论分析与综合
  • 批准号:
    RGPIN-2020-06285
  • 财政年份:
    2021
  • 资助金额:
    $ 2.4万
  • 项目类别:
    Discovery Grants Program - Individual
Forecasting Covid-19 Epidemic in Canada with Spatial-Temporal Models That Exploit Population Behaviour on Twitter
利用 Twitter 上人群行为的时空模型预测加拿大的 Covid-19 疫情
  • 批准号:
    550139-2020
  • 财政年份:
    2020
  • 资助金额:
    $ 2.4万
  • 项目类别:
    Alliance Grants
Coding for Flash Memory
闪存编码
  • 批准号:
    RGPIN-2015-06596
  • 财政年份:
    2019
  • 资助金额:
    $ 2.4万
  • 项目类别:
    Discovery Grants Program - Individual
Coding for Flash Memory
闪存编码
  • 批准号:
    RGPIN-2015-06596
  • 财政年份:
    2018
  • 资助金额:
    $ 2.4万
  • 项目类别:
    Discovery Grants Program - Individual
Coding for Flash Memory
闪存编码
  • 批准号:
    RGPIN-2015-06596
  • 财政年份:
    2017
  • 资助金额:
    $ 2.4万
  • 项目类别:
    Discovery Grants Program - Individual
Coding for Flash Memory
闪存编码
  • 批准号:
    RGPIN-2015-06596
  • 财政年份:
    2016
  • 资助金额:
    $ 2.4万
  • 项目类别:
    Discovery Grants Program - Individual
Coding for Flash Memory
闪存编码
  • 批准号:
    RGPIN-2015-06596
  • 财政年份:
    2015
  • 资助金额:
    $ 2.4万
  • 项目类别:
    Discovery Grants Program - Individual
Incremental redundancy and feedback schemes for practical applications
实际应用的增量冗余和反馈方案
  • 批准号:
    293239-2010
  • 财政年份:
    2014
  • 资助金额:
    $ 2.4万
  • 项目类别:
    Discovery Grants Program - Individual
Distributed compression of massive weather data with spatial and temporal correlations
具有时空相关性的海量天气数据的分布式压缩
  • 批准号:
    469190-2014
  • 财政年份:
    2014
  • 资助金额:
    $ 2.4万
  • 项目类别:
    Engage Grants Program

相似海外基金

Single cell RNA-seq data-driven method for cell type identification by information theoretic analysis
单细胞 RNA-seq 数据驱动的信息论分析细胞类型识别方法
  • 批准号:
    22K15091
  • 财政年份:
    2022
  • 资助金额:
    $ 2.4万
  • 项目类别:
    Grant-in-Aid for Early-Career Scientists
Information-theoretic analysis and synthesis in deep learning
深度学习中的信息论分析与综合
  • 批准号:
    RGPIN-2020-06285
  • 财政年份:
    2022
  • 资助金额:
    $ 2.4万
  • 项目类别:
    Discovery Grants Program - Individual
Information-theoretic analysis and synthesis in deep learning
深度学习中的信息论分析与综合
  • 批准号:
    RGPIN-2020-06285
  • 财政年份:
    2021
  • 资助金额:
    $ 2.4万
  • 项目类别:
    Discovery Grants Program - Individual
Information theoretic learning for sound analysis
用于声音分析的信息论学习
  • 批准号:
    2594237
  • 财政年份:
    2021
  • 资助金额:
    $ 2.4万
  • 项目类别:
    Studentship
CRII: CIF: New Paradigms in Generalization and Information-Theoretic Analysis of Deep Neural Networks
CRII:CIF:深度神经网络泛化和信息论分析的新范式
  • 批准号:
    1947801
  • 财政年份:
    2020
  • 资助金额:
    $ 2.4万
  • 项目类别:
    Standard Grant
Developing an information-theoretic predictive factor analysis method with application to transdiagnostic psychometric and neurocognitive data
开发应用于跨诊断心理测量和神经认知数据的信息论预测因素分析方法
  • 批准号:
    2587468
  • 财政年份:
    2020
  • 资助金额:
    $ 2.4万
  • 项目类别:
    Studentship
CIF: Small: An Information Theoretic Framework for Minimizing Supervision in Image/Video Analysis
CIF:小:最小化图像/视频分析中的监督的信息理论框架
  • 批准号:
    2008020
  • 财政年份:
    2020
  • 资助金额:
    $ 2.4万
  • 项目类别:
    Standard Grant
How and Why Fish School: An Information-theoretic Analysis of Coordinated Swimming
鱼群的方式和原因:协调游泳的信息论分析
  • 批准号:
    1901697
  • 财政年份:
    2019
  • 资助金额:
    $ 2.4万
  • 项目类别:
    Standard Grant
Elucidating the mechanisms of split-brain by the information-theoretic analysis of brain activity
通过大脑活动的信息论分析阐明裂脑的机制
  • 批准号:
    18H02713
  • 财政年份:
    2018
  • 资助金额:
    $ 2.4万
  • 项目类别:
    Grant-in-Aid for Scientific Research (B)
The neural representation of vocal emotion: representational similarity analysis and information-theoretic approaches
声音情感的神经表征:表征相似性分析和信息论方法
  • 批准号:
    BB/M009742/1
  • 财政年份:
    2015
  • 资助金额:
    $ 2.4万
  • 项目类别:
    Research Grant
{{ showInfoDetail.title }}

作者:{{ showInfoDetail.author }}

知道了