权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

CAREER: Towards theoretical foundations of neural network based representation learning

职业：迈向基于神经网络的表示学习的理论基础

基本信息

批准号：
2145703
负责人：
Yuanzhi Li
金额：
$ 64.09万
依托单位：
Carnegie-Mellon University
依托单位国家：
美国
项目类别：
Continuing Grant
财政年份：
2022
资助国家：
美国
起止时间：
2022-02-01 至 2027-01-31
项目状态：
未结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=2145703&HistoricalAwards=false
关键词：
CAREER Towards theoretical foundations neural

项目摘要

Building up good representations of input data has always been a central ingredient in machine learning. In computer vision, for example, one would like to have features representing the main objects in the image. In natural language processing, one would like to have features indicating the relationship between different words. A new paradigm shift in machine learning based on deep learning techniques has demonstrated the ability of machines to automatically learn good representations from the training data set without any prior knowledge. However, although these features are really useful for machines to learn the data set, are they actually "good" according to human standards? The project aims to contribute to the fundamental understanding of deep representation learning and inform the practical advancement of deep learning, improving its interpretability, robustness, and efficiency in large data regimes. The investigator will also develop a new graduate-level course and a public interactive software through the course of this project. The project aims to build a comprehensive theory for the new generation of neural-network-based representation-learning techniques. This includes the central questions of characterizing the statistical properties of the representations and how they are encoded in an actual neural network. This project has three major components. The first thrust is to characterize when would minimizing the training objective of the representation learning task leads to a unique representation in the neural network: leveraging the new theoretical development, the investigator will build up new training objectives that encourage such uniqueness. The second thrust is to theoretically study what representations can be efficiently learned by deep-learning models, and how are they encoded in the hidden weights of the neural networks after training. Finally, the investigator will study what statistical properties of the learned representations made them good for downstream tasks, which is critical to improving the interpretability of these neural network-based representations. Moreover, it will allow humans to better interact with deep-learning models for broader applications such as self-driving cars.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

建立良好的输入数据表示一直是机器学习的核心内容。例如，在计算机视觉中，人们希望具有代表图像中主要对象的特征。在自然语言处理中，人们希望具有指示不同单词之间的关系的特征。基于深度学习技术的机器学习的新范式转变已经证明了机器在没有任何先验知识的情况下自动从训练数据集中学习良好表示的能力。然而，尽管这些功能对于机器学习数据集确实很有用，但按照人类的标准，它们真的“好”吗？该项目旨在促进对深度表示学习的基本理解，并为深度学习的实际进展提供信息，提高其在大数据体系中的可解释性，鲁棒性和效率。调查员还将通过该项目的过程开发一个新的研究生课程和一个公共互动软件。该项目旨在为新一代基于神经网络的表示学习技术建立一个全面的理论。这包括表征表征的统计特性以及它们如何在实际神经网络中编码的核心问题。该项目有三个主要组成部分。第一个重点是表征何时最小化表征学习任务的训练目标会导致神经网络中的独特表征：利用新的理论发展，研究人员将建立新的训练目标，鼓励这种独特性。第二个重点是从理论上研究深度学习模型可以有效地学习哪些表示，以及它们在训练后如何编码到神经网络的隐藏权重中。最后，研究人员将研究学习到的表示的统计特性使它们适合下游任务，这对于提高这些基于神经网络的表示的可解释性至关重要。此外，它将使人类能够更好地与深度学习模型进行交互，以实现更广泛的应用，如自动驾驶汽车。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。