权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

Collaborative Research: SCALE MoDL: Representation Theoretic Foundations of Deep Learning

合作研究：SCALE MoDL：深度学习的表示理论基础

基本信息

批准号：
2134274
负责人：
Qi Yu
金额：
$ 30万
依托单位：
University of California-San Diego
依托单位国家：
美国
项目类别：
Continuing Grant
财政年份：
2022
资助国家：
美国
起止时间：
2022-01-01 至 2024-12-31
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=2134274&HistoricalAwards=false
关键词：
Collaborative Research SCALE MoDL Representation

项目摘要

In the past decade, deep learning has had transformative impacts across society. However, progress has often relied on heuristic methods, massive data, and great computing power. This comes with limited theoretical understanding and has at times given rise to failures of generalization and vulnerable performance in extreme scenarios. This project will address these limitations by developing strong theoretical foundations for deep learning using representation theory, which is the mathematical study of symmetry. Symmetry plays a key role in human reasoning. Greater understanding of the role symmetry plays in deep learning will unlock a variety of improved models. These include models that can learn from scientific knowledge and not just raw data, models with trustable, guaranteed performance, and models that can recombine patterns they have already learned — as humans do easily — to generalize to new situations more rapidly. An explicit goal of this project is to broaden research into why deep learning works. To this end, the investigators will integrate the research into education and establish a mentorship program for high school students from groups underrepresented in science.The goal of the research is to understand the role of representation theory in enabling efficient optimization and improved generalization of deep learning even in domains with approximate or unknown symmetry. This project pursues three lines of research that will broaden the impact of representation theory in deep learning beyond strict inductive biases. The first is the trade-off between the degree of symmetry in the model and the degree of symmetry in the domain. This line of research will study networks that combine equivariant and non-equivariant features. The second line of research will examine learning symmetry directly from data to improve generalization in domains without known symmetries. The third aim is to develop a theoretical basis for deep learning using quiver representations. This perspective reveals the symmetry of the structure of deep-learning models themselves, through their parameter spaces, even when the domains have no obvious symmetry.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

在过去的十年中，深度学习对整个社会产生了变革性的影响。然而，进步往往依赖于启发式方法，海量数据和强大的计算能力。这伴随着有限的理论理解，有时会导致泛化失败和极端情况下的脆弱性能。该项目将通过使用表征理论为深度学习开发强大的理论基础来解决这些限制，表征理论是对称性的数学研究。对称性在人类推理中起着关键作用。更好地理解对称性在深度学习中的作用将解锁各种改进的模型。这些模型包括可以从科学知识而不仅仅是原始数据中学习的模型，具有可信赖的，有保证的性能的模型，以及可以重新组合他们已经学习的模式的模型-就像人类很容易做的那样-更快地推广到新的情况。这个项目的一个明确目标是扩大对深度学习为什么有效的研究。为此，研究人员将把这项研究融入教育，并为来自科学领域代表性不足的高中生建立导师计划。研究的目标是了解表征理论在实现深度学习的有效优化和改进泛化方面的作用，即使在近似或未知对称性的领域。该项目追求三条研究路线，将扩展表征理论在深度学习中的影响，超越严格的归纳偏见。首先是模型中的对称度和域中的对称度之间的权衡。这条研究路线将研究结合了联合收割机等变和非等变特征的网络。第二条研究路线将研究直接从数据中学习对称性，以提高没有已知对称性的领域的泛化能力。第三个目标是为深度学习提供理论基础。该奖项反映了NSF的法定使命，通过使用基金会的知识价值和更广泛的影响评审标准进行评估，被认为值得支持。