权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

Collaborative Research: Transferable, Hierarchical, Expressive, Optimal, Robust, Interpretable Networks

协作研究：可转移、分层、富有表现力、最优、稳健、可解释的网络

基本信息

批准号：
2031985
负责人：
Rene Vidal
金额：
$ 165万
依托单位：
Johns Hopkins University
依托单位国家：
美国
项目类别：
Continuing Grant
财政年份：
2020
资助国家：
美国
起止时间：
2020-09-01 至 2025-08-31
项目状态：
未结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=2031985&HistoricalAwards=false
关键词：
Collaborative Research Transferable Hierarchical Expressive

项目摘要

Recent advances in deep learning have led to many disruptive technologies: from automatic speech recognition systems, to automated supermarkets, to self-driving cars. However, the complex and large-scale nature of deep networks makes them hard to analyze and, therefore, they are mostly used as black-boxes without formal guarantees on their performance. For example, deep networks provide a self-reported confidence score, but they are frequently inaccurate and uncalibrated, or likely to make large mistakes on rare cases. Moreover, the design of deep networks remains an art and is largely driven by empirical performance on a dataset. As deep learning systems are increasingly employed in our daily lives, it becomes critical to understand if their predictions satisfy certain desired properties. The goal of this NSF-Simons Research Collaboration on the Mathematical and Scientific Foundations of Deep Learning is to develop a mathematical, statistical and computational framework that helps explain the success of current network architectures, understand its pitfalls, and guide the design of novel architectures with guaranteed confidence, robustness, interpretability, optimality, and transferability. This project will train a diverse STEM workforce with data science skills that are essential for the global competitiveness of the US economy by creating new undergraduate and graduate programs in the foundations of data science and organizing a series of collaborative research events, including semester research programs and summer schools on the foundations of deep learning. This project will also impact women and underrepresented minorities by involving undergraduates in the foundations of data science.Deep networks have led to dramatic improvements in the performance of pattern recognition systems. However, the mathematical reasons for this success remain elusive. For instance, it is not clear why deep networks generalize or transfer to new tasks, or why simple optimization strategies can reach a local or global minimum of the associated non-convex optimization problem. Moreover, there is no principled way of designing the architecture of the network so that it satisfies certain desired properties, such as expressivity, transferability, optimality and robustness. This project brings together a multidisciplinary team of mathematicians, statisticians, theoretical computer scientists, and electrical engineers to develop the mathematical and scientific foundations of deep learning. The project is divided in four main thrusts. The analysis thrust will use principles from approximation theory, information theory, statistical inference, and robust control to analyze properties of deep networks such as expressivity, interpretability, confidence, fairness and robustness. The learning thrust will use principles from dynamical systems, non-convex and stochastic optimization, statistical learning theory, adaptive control, and high-dimensional statistics to design and analyze learning algorithms with guaranteed convergence, optimality and generalization properties. The design thrust will use principles from algebra, geometry, topology, graph theory and optimization to design and learn network architectures that capture algebraic, geometric and graph structures in both the data and the task. The transferability thrust will use principles from multiscale analysis and modeling, reinforcement learning, and Markov decision processes to design and study data representations that are suitable for learning from and transferring to multiple tasks.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

深度学习的最新进展带来了许多颠覆性技术：从自动语音识别系统到自动超市，再到自动驾驶汽车。然而，深度网络的复杂性和大规模性质使得它们难以分析，因此，它们大多被用作黑盒，没有对其性能的正式保证。例如，深度网络提供了一个自我报告的信心评分，但它们经常是不准确和未经校准的，或者在极少数情况下可能会犯很大的错误。此外，深度网络的设计仍然是一门艺术，很大程度上是由数据集上的经验性能驱动的。随着深度学习系统越来越多地应用于我们的日常生活，了解它们的预测是否满足某些期望属性变得至关重要。NSF-Simons深度学习的数学和科学基础研究合作的目标是开发一个数学、统计和计算框架，帮助解释当前网络架构的成功，理解其缺陷，并指导具有保证信心、鲁棒性、可解释性、最优性和可转移性的新架构的设计。该项目将通过创建新的数据科学基础本科和研究生课程，并组织一系列合作研究活动，包括学期研究项目和深度学习基础暑期学校，培养具有数据科学技能的多样化STEM劳动力，这对美国经济的全球竞争力至关重要。该项目还将通过让本科生参与数据科学基础，影响女性和代表性不足的少数族裔。深度网络极大地改善了模式识别系统的性能。然而，这种成功的数学原因仍然难以捉摸。例如，为什么深度网络泛化或转移到新的任务中，或者为什么简单的优化策略可以达到相关非凸优化问题的局部或全局最小值，这一点尚不清楚。此外，没有原则性的方法来设计网络架构，使其满足某些期望的属性，如表达性、可转移性、最优性和鲁棒性。该项目汇集了一个由数学家、统计学家、理论计算机科学家和电气工程师组成的多学科团队，以发展深度学习的数学和科学基础。该项目分为四个主要部分。分析推力将使用近似理论、信息论、统计推断和鲁棒控制的原理来分析深度网络的特性，如表达性、可解释性、置信度、公平性和鲁棒性。学习推力将使用动力系统、非凸和随机优化、统计学习理论、自适应控制和高维统计的原理来设计和分析具有保证收敛性、最优性和泛化性质的学习算法。设计主旨将使用代数、几何、拓扑、图论和优化的原理来设计和学习网络架构，以捕获数据和任务中的代数、几何和图结构。可转移性推力将使用多尺度分析和建模、强化学习和马尔可夫决策过程的原理来设计和研究适合从多个任务中学习和转移的数据表示。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。