Structure-preserving deep learning
Structure-preserving deep learning
复制标题
DOI：
10.1017/s0956792521000139
复制
发表时间：
2020-06
影响因子：
1.9
通讯作者：
E. Celledoni;Matthias Joachim Ehrhardt;Christian Etmann;R. McLachlan;B. Owren;C. Schönlieb;Ferdia Sherry
中科院分区：
文献类型：
--
作者：
E. Celledoni;Matthias Joachim Ehrhardt;Christian Etmann;R. McLachlan;B. Owren;C. Schönlieb;Ferdia Sherry
在过去的几年里，深度学习已经成为一个备受关注的话题，这主要是由于在解决大规模图像处理任务方面取得了成功。应用深度学习涉及多个具有挑战性的数学问题：大多数深度学习方法需要解决困难的优化问题，并且需要对计算工作量，数据量和模型复杂性之间的权衡有很好的理解，才能成功地为给定问题设计深度学习方法。深度学习方面取得的大量进展是基于启发式探索，但人们越来越多地努力从数学上理解现有深度学习方法中的结构，并系统地设计新的深度学习方法以保留深度学习中的某些类型的结构。在本文中，我们回顾了这些方向：一些深度神经网络可以被理解为动力系统的离散，神经网络可以被设计成具有理想的性质，如可逆性或群等变性，并提出了基于共形哈密顿系统和黎曼流形的新算法框架来解决优化问题。我们通过讨论一些我们认为是未来研究的有趣方向的开放问题来结束我们对每个主题的回顾。
Over the past few years, deep learning has risen to the foreground as a topic of massive interest, mainly as a result of successes obtained in solving large-scale image processing tasks. There are multiple challenging mathematical problems involved in applying deep learning: most deep learning methods require the solution of hard optimisation problems, and a good understanding of the trade-off between computational effort, amount of data and model complexity is required to successfully design a deep learning approach for a given problem.. A large amount of progress made in deep learning has been based on heuristic explorations, but there is a growing effort to mathematically understand the structure in existing deep learning methods and to systematically design new deep learning methods to preserve certain types of structure in deep learning. In this article, we review a number of these directions: some deep neural networks can be understood as discretisations of dynamical systems, neural networks can be designed to have desirable properties such as invertibility or group equivariance and new algorithmic frameworks based on conformal Hamiltonian systems and Riemannian manifolds to solve the optimisation problems have been proposed. We conclude our review of each of these topics by discussing some open problems that we consider to be interesting directions for future research.