Man, Machine, and Mathematics: A mathe-physical search for differential equation solutions via deep learning

好的数学问题很少在显式的、常规的意义上是可解的。这一现象的一个显著例子是在微分方程组领域，在那里通常存在精确的解，但在分析上是不可能找到的。发展近似方法来估计这类参数良好但不明确可知的解是该领域内的一项重要努力。该项目为通过深度学习建立此类近似提供了一个蓝图。我们的方法建立在一大类微分方程所满足的假设上，例如方程算子的Frechet可微性和紧致的感兴趣区域。我们的目标是证明神经网络解模型几乎总是能够在这样的假设下找到。我们还希望提供关于这些模型预期误差的明确结果，以及量化和最小化这些误差的技术。最后，我们的目标是在模型大小、体系结构、优化运行时间等方面提供严格的保证，以寻找精度达到预定容差的模型。该项目有望通过增加两个新的方面来推动基于神经网络的微分方程求解领域的发展。首先，通过将数值方法和误差分析的思想融合到对参数化对象的深度学习中，我们希望使该领域摆脱对替代标记的依赖，如损失函数，作为误差分析的手段。其次，我们的目标是通过利用计算基础设施和可以利用它们的算法的新进步，为高效的建模提供一套严格的先验可决定的策略。事实上，该项目已经在动力系统的微分方程建模(物理评论E105,065305)和稀疏深层网络(Redman等人，ICML2022)方面取得了第一次成功。考虑到后一项工作中开发的结果的普遍性(它涉及所有的深层网络，而不仅仅是微分方程解模型)，我们相信该项目将带来超出其最初计划范围的重大结果。要实现这些目标，将需要在随机动力系统、随机计算方法、方程求解的深度学习方法和优化/复杂性分析等领域创造新的技术，其中每一项都在CDT的研究和培训任务中发挥核心作用。

Well posed mathematical problems are seldom solvable in an explicit, conventional sense. A notable example of this phenomenon is within the field of differential equations, where an exact solution often exists but is analytically impossible to find. The development of approximation methods to estimate such well-parametrized but not explicitly knowable solutions is an important endeavour within the field.This project presents a blueprint for building such approximations via deep learning. We build our methods on assumptions satisfied by a large class of differential equations, such as Frechet differentiability of the equation operators and compact domains of interest. We aim to demonstrate that neural network solution models are almost always capable of being found under such assumptions. We also hope to present explicit results on the errors expected from these models, alongside techniques for quantifying and minimising those errors. Finally, we aim to provide strict guarantees on the model sizes, architectures, optimization run-times, etc., needed to search for models that are accurate up to pre-specified tolerances.The project will hope to advance the field of neural network-based differential equation solving by adding two novel facets to it. First, by fusing ideas from numerical methods and error analysis into deep learning of parametrized objects, we hope to move the field away from its reliance on surrogate markers, like loss functions, as a means of error analysis. In turn, that will also lead us to methods of efficient error correction.Second, we aim to provide a rigorous set of a priori-decidable strategies for efficient model building by leveraging the emerging advances in computing infrastructure and the algorithms that can tap into them.Indeed, the project has already seen its first successes in the modelling of dynamical systems' differential equations (Physical Review E 105, 065305) and in sparsifying deep networks (Redman et al., ICML 2022). Given the generality of the results developed in the latter work (it concerns all deep networks, not just differential equation solution models), we believe the project will lead to significant results beyond its originally planned scope.Success in these objectives will require the creation of new techniques within the fields of random dynamical systems, stochastic computational methods, deep learning methods for equation solving, and optimization/complexity analysis, amongst others, each of which plays a central role in the CDT's research and training missions.