Man, Machine, and Mathematics: A mathe-physical search for differential equation solutions via deep learning

良定的数学问题很少能在明确的、传统的意义上得到解决。这种现象的一个显著例子是在微分方程领域，其中经常存在精确解，但在分析上不可能找到。开发近似方法来估计这些参数化良好但不明确可知的解决方案是该领域的一项重要工作。该项目为通过深度学习构建此类近似提供了蓝图。我们建立我们的方法的假设满足一大类微分方程，如Frechet微分方程运营商和紧凑感兴趣的领域。我们的目标是证明在这样的假设下几乎总是能够找到神经网络解决方案模型。我们还希望对这些模型的预期误差给出明确的结果，以及量化和最小化这些误差的技术。最后，我们的目标是在模型大小、架构、优化运行时间等方面提供严格的保证，该项目希望通过增加两个新的方面来推进基于神经网络的微分方程求解领域。首先，通过将数值方法和误差分析的思想融合到参数化对象的深度学习中，我们希望将该领域从依赖替代标记（如损失函数）作为误差分析的手段中解脱出来。反过来，这也将引导我们找到有效的纠错方法。第二，我们的目标是通过利用计算基础设施的新兴进步和可以利用它们的算法，为有效的模型构建提供一套严格的优先级可判定策略。事实上，该项目已经在动力系统微分方程建模方面取得了初步成功（Physical Review E 105，065305）和稀疏化深度网络（Redman等人，ICML 2022）。考虑到后者工作中得出的结果的一般性，（它涉及所有深度网络，而不仅仅是微分方程求解模型），我们相信该项目将导致超出其最初计划范围的重大成果。这些目标的成功将需要在随机动力系统，随机计算方法，方程求解的深度学习方法，和优化/复杂性分析等，其中每一个都在CDT的研究和培训任务中发挥着核心作用。

Well posed mathematical problems are seldom solvable in an explicit, conventional sense. A notable example of this phenomenon is within the field of differential equations, where an exact solution often exists but is analytically impossible to find. The development of approximation methods to estimate such well-parametrized but not explicitly knowable solutions is an important endeavour within the field.This project presents a blueprint for building such approximations via deep learning. We build our methods on assumptions satisfied by a large class of differential equations, such as Frechet differentiability of the equation operators and compact domains of interest. We aim to demonstrate that neural network solution models are almost always capable of being found under such assumptions. We also hope to present explicit results on the errors expected from these models, alongside techniques for quantifying and minimising those errors. Finally, we aim to provide strict guarantees on the model sizes, architectures, optimization run-times, etc., needed to search for models that are accurate up to pre-specified tolerances.The project will hope to advance the field of neural network-based differential equation solving by adding two novel facets to it. First, by fusing ideas from numerical methods and error analysis into deep learning of parametrized objects, we hope to move the field away from its reliance on surrogate markers, like loss functions, as a means of error analysis. In turn, that will also lead us to methods of efficient error correction.Second, we aim to provide a rigorous set of a priori-decidable strategies for efficient model building by leveraging the emerging advances in computing infrastructure and the algorithms that can tap into them.Indeed, the project has already seen its first successes in the modelling of dynamical systems' differential equations (Physical Review E 105, 065305) and in sparsifying deep networks (Redman et al., ICML 2022). Given the generality of the results developed in the latter work (it concerns all deep networks, not just differential equation solution models), we believe the project will lead to significant results beyond its originally planned scope.Success in these objectives will require the creation of new techniques within the fields of random dynamical systems, stochastic computational methods, deep learning methods for equation solving, and optimization/complexity analysis, amongst others, each of which plays a central role in the CDT's research and training missions.