COFNET: Compositional functions networks - adaptive learning for high-dimensional approximation and uncertainty quantification

高维近似任务在科学计算和数据科学的所有领域都无处不在，包括偏微分方程（PDEs）的解、机器学习和不确定性量化（UQ）。随着最近深度神经网络（nn）和树张量网络（tn）的成功，以前棘手的问题变得可行，对可靠算法和更好地理解理论方面的需求比以往任何时候都大。这一成功引起了人们对函数组合分析的强烈兴趣，并激发了新的近似工具的引入，在保持tn的方便数学结构的同时，实现了更接近nn的更高表达性。第一个主要目标是提出和分析基于组合函数网络（COFNETs）的新逼近工具族，这些网络是树状结构的函数组合。这些新工具位于树形tn和神经网络之间，将结合两者的优点，即(i)可靠学习的有益数学结构，以及（ii）在许多应用中类似于神经网络的性能，包括动态系统的近似。我们的目的是从近似和统计学习的角度分析COFNETs的基本性质。此外，我们的目标是为这些新工具开发鲁棒和高效的算法，包括压缩技术和具有有限数据和低计算资源的自适应学习程序。我们预计这些结果将对基于网络的学习方法的发展产生重大影响，特别是树神经网络（COFNETs的一个特殊案例）以及深度神经网络。第二个主要目标是关注正向和逆UQ中的挑战性问题。重点是将高维随机微分方程解释为随机微分方程（SDEs）解的函数。这些自然呈现出一种组合结构。因此，COFNETs有望达到与UQ中最先进的方法相似的性能，但也允许解决其解决方案在通常意义上不具有任何规律性的新类别问题。COFNETs还应使sde的功能方法变得易于处理，并提供一种有效的替代方法，例如蒙特卡罗方法。对于贝叶斯反问题，相互作用的粒子解释允许从动力系统的稳态确定后验分布。我们的目标是分析COFNETs的近似，并将开发有效的重建算法。在最优的运输设置中，我们将考虑使用COFNETS构建运输地图，以缓解各种高维问题的维度诅咒。除了在昆士兰大学的应用之外，项目的理论和实践成果应该适用于广泛的问题，并为一个有前途的新研究方向打开大门，希望对几个科学领域有益。

High-dimensional approximation tasks are ubiquitous in all areas of scientific computing and data science, including the solution of partial differential equations (PDEs), machine learning and uncertainty quantification (UQ). With the recent success of deep Neural Networks (NNs) and tree Tensor Networks (TNs), previously intractable problems have become feasible and the demand for reliable algorithms and a better understanding of theoretical aspects is greater than ever. This success caused a stark interest in the analysis of compositions of functions and motivates the introduction of new approximation tools preserving convenient mathematical structures of TNs while achieving a higher expressivity closer to NNs.The first main objective is to propose and analyze new families of approximation tools based on compositional functions networks (coined COFNETs) that are tree-structured compositions of functions. These new tools lie in between tree TNs and NNs and will combine the best of both worlds, namely (i) a beneficial mathematical structure for reliable learning, and (ii) a performance similar to NNs for many applications, including the approximations of dynamical systems. We aim to analyze the fundamental properties of COFNETs from approximation and statistical learning perspectives. Also, we aim at developing robust and efficient algorithms for these new tools, including compression techniques and adaptive learning procedures with limited data and low computational resources.We expect the results to have a major impact on the development of network-based learning methods, in particular tree TNs (a particular case of COFNETs) but also deep NNs.The second main objective concerns challenging problems in forward and inverse UQ. The focus lies on high-dimensional random PDEs interpreted as functions of the solutions of stochastic differential equations (SDEs). These naturally exhibit a compositional structure. Hence, COFNETs are expected to achieve a similar performance as state-of-the-art methods in UQ but also allow to address new classes of problems whose solutions do not possess any regularity in a usual sense. COFNETs should also make functional approaches to SDEs become tractable and provide an efficient alternative e.g. to Monte-Carlo methods. For Bayesian inverse problems, an interacting particle interpretation allows to determine the posterior distribution from the steady state of a dynamical system. We aim to analyse the approximation by COFNETs and will develop efficient reconstruction algorithms. In an optimal transport setting, we will consider the construction of transport maps with COFNETS, alleviating the curse of dimensionality for a wide range of high-dimensional problems.Beyond applications in UQ, the theoretical and practical outcomes of the project should be applicable to a wide range of problems and open the door for a promising new research direction which hopefully becomes beneficial for several scientific fields.