Learning Complex Stochastic Systems

微分方程经常被用来模拟各种系统的时间演化。然而，大多数现实系统，包括那些来自生物学、环境科学、工程学、物理学、医学和金融市场的系统，其行为表现出随机性。因此，对这类系统的准确分析需要包含这种随机性的微分方程式。随机微分方程是实现这一目的的有力工具。理解这些系统的行为不仅需要建立数学模型，还需要将它们与现有数据相结合。这反过来需要各种类型的学习算法。通过严格的数学分析来判断这些算法的有效性是很重要的，这也是本项目的主要目标。然而，这些随机系统的动力学是错综复杂的相关结构，并且在研究这种复杂数据的学习方法的文献中严重缺乏数学结果。研究人员所做的工作将通过得出数学结果来填补这一空白，这些结果不仅能够回答算法在较长时间段内观察到的数据变得更准确的问题，而且能够提供关于如何微调关键参数以实现最佳效率的宝贵见解。在严格数学的支持下，构建这样的数据驱动的随机模型增强了我们对跨多个领域的复杂系统的理解，并使我们能够在存在随机性的情况下做出明智的决策。该项目将涉及本科生和研究生，并将通过理论知识、实际应用和编程实践经验相结合的方式向他们传授宝贵的技能。这将使他们能够在数字时代脱颖而出，并适应日益由数据驱动和技术进步的世界的需求。该项目的成果将通过在著名科学期刊上的出版物和在国内外会议上的演讲来传播。该项目将研究广泛类别的随机微分方程式(SDE)的重要学习问题。这些问题存在于随机分析和统计学习理论的交界处，在概率统计和机器学习文献中缺乏解决这些问题的理论结果。该项目分为三个相互关联的部分，每个部分都扮演着重要的角色。第一部分将讨论参数推断的重要问题，包括点估计和假设检验。它将得到一类广义随机微分方程的有限维参数估计的渐近结果，包括大数定律、中心极限定理和大偏差原理。与这一方向的一些现有工作不同的是，假设数据是连续轨迹的形式，调查者的工作将考虑只有离散数据点的现实情况。由于渐近分析要求时间范围达到无穷大，观测之间的时间间隔(或离散化步长)对这些估计器在长时间内的精度的影响尚不清楚，而且众所周知，基于SDE的连续轨迹对估计器进行幼稚的离散化会导致错误的推断。该项目将采用适当的比例调整框架来量化这一影响，并分析不同比例调整制度下的误差。下一步，这些结果将被用来设计复合假设检验问题的检验，使得第I类错误的概率迅速衰减，并且在具有类似第I类错误的一类检验中是渐近一致强大的。该项目的第二部分涉及决策这一重要专题。决策涉及根据模型参数对适当的成本函数进行(有约束的)最小化。由于后两个量是未知的，因此在实践中需要数据驱动版本的这种最小化问题。特别是，有必要构造适当的代价函数估计器，以使基于其最小化的决策接近于真实决策。调查者将根据大偏差分析和第一部分的结果研究一种新的方法，目的是确保在适当的条件下，这可以以非常高的概率实现。第三部分是关于SDE的非参数学习。最后一部分属于无限维学习理论领域，其目标是学习基于SDE的模型的全部驱动函数，而不是估计有限维参数。为此，将开发一个结合贝叶斯技术和再生核希尔伯特空间理论的严格计算框架，并研究由此产生的学习算法的理论性质。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Differential equations are often used to model temporal evolutions of a variety of systems. However, most realistic systems including those arising from biology, environmental science, engineering, physics, medicine and financial markets exhibit randomness in their behavior. Accurate analysis of such systems thus needs differential equations that can incorporate this randomness. Stochastic differential equations are powerful tools for this purpose. Understanding behaviors of these systems requires not just building mathematical models but integrating them with available data. This in turn requires various types of learning algorithms. It is important to judge the effectiveness of these algorithms by rigorous mathematical analysis, which is the primary objective of this project. The dynamics of these stochastic systems are however intricate with convoluted correlation structures, and there is a critical lack of mathematical results in the literature investigating learning methods for such complex data. The work done by the investigator will fill some of this gap by deriving mathematical results that will not only be able to answer if the algorithms become more accurate with data observed over longer periods of time but will be able to provide valuable insight on how to fine-tune the key parameters for optimal efficiency. Building such data-driven stochastic models backed by rigorous mathematics enhances our understanding of complex systems across multiple domains and empowers informed decision-making in the presence of randomness. The project will involve undergraduate and graduate students and will teach them valuable skills through a combination of theoretical knowledge, practical application, and hands-on experience with coding. It will enable them to excel in the digital age and adapt to the demands of an increasingly data-driven and technologically advanced world. The results of the project will be disseminated through publications in well-known scientific journals and presentations at domestic and international conferences.The project will study important learning problems for a broad class of stochastic differential equations (SDEs). These problems lie on the interface of stochastic analysis and statistical learning theory, and there is a paucity of theoretical results in probability, statistics and machine learning literature addressing them. The project is divided into three interconnected parts, each of which plays an important role in the other. Part I will address important problems on parametric inference including point estimation and testing of hypotheses. It will derive asymptotic results including law of large numbers, central limit theorems and large deviation principles for estimators of a finite dimensional parameter of a broad class of SDEs. Unlike some existing works in this direction which assume data to be in the form of a continuous trajectory, the investigator's work will consider the realistic case of availability of only discrete data points. Since asymptotic analysis requires the time horizon to go to infinity, the effect of time-gap (or discretization step) between the observations on the accuracy of these estimators over long time is not clear, and it is known that naive discretization of estimators based on a continuous trajectory of an SDE can lead to erroneous inference. The project will introduce appropriate scaling frameworks to quantify this effect and analyze the errors in different scaling regimes. Next, these results will be utilized to design tests for composite hypotheses-testing problems so that the probability of type I error decays rapidly and which are asymptotically uniformly powerful within a class of tests having similar level of type I error. Part II of the project concerns itself with the important topic of decision-making. Decision-making involves (constrained) minimization of suitable cost functions depending on model parameters. Since these latter quantities are unknown, data-driven versions of such minimization problems are necessary in practice. In particular, it is necessary to construct suitable estimators of the cost functions so that decisions based on their minimization are close to the true decisions. The investigator will study a novel approach based on large deviation analysis and results of Part I which aims to guarantee that under appropriate conditions this can be achieved with a very high probability. Part III is devoted to nonparametric learning of SDEs. The last part falls in the realm of infinite-dimensional learning theory where the goal is to learn the entire driving functions of the SDE-based models as opposed to estimating finite-dimensional parameters. A rigorous computational framework combining Bayesian techniques with the theory of Reproducing Kernel Hilbert Space will be developed toward this end, and the theoretical properties of the resulting learning algorithms will also be studied.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.