Learning Complex Stochastic Systems

微分方程经常被用来模拟各种系统的时间演化。然而，大多数现实系统，包括那些来自生物学、环境科学、工程学、物理学、医学和金融市场的系统，其行为表现出随机性。因此，对这类系统的精确分析需要包含这种随机性的微分方程。随机微分方程是实现这一目的的有力工具。理解这些系统的行为不仅需要建立数学模型，还需要将它们与可用数据集成。这反过来又需要各种类型的学习算法。通过严格的数学分析来判断这些算法的有效性是很重要的，这是本项目的主要目标。然而，这些随机系统的动力学是复杂的，具有复杂的相关结构，并且在研究这种复杂数据的学习方法的文献中严重缺乏数学结果。研究者所做的工作将通过推导数学结果来填补这一空白，这些数学结果不仅能够回答如果算法在更长的时间内观察到的数据变得更加准确，而且能够提供关于如何微调关键参数以达到最佳效率的有价值的见解。建立这种由严格数学支持的数据驱动的随机模型，增强了我们对跨多个领域的复杂系统的理解，并在随机性存在的情况下赋予了明智的决策能力。该项目将涉及本科生和研究生，并将通过理论知识、实际应用和编程实践经验相结合，教授他们宝贵的技能。这将使他们能够在数字时代脱颖而出，适应日益数据驱动和技术先进的世界的需求。该项目的成果将通过知名科学期刊的出版物和在国内和国际会议上的发言加以传播。该项目将研究一类广泛的随机微分方程（SDEs）的重要学习问题。这些问题是随机分析和统计学习理论的交叉点，在概率论、统计学和机器学习文献中缺乏解决这些问题的理论结果。该项目分为三个相互关联的部分，每个部分在另一个部分中发挥重要作用。第一部分将讨论参数推理的重要问题，包括点估计和假设检验。对一类广义SDEs的有限维参数估计量，给出了大数定律、中心极限定理和大偏差原理等渐近结果。不像在这个方向上的一些现有工作，假设数据是连续轨迹的形式，研究者的工作将考虑只有离散数据点可用性的现实情况。由于渐近分析要求时间范围趋近于无穷大，观测值之间的时间间隔（或离散化步长）在长时间内对这些估计器精度的影响尚不清楚，并且已知基于SDE连续轨迹的估计器的朴素离散化可能导致错误的推断。该项目将引入适当的缩放框架来量化这种影响，并分析不同缩放制度下的误差。接下来，这些结果将用于设计复合假设检验问题的检验，使I型错误的概率迅速衰减，并且在具有类似I型错误水平的一类检验中渐近一致强大。项目的第二部分关注的是决策这一重要主题。决策涉及（约束）最小化合适的成本函数取决于模型参数。由于后面这些量是未知的，因此在实践中需要数据驱动的最小化问题版本。特别是，有必要构造合适的成本函数估计器，以便基于最小化的决策接近真实决策。研究者将研究一种基于大偏差分析和第一部分结果的新方法，旨在保证在适当的条件下，这可以以非常高的概率实现。第三部分致力于SDEs的非参数学习。最后一部分属于无限维学习理论的领域，其目标是学习基于sde的模型的整个驱动函数，而不是估计有限维参数。为此，将开发一个将贝叶斯技术与再现核希尔伯特空间理论相结合的严格计算框架，并研究由此产生的学习算法的理论性质。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Differential equations are often used to model temporal evolutions of a variety of systems. However, most realistic systems including those arising from biology, environmental science, engineering, physics, medicine and financial markets exhibit randomness in their behavior. Accurate analysis of such systems thus needs differential equations that can incorporate this randomness. Stochastic differential equations are powerful tools for this purpose. Understanding behaviors of these systems requires not just building mathematical models but integrating them with available data. This in turn requires various types of learning algorithms. It is important to judge the effectiveness of these algorithms by rigorous mathematical analysis, which is the primary objective of this project. The dynamics of these stochastic systems are however intricate with convoluted correlation structures, and there is a critical lack of mathematical results in the literature investigating learning methods for such complex data. The work done by the investigator will fill some of this gap by deriving mathematical results that will not only be able to answer if the algorithms become more accurate with data observed over longer periods of time but will be able to provide valuable insight on how to fine-tune the key parameters for optimal efficiency. Building such data-driven stochastic models backed by rigorous mathematics enhances our understanding of complex systems across multiple domains and empowers informed decision-making in the presence of randomness. The project will involve undergraduate and graduate students and will teach them valuable skills through a combination of theoretical knowledge, practical application, and hands-on experience with coding. It will enable them to excel in the digital age and adapt to the demands of an increasingly data-driven and technologically advanced world. The results of the project will be disseminated through publications in well-known scientific journals and presentations at domestic and international conferences.The project will study important learning problems for a broad class of stochastic differential equations (SDEs). These problems lie on the interface of stochastic analysis and statistical learning theory, and there is a paucity of theoretical results in probability, statistics and machine learning literature addressing them. The project is divided into three interconnected parts, each of which plays an important role in the other. Part I will address important problems on parametric inference including point estimation and testing of hypotheses. It will derive asymptotic results including law of large numbers, central limit theorems and large deviation principles for estimators of a finite dimensional parameter of a broad class of SDEs. Unlike some existing works in this direction which assume data to be in the form of a continuous trajectory, the investigator's work will consider the realistic case of availability of only discrete data points. Since asymptotic analysis requires the time horizon to go to infinity, the effect of time-gap (or discretization step) between the observations on the accuracy of these estimators over long time is not clear, and it is known that naive discretization of estimators based on a continuous trajectory of an SDE can lead to erroneous inference. The project will introduce appropriate scaling frameworks to quantify this effect and analyze the errors in different scaling regimes. Next, these results will be utilized to design tests for composite hypotheses-testing problems so that the probability of type I error decays rapidly and which are asymptotically uniformly powerful within a class of tests having similar level of type I error. Part II of the project concerns itself with the important topic of decision-making. Decision-making involves (constrained) minimization of suitable cost functions depending on model parameters. Since these latter quantities are unknown, data-driven versions of such minimization problems are necessary in practice. In particular, it is necessary to construct suitable estimators of the cost functions so that decisions based on their minimization are close to the true decisions. The investigator will study a novel approach based on large deviation analysis and results of Part I which aims to guarantee that under appropriate conditions this can be achieved with a very high probability. Part III is devoted to nonparametric learning of SDEs. The last part falls in the realm of infinite-dimensional learning theory where the goal is to learn the entire driving functions of the SDE-based models as opposed to estimating finite-dimensional parameters. A rigorous computational framework combining Bayesian techniques with the theory of Reproducing Kernel Hilbert Space will be developed toward this end, and the theoretical properties of the resulting learning algorithms will also be studied.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.