Deep Neural Networks for Solving Non-Markov Optimization Problems

机器学习（ML）方法作为一组非常有效地解决人工智能和数据科学中的大规模优化问题的工具，最近获得了相当大的关注。这些方法中存在的神经网络架构具有一些通用的近似属性，允许用户应用软件工具，对数据进行最小的预处理或根据问题的规格定制算法。然而，在大多数机器学习运行良好的情况下，数学分析还没有为基本问题提供令人满意的答案：为什么机器学习方法对解决这个问题如此有效？该项目的目的是研究如何将机器学习方法应用于求解非马尔可夫动态规划（DPs），并针对该领域的一些具体问题回答这个基本问题。研究生参与该项目的研究。研究人员分析了一种求解非马尔可夫差分的新方法，其中通过训练神经网络系统获得策略逼近函数。其主要思想类似于最近开发的解决高维后向随机微分方程（BSDEs）的方法，其中所谓的深度BSDE求解器学习高维输入的函数来近似马尔可夫DP的最优控制。这个项目的目标与其他工作不同，因为重点是非马尔可夫DPs和来自路径依赖的障碍。深入探讨的问题包括蒙特卡罗粒子方法生成的训练集所需的精度水平，以及隐式正则化在TensorFlow算法中所起的作用。该项目还考虑了更传统的理论概念，这些概念可能为该方法的一般有效性提供证明，例如连续函数的Kolmogorov-Arnold表示和神经网络中使用的s型函数类型。当应用于具有非线性滤波的dp时，这些结果有助于提高对Deep BSDE背后数学的理论理解，并有助于回答有关该方法的重要问题。年代的有效性。研究生参与该项目的研究。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Machine learning (ML) methods have recently gained considerable attention as a set of tools that are very effective for solving large-scale optimization problems in artificial intelligence and data science. The neural network architecture that is present in many of these methods has some universal approximation properties that allow users to apply software tools with minimal preprocessing of data or tailoring of algorithms to the specifications of the problem. However, in most instances where ML works well, mathematical analysis does not yet offer a satisfactory answer to the fundamental question: Why is a machine learning method so effective for solving this problem? The aim of this project is to investigate how ML methods can be applied to solving non-Markov dynamic programs (DPs), and to answer this fundamental question for some specific problems in this area. Graduate students participate in the research of the project.The investigator analyzes a new method for solving non-Markov DPs, wherein a policy-approximation function is obtained by training a system of neural networks. The main idea is similar to recently-developed methods for solving high-dimensional backward stochastic differential equations (BSDEs), wherein the so-called Deep BSDE Solver learns a function of a high-dimensional input to approximate the optimal control for a Markovian DP. The aims of this project differ from those of other work because the focus is non-Markov DPs and the hurdles that come from path dependence. Issues that are explored in depth include the level of accuracy needed in the training set generated by a Monte Carlo particle method, and the role that implicit regularization plays in the TensorFlow algorithms. The project also considers more conventional theoretical concepts that may provide proof of this method's general effectiveness, such as the Kolmogorov-Arnold representation for continuous functions and the types of sigmoidal functions used in neural networks. The results contribute to an improved theoretical understanding of the mathematics behind Deep BSDE when applied to DPs with nonlinear filtering, and help answer important questions regarding the method?s effectiveness. Graduate students participate in the research of the project.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.