Deep Neural Networks for Solving Non-Markov Optimization Problems

机器学习（ML）方法最近获得了相当大的关注，作为一组工具，对于解决人工智能和数据科学中的大规模优化问题非常有效。 这些方法中的许多方法中存在的神经网络架构具有一些通用的近似属性，允许用户应用软件工具，只需对数据进行最少的预处理或根据问题的规格定制算法。 然而，在大多数ML工作良好的情况下，数学分析还没有为基本问题提供令人满意的答案：为什么机器学习方法在解决这个问题方面如此有效？ 该项目的目的是研究如何将ML方法应用于解决非马尔可夫动态规划（DP），并回答该领域中一些特定问题的基本问题。 研究生参与了该项目的研究。研究人员分析了一种新的方法来解决非马尔可夫DP，其中的政策近似函数是通过训练一个系统的神经网络。 其主要思想类似于最近开发的用于求解高维倒向随机微分方程（BSDEs）的方法，其中所谓的深度BSDEs求解器学习高维输入的函数以近似马尔可夫DP的最优控制。 这个项目的目标不同于其他工作，因为重点是非马尔可夫DP和来自路径依赖的障碍。 深入探讨的问题包括Monte Carlo粒子方法生成的训练集所需的准确度水平，以及隐式正则化在TensorFlow算法中的作用。 该项目还考虑了更传统的理论概念，这些概念可以证明这种方法的普遍有效性，例如连续函数的Kolmogorov-Arnold表示和神经网络中使用的S形函数类型。 当应用于具有非线性滤波的DP时，结果有助于提高对Deep BSDE背后数学的理论理解，并帮助回答有关该方法的重要问题？的有效性。 研究生参与该项目的研究。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Machine learning (ML) methods have recently gained considerable attention as a set of tools that are very effective for solving large-scale optimization problems in artificial intelligence and data science. The neural network architecture that is present in many of these methods has some universal approximation properties that allow users to apply software tools with minimal preprocessing of data or tailoring of algorithms to the specifications of the problem. However, in most instances where ML works well, mathematical analysis does not yet offer a satisfactory answer to the fundamental question: Why is a machine learning method so effective for solving this problem? The aim of this project is to investigate how ML methods can be applied to solving non-Markov dynamic programs (DPs), and to answer this fundamental question for some specific problems in this area. Graduate students participate in the research of the project.The investigator analyzes a new method for solving non-Markov DPs, wherein a policy-approximation function is obtained by training a system of neural networks. The main idea is similar to recently-developed methods for solving high-dimensional backward stochastic differential equations (BSDEs), wherein the so-called Deep BSDE Solver learns a function of a high-dimensional input to approximate the optimal control for a Markovian DP. The aims of this project differ from those of other work because the focus is non-Markov DPs and the hurdles that come from path dependence. Issues that are explored in depth include the level of accuracy needed in the training set generated by a Monte Carlo particle method, and the role that implicit regularization plays in the TensorFlow algorithms. The project also considers more conventional theoretical concepts that may provide proof of this method's general effectiveness, such as the Kolmogorov-Arnold representation for continuous functions and the types of sigmoidal functions used in neural networks. The results contribute to an improved theoretical understanding of the mathematics behind Deep BSDE when applied to DPs with nonlinear filtering, and help answer important questions regarding the method?s effectiveness. Graduate students participate in the research of the project.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.