Mathematical Sciences: Approximation, Estimation, and Computation Properties of Neural Networks and Related Parsimonious Models

建议：DMS 9505168 PI：安德鲁·巴伦研究所：耶鲁大学标题：神经网络和相关简约模型的逼近、估计和计算特性摘要：用于函数逼近和估计的人工神经网络和相关简约模型最近在科学和工程领域引起了关注。作者的工作揭示了这些方法的几个有趣的方面。用经验过程概率论的方法得到了神经网络的逼近界，包括神经网络的平均平方误差和最大误差的界以及相关的逼近。这些逼近界揭示了某些非参数(无限维)函数类的收敛速度对输入空间的维度不敏感，这些非参数(无限维)函数类由有限维函数族的凸壳的闭包来指定。因此，对这些非参数类中的函数进行准确的统计估计是可能的，而无需求助于指数级的大样本大小。不幸的是，神经网络估计的计算可能是一项极其困难的任务。研究人员研究了精确近似、估计和计算问题是如何相互交织在一起的。在这项研究中，他们调查了通过计算上可行的算法来近似和准确估计这些函数的能力的基本数学、统计和计算限制。在各种科学和工程任务中使用的经验建模技术处理如何结合大量可观测量以最好地预测或近似响应变量的问题。输入-响应关系可以用相当复杂的函数来描述，并且可能希望通过少量基本的、相对简单的函数的组合来近似它。这些模型在逼近和统计估计方面与经典技术的不同之处在于，组合的函数不是预先固定的，而是根据关于预期响应变量的已知或观察到的选择和调整，以提供最佳拟合。研究人员正在量化这些可调选择在数学和统计学上的优势。人工神经网络和相关技术是现代自适应和高性能计算模型的核心。研究人员研究了这些模型在计算上可行的限度。普遍存在对科学方法的准确预测和经验建模的需求，特别是对国家战略主题的需求是这项研究的动机因素。

Proposals: DMS 9505168 PI: Andrew Barron Ilstitution: Yale University Title: APPROXIMATION, ESTIMATION, AND COMPUTATION PROPERTIES OF NEURAL NETWORKS AND RELATED PARSIMONIOUS MODELS Abstract: Artificial neural networks and related parsimonious models for function approximation and estimation have attracted recent attention in science and engineering. Work by the authors has uncovered several interesting aspects of these methods. Approximation bounds have been obtained by methods taken from the probability theory of empirical processes, including bounds on the average squared error and the maximal error of neural network and related approximations. These approximation bounds reveal a rate of convergence that is insensitive to the dimension of the input space for certain nonparametric (infinite dimensional) classes of functions, specified via the closure of convex hulls of finite dimensional families of functions. As a consequence accurate statistical estimation of functions in these nonparametric classes is possible without recourse to exponentially large sample sizes. Unfortunately, computation of neural net estimates can be an extremely difficult task. The investigators study how the problems of accurate approximation, estimation, and computation are intertwined. In this research they investigate fundamental mathematical, statistical, and computational limits of the capacity to approximate and to estimate these functions accurately by computationally feasible algorithms. Empirical modeling techniques used in a variety of scientific and engineering tasks deal with the problem of how to combine a large number of observable quantities to best predict or approximate a response variable. The input - response relation may be described by a rather complicated function, and it may be desirable to approximate it by a combination of a small number of elementary, comparatively simpler, functions. These models dif fer from classical techniques in approximation and statistical estimation in that the functions that are combined are not fixed in advance, but rather selected and adjusted according to what is known or observed concerning the intended response variable so as to provide the best fit. The investigators are quantifying the mathematical and statistical advantages of these adjustable selections. Artificial neural networks and related techniques are at the heart of modern models for adaptive and high performance computation. The investigators study the limits of what is computationally feasible with these models. The ubiquity of requirements for accurate prediction and empirical modeling for use of the scientific method in general and for nationally strategic topics in particular are motivating factors in this research.