Mathematical Sciences: Approximation, Estimation, and Computation Properties of Neural Networks and Related Parsimonious Models

题目：神经网络的近似、估计和计算性质及相关的简约模型摘要：人工神经网络及相关的函数近似和估计的简约模型是近年来科学和工程领域关注的热点。作者的工作揭示了这些方法的几个有趣的方面。用经验过程概率论的方法得到了近似界，包括神经网络的平均平方误差和最大误差的界以及相关的近似。这些近似界揭示了对某些非参数（无限维）函数类的输入空间维数不敏感的收敛速度，通过有限维函数族的凸包的闭包来指定。因此，对这些非参数类中的函数进行精确的统计估计是可能的，而不需要借助指数级大的样本量。不幸的是，神经网络估计的计算可能是一项极其困难的任务。研究人员研究精确的近似、估计和计算问题是如何交织在一起的。在这项研究中，他们研究了基本的数学、统计和计算极限的能力，以近似和估计这些函数准确地计算可行的算法。在各种科学和工程任务中使用的经验建模技术处理的问题是如何结合大量的可观察量来最好地预测或近似一个响应变量。输入-响应关系可以用一个相当复杂的函数来描述，并且可能需要用少量基本的、相对简单的函数的组合来近似它。这些模型与经典的近似和统计估计技术不同，组合的函数不是预先固定的，而是根据已知或观察到的预期响应变量进行选择和调整，以提供最佳拟合。研究人员正在量化这些可调节选择的数学和统计优势。人工神经网络和相关技术是现代自适应和高性能计算模型的核心。研究人员研究了这些模型在计算上可行的极限。普遍需要准确的预测和经验建模，以便使用科学方法，特别是国家战略主题，这是这项研究的激励因素。

Proposals: DMS 9505168 PI: Andrew Barron Ilstitution: Yale University Title: APPROXIMATION, ESTIMATION, AND COMPUTATION PROPERTIES OF NEURAL NETWORKS AND RELATED PARSIMONIOUS MODELS Abstract: Artificial neural networks and related parsimonious models for function approximation and estimation have attracted recent attention in science and engineering. Work by the authors has uncovered several interesting aspects of these methods. Approximation bounds have been obtained by methods taken from the probability theory of empirical processes, including bounds on the average squared error and the maximal error of neural network and related approximations. These approximation bounds reveal a rate of convergence that is insensitive to the dimension of the input space for certain nonparametric (infinite dimensional) classes of functions, specified via the closure of convex hulls of finite dimensional families of functions. As a consequence accurate statistical estimation of functions in these nonparametric classes is possible without recourse to exponentially large sample sizes. Unfortunately, computation of neural net estimates can be an extremely difficult task. The investigators study how the problems of accurate approximation, estimation, and computation are intertwined. In this research they investigate fundamental mathematical, statistical, and computational limits of the capacity to approximate and to estimate these functions accurately by computationally feasible algorithms. Empirical modeling techniques used in a variety of scientific and engineering tasks deal with the problem of how to combine a large number of observable quantities to best predict or approximate a response variable. The input - response relation may be described by a rather complicated function, and it may be desirable to approximate it by a combination of a small number of elementary, comparatively simpler, functions. These models dif fer from classical techniques in approximation and statistical estimation in that the functions that are combined are not fixed in advance, but rather selected and adjusted according to what is known or observed concerning the intended response variable so as to provide the best fit. The investigators are quantifying the mathematical and statistical advantages of these adjustable selections. Artificial neural networks and related techniques are at the heart of modern models for adaptive and high performance computation. The investigators study the limits of what is computationally feasible with these models. The ubiquity of requirements for accurate prediction and empirical modeling for use of the scientific method in general and for nationally strategic topics in particular are motivating factors in this research.