Mathematical Sciences: Approximation, Estimation, and Computation Properties of Neural Networks and Related Parsimonious Models

提案：DMS 9505199 PI：Lee Jones和Yuly Makovoz机构：马萨诸塞州洛厄尔大学标题： 近似、估计和变换性质 神经网络及其相关的简约模型 摘要： 人工神经网络与函数的简约模型 近似和估计最近在科学上引起了注意 与工程学 作者的工作揭示了几个有趣的 这些方法的各个方面。 近似界已获得由 方法取自经验过程的概率论， 包括平均平方误差和最大误差的界限， 神经网络和相关的近似。 这些近似边界 揭示了一个收敛的速度，是不敏感的维度， 某些非参数（无限维）类的输入空间 函数，通过有限维函数族的凸包的闭包指定。 因此，准确的统计估计 在这些非参数类中的函数是可能的， 到指数级的大样本量。 不幸的是，神经计算 净估计数可能是一项极其困难的任务。 研究人员研究 精确近似、估计和计算的问题是如何 交织在一起 在这项研究中，他们研究了基础数学， 统计和计算能力的极限， 通过计算上可行的方法来准确地估计这些函数， 算法 在各种科学和工程任务中使用的经验建模技术处理如何将大量可观测量联合收割机组合以最佳地预测或近似的问题 响应变量。 输入-响应关系可以被描述为 通过一个相当复杂的功能，它可能是可取的， 用少量的 基本的，比较简单的功能。这些模型 与近似和统计估计中的经典技术的不同之处在于， 不是事先固定的，而是根据情况选择和调整的。 关于预期响应变量的已知或观察 以便提供最佳配合。 调查人员正在量化 这些可调整的选择的数学和统计优势。 人工神经网络和相关技术是用于自适应和高性能计算的现代模型的核心。 研究人员研究了计算的极限， 这些模型都是可行的。 普遍存在的准确预测和经验建模的要求，使用科学方法，特别是国家战略主题是本研究的激励因素。

Proposals: DMS 9505199 PIs: Lee Jones and Yuly Makovoz Institution: University of Massachusetts at Lowell Title: APPROXIMATION, ESTIMATION, AND COMPUTATION PROPERTIES OF NEURAL NETWORKS AND RELATED PARSIMONIOUS MODELS Abstract: Artificial neural networks and related parsimonious models for function approximation and estimation have attracted recent attention in science and engineering. Work by the authors has uncovered several interesting aspects of these methods. Approximation bounds have been obtained by methods taken from the probability theory of empirical processes, including bounds on the average squared error and the maximal error of neural network and related approximations. These approximation bounds reveal a rate of convergence that is insensitive to the dimension of the input space for certain nonparametric (infinite dimensional) classes of functions, specified via the closure of convex hulls of finite dimensional families of functions. As a consequence accurate statistical estimation of functions in these nonparametric classes is possible without recourse to exponentially large sample sizes. Unfortunately, computation of neural net estimates can be an extremely difficult task. The investigators study how the problems of accurate approximation, estimation, and computation are intertwined. In this research they investigate fundamental mathematical, statistical, and computational limits of the capacity to approximate and to estimate these functions accurately by computationally feasible algorithms. Empirical modeling techniques used in a variety of scientific and engineering tasks deal with the problem of how to combine a large number of observable quantities to best predict or approximate a response variable. The input - response relation may be described by a rather complicated function, and it may be desirable to approximate it by a combination of a small number of elementary, comparatively s impler, functions. These models differ from classical techniques in approximation and statistical estimation in that the functions that are combined are not fixed in advance, but rather selected and adjusted according to what is known or observed concerning the intended response variable so as to provide the best fit. The investigators are quantifying the mathematical and statistical advantages of these adjustable selections. Artificial neural networks and related techniques are at the heart of modern models for adaptive and high performance computation. The investigators study the limits of what is computationally feasible with these models. The ubiquity of requirements for accurate prediction and empirical modeling for use of the scientific method in general and for nationally strategic topics in particular are motivating factors in this research.