Complexity of Neural Networks for Applications

神经网络应用的复杂性

• 批准号: 9720145
• 负责人: Mark Kon
• 金额: $ 7.5万
• 依托单位: Trustees of Boston University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 1997
• 资助国家: 美国
• 起止时间: 1997-08-15 至 2002-03-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=9720145&HistoricalAwards=false
• 关键词: Complexity; Neural; Networks; Applications

Kon 9720145 The investigator studies neural network architectures and applications of wavelet techniques to investigate neural networks' complexity. Wavelets have been established as a rich and useful family of expansion functions. The investigator studies further the recovery of functions (in particular representations of visual images) from their wavelet transforms. Issues of stability and complexity, which have not up to now been addressed in his proof of the Marr conjecture and related analysis of the Mallat conjecture, are studied. An important current question regards the complexity of such networks (i.e., their essential size) for the completion of desired tasks. The investigator studies two types, so-called functional and logical networks. Functional networks have received a good deal of attention, and a coherent theory has established that they are essentially orthogonal (or more general) expansion engines. The homology between the structure of networks and expansion tasks has allowed establishing the connection of wavelet convergence results with network complexity issues. The investigator examines the class of so-called logical networks as a needed completion of available network architectures for the execution of intelligent tasks. In addition he works to show that wavelet-based neural networks achieve lower bounds on complexities of neural nets for given tasks. These results move toward a general complexity theory for neural nets on the order of current computational complexity theory for serial and parallel computer architectures. Such a complexity theory is expected to be a hybrid of current discrete and continuous computational complexity theories. The global purpose of this project is a mathematical study of neural network architectures that implement some of the theoretical complexity results that the investigator obtains. Neural networks as models of parallel distributed computing are currently the leading architectures holdi ng a promise of artificially emulating intelligent systems, as has been indicated in many of their current applications (including mortgage decisions, commercial stock market analysis applications, chemical and thermal homeostasis control systems, satellite image analysis, etc.). A major unanswered question in the development of such systems is the fundamental issue of how large a network needs to be in order to perform specific intelligent functions. One type of task that current so-called "functional" neural architectures have difficulty in dealing with is artificial visual recognition and related tasks involved in the general area of robotics. This difficulty seems to be an inherent part of the functional neural architectures under current study, and the investigator develops architectures involving so-called "logical" components, which act essentially as algorithmic engines. In particular such network architectures are necessary for artificial vision tasks, and prototypes of such tasks are simulated computationally with the aid of graduate students working on the project. Wavelets are currently considered to be one of the most useful tools for representing the types of input-output functions implemented in neural networks. A more general question regarding the complexity and size of neural networks accomplishing real-world tasks is addressed through application of wavelet techniques to network construction. In particular, functional neural networks may achieve their optimal performance using wavelets as activation functions. There is a larger question here regarding whether wavelet techniques are the best possible for the implementation of functional neural network architectures, which is a conjecture the investigator has made and investigates. The computational aspects of the project are aided by associated groups at Howard University and Bryn Mawr College, the Howard group involving a number of graduate students.

Kon 9720145 研究者研究神经网络结构和小波技术的应用，以研究神经网络的复杂性。 波函数是一个丰富而有用的展开函数族。 调查员进一步研究恢复功能（特别是视觉图像的表示）从他们的小波变换。 稳定性和复杂性的问题，这还没有解决到现在为止，在他的证明的马尔猜想和相关的分析的Mallat猜想，进行了研究。 当前的一个重要问题是关于这种网络的复杂性（即， 它们的基本尺寸）用于完成所期望的任务。 研究人员研究了两种类型，即所谓的功能网络和逻辑网络。 泛函网络已经受到了很大的关注，并且一个连贯的理论已经确定它们本质上是正交的（或更一般的）扩展引擎。 网络结构和扩展任务之间的同源性允许建立小波收敛结果与网络复杂性问题之间的联系。 调查员检查所谓的逻辑网络类作为执行智能任务所需的可用网络架构的完成。 此外，他的作品表明，基于小波的神经网络实现了较低的界限，神经网络的复杂性，为给定的任务。 这些结果走向一个通用的复杂性理论的神经网络的顺序目前的计算复杂性理论的串行和并行计算机体系结构。 这样的复杂性理论预计将是一个混合目前的离散和连续计算复杂性理论。 该项目的全球目的是对神经网络架构进行数学研究，以实现研究人员获得的一些理论复杂性结果。 作为并行分布式计算模型的神经网络是目前领先的体系结构，它有望人工仿真智能系统，正如其当前的许多应用（包括抵押贷款决策、商业股票市场分析应用、化学和热稳态控制系统、卫星图像分析等）所表明的那样。 在这种系统的开发中，一个主要的未回答的问题是一个基本问题，即为了执行特定的智能功能，需要多大的网络。 当前所谓的“功能”神经架构难以处理的一种类型的任务是人工视觉识别和涉及机器人一般领域的相关任务。 这种困难似乎是一个固有的一部分，根据目前的研究功能的神经架构，和研究人员开发的架构，涉及所谓的“逻辑”组件，其基本上作为算法引擎。 特别是这样的网络架构是必要的人工视觉任务，和原型的这些任务的计算模拟的帮助下，研究生工作的项目。 目前，Wavelet被认为是表示神经网络中实现的输入输出函数类型的最有用的工具之一。 一个更普遍的问题，关于完成现实世界的任务的神经网络的复杂性和规模，通过应用小波技术网络建设解决。 特别地，函数神经网络可以使用小波作为激活函数来实现其最佳性能。 这里有一个更大的问题，关于小波技术是否是实现功能神经网络架构的最佳可能，这是研究人员提出并研究的一个猜想。 该项目的计算方面得到了霍华德大学和布林莫尔学院相关小组的帮助，霍华德小组涉及一些研究生。