Complexity of Neural Networks for Applications

Kon 9720145研究了神经网络结构和小波技术的应用，以研究神经网络的复杂性。小波作为一种丰富而有用的展开函数族已经被建立起来。研究者进一步研究了从函数的小波变换中恢复函数(特别是视觉图像的表示)的问题。研究了稳定性和复杂性问题，这些问题在他对Marr猜想的证明和对Mallat猜想的相关分析中还没有得到解决。当前的一个重要问题涉及完成预期任务所需的这类网络的复杂性(即其基本规模)。研究者研究了两种类型的网络，即所谓的功能网络和逻辑网络。功能网络受到了很多关注，一个连贯的理论已经确立了它们本质上是正交的(或更一般的)扩展引擎。网络结构和扩展任务之间的同源性使得小波收敛结果与网络复杂性问题建立了联系。研究人员将所谓的逻辑网络作为执行智能任务的可用网络体系结构的必要补充进行检查。此外，他还证明，对于给定的任务，基于小波的神经网络实现了神经网络复杂性的下界。这些结果趋向于神经网络的一般复杂性理论，其量级相当于当前的串并行计算机体系结构的计算复杂性理论。这样的复杂性理论有望成为当前离散和连续计算复杂性理论的混合体。这个项目的总体目的是对神经网络结构进行数学研究，实现研究人员获得的一些理论上的复杂性结果。神经网络作为并行分布式计算的模型，目前是人工模拟智能系统的主要体系结构，其许多当前应用(包括抵押贷款决策、商业股票市场分析应用、化学和热稳态控制系统、卫星图像分析等)都表明了这一点。在这种系统的开发中，一个尚未回答的主要问题是一个网络需要多大才能执行特定的智能功能这一根本问题。目前所谓的“功能性”神经体系结构难以处理的一种任务是人工视觉识别以及机器人一般领域中涉及的相关任务。在目前的研究中，这一困难似乎是功能神经体系结构的固有部分，研究人员开发了涉及所谓的“逻辑”组件的体系结构，这些组件基本上起到算法引擎的作用。特别是，这种网络体系结构对于人工视觉任务是必要的，并且这种任务的原型在从事该项目的研究生的帮助下进行了计算模拟。小波目前被认为是表示神经网络中实现的输入输出函数类型的最有用的工具之一。通过将小波技术应用于网络构建，解决了一个更一般的问题，即完成真实世界任务的神经网络的复杂性和规模。具体地说，使用小波作为激励函数，函数神经网络可以达到其最佳性能。这里有一个更大的问题，关于小波技术是否是实现功能神经网络结构的最佳可能，这是研究人员做出的一个猜测并进行了调查。该项目的计算方面得到了霍华德大学和布林·莫尔学院的相关小组的帮助，霍华德小组涉及许多研究生。

Kon 9720145 The investigator studies neural network architectures and applications of wavelet techniques to investigate neural networks' complexity. Wavelets have been established as a rich and useful family of expansion functions. The investigator studies further the recovery of functions (in particular representations of visual images) from their wavelet transforms. Issues of stability and complexity, which have not up to now been addressed in his proof of the Marr conjecture and related analysis of the Mallat conjecture, are studied. An important current question regards the complexity of such networks (i.e., their essential size) for the completion of desired tasks. The investigator studies two types, so-called functional and logical networks. Functional networks have received a good deal of attention, and a coherent theory has established that they are essentially orthogonal (or more general) expansion engines. The homology between the structure of networks and expansion tasks has allowed establishing the connection of wavelet convergence results with network complexity issues. The investigator examines the class of so-called logical networks as a needed completion of available network architectures for the execution of intelligent tasks. In addition he works to show that wavelet-based neural networks achieve lower bounds on complexities of neural nets for given tasks. These results move toward a general complexity theory for neural nets on the order of current computational complexity theory for serial and parallel computer architectures. Such a complexity theory is expected to be a hybrid of current discrete and continuous computational complexity theories. The global purpose of this project is a mathematical study of neural network architectures that implement some of the theoretical complexity results that the investigator obtains. Neural networks as models of parallel distributed computing are currently the leading architectures holdi ng a promise of artificially emulating intelligent systems, as has been indicated in many of their current applications (including mortgage decisions, commercial stock market analysis applications, chemical and thermal homeostasis control systems, satellite image analysis, etc.). A major unanswered question in the development of such systems is the fundamental issue of how large a network needs to be in order to perform specific intelligent functions. One type of task that current so-called "functional" neural architectures have difficulty in dealing with is artificial visual recognition and related tasks involved in the general area of robotics. This difficulty seems to be an inherent part of the functional neural architectures under current study, and the investigator develops architectures involving so-called "logical" components, which act essentially as algorithmic engines. In particular such network architectures are necessary for artificial vision tasks, and prototypes of such tasks are simulated computationally with the aid of graduate students working on the project. Wavelets are currently considered to be one of the most useful tools for representing the types of input-output functions implemented in neural networks. A more general question regarding the complexity and size of neural networks accomplishing real-world tasks is addressed through application of wavelet techniques to network construction. In particular, functional neural networks may achieve their optimal performance using wavelets as activation functions. There is a larger question here regarding whether wavelet techniques are the best possible for the implementation of functional neural network architectures, which is a conjecture the investigator has made and investigates. The computational aspects of the project are aided by associated groups at Howard University and Bryn Mawr College, the Howard group involving a number of graduate students.