Complexity of Neural Networks for Applications

研究者研究神经网络架构和小波技术的应用来研究神经网络的复杂性。小波是一类丰富而有用的展开函数。研究者进一步研究了从它们的小波变换中恢复函数（特别是视觉图像的表示）。他对马尔猜想的证明和马拉猜想的相关分析中尚未涉及的稳定性和复杂性问题进行了研究。当前的一个重要问题是，这些网络的复杂性（即它们的基本大小）是完成预期任务的必要条件。研究者研究了两种类型，即所谓的功能网络和逻辑网络。功能网络已经得到了大量的关注，并且一个连贯的理论已经确定它们本质上是正交的（或更一般的）扩展引擎。网络结构与扩展任务之间的同源性使得小波收敛结果与网络复杂性问题之间建立了联系。研究者将所谓的逻辑网络视为执行智能任务所需的可用网络架构的完成。此外，他还展示了基于小波的神经网络在给定任务中实现了神经网络复杂性的下界。这些结果在当前串行和并行计算机体系结构的计算复杂性理论的基础上，向神经网络的一般复杂性理论迈进。这种复杂性理论有望成为当前离散和连续计算复杂性理论的混合。该项目的总体目的是对神经网络架构进行数学研究，实现研究者获得的一些理论复杂性结果。神经网络作为并行分布式计算的模型，目前是人工模拟智能系统的领先架构，正如其当前的许多应用（包括抵押贷款决策、商业股票市场分析应用、化学和热稳态控制系统、卫星图像分析等）所表明的那样。在这类系统的发展中，一个悬而未决的主要问题是一个基本问题，即一个网络需要多大才能执行特定的智能功能。目前所谓的“功能性”神经结构难以处理的一类任务是人工视觉识别和涉及机器人一般领域的相关任务。这种困难似乎是当前研究中功能性神经架构的固有部分，研究者开发了涉及所谓“逻辑”组件的架构，这些组件本质上充当算法引擎。特别是这样的网络架构对于人工视觉任务是必要的，并且在研究生的帮助下对这些任务的原型进行了计算模拟。小波目前被认为是表示神经网络中实现的输入输出函数类型的最有用的工具之一。通过将小波技术应用于网络构建，解决了关于完成现实世界任务的神经网络的复杂性和大小的一个更普遍的问题。特别是，函数神经网络可以使用小波作为激活函数来实现其最佳性能。这里有一个更大的问题，关于小波技术是否是实现功能神经网络架构的最佳方法，这是研究者提出并研究的一个猜想。该项目的计算方面得到了霍华德大学和布林莫尔学院的相关小组的帮助，霍华德小组包括一些研究生。

Kon 9720145 The investigator studies neural network architectures and applications of wavelet techniques to investigate neural networks' complexity. Wavelets have been established as a rich and useful family of expansion functions. The investigator studies further the recovery of functions (in particular representations of visual images) from their wavelet transforms. Issues of stability and complexity, which have not up to now been addressed in his proof of the Marr conjecture and related analysis of the Mallat conjecture, are studied. An important current question regards the complexity of such networks (i.e., their essential size) for the completion of desired tasks. The investigator studies two types, so-called functional and logical networks. Functional networks have received a good deal of attention, and a coherent theory has established that they are essentially orthogonal (or more general) expansion engines. The homology between the structure of networks and expansion tasks has allowed establishing the connection of wavelet convergence results with network complexity issues. The investigator examines the class of so-called logical networks as a needed completion of available network architectures for the execution of intelligent tasks. In addition he works to show that wavelet-based neural networks achieve lower bounds on complexities of neural nets for given tasks. These results move toward a general complexity theory for neural nets on the order of current computational complexity theory for serial and parallel computer architectures. Such a complexity theory is expected to be a hybrid of current discrete and continuous computational complexity theories. The global purpose of this project is a mathematical study of neural network architectures that implement some of the theoretical complexity results that the investigator obtains. Neural networks as models of parallel distributed computing are currently the leading architectures holdi ng a promise of artificially emulating intelligent systems, as has been indicated in many of their current applications (including mortgage decisions, commercial stock market analysis applications, chemical and thermal homeostasis control systems, satellite image analysis, etc.). A major unanswered question in the development of such systems is the fundamental issue of how large a network needs to be in order to perform specific intelligent functions. One type of task that current so-called "functional" neural architectures have difficulty in dealing with is artificial visual recognition and related tasks involved in the general area of robotics. This difficulty seems to be an inherent part of the functional neural architectures under current study, and the investigator develops architectures involving so-called "logical" components, which act essentially as algorithmic engines. In particular such network architectures are necessary for artificial vision tasks, and prototypes of such tasks are simulated computationally with the aid of graduate students working on the project. Wavelets are currently considered to be one of the most useful tools for representing the types of input-output functions implemented in neural networks. A more general question regarding the complexity and size of neural networks accomplishing real-world tasks is addressed through application of wavelet techniques to network construction. In particular, functional neural networks may achieve their optimal performance using wavelets as activation functions. There is a larger question here regarding whether wavelet techniques are the best possible for the implementation of functional neural network architectures, which is a conjecture the investigator has made and investigates. The computational aspects of the project are aided by associated groups at Howard University and Bryn Mawr College, the Howard group involving a number of graduate students.