Information Theoretic Learning for Pattern Recognition and Signal Processing

9900394原理本研究的重点是开发和评估一类用于信息理论学习（ITL）的新算法。传统的学习设计通常是基于最小化平方误差或熵的度量，这在计算上很难处理。这个项目将尝试估计和使用一种新的熵的度量，基于Renyi的概念。该算法的吸引力在于，它可以很容易地与Parzen窗口估计器集成，从而产生几个实用的标准来适应通用映射器，无论是在无监督范式还是有监督范式下。如果这项研究取得成功，那么对研究和应用学习系统感兴趣的科学界将可以使用一种新颖的、相当通用的算法。本研究的自然目标是进一步发展ITL算法，研究其在学习中的各种重要问题中的应用。具体而言，该项目将研究熵估计器和互信息估计器的性质，降低算法计算复杂度的方法，将其扩展到时间信号，设置其参数并获得其可扩展性。它还将研究互信息优化的新距离措施，以及在模拟VLSI中实现“熵芯片”的可行性，该芯片将使用物理定律进行计算。研究小组将使用新开发的ITL类算法研究信息过滤、独立成分分析和盲源分离方面的问题。这些领域本身就很重要，并且在这项研究中有进一步发展的势头。但是，该团队还将使用通用的计算基础设施来估计熵和来自示例的互信息，以比较所提出的ITL算法与每个领域中最佳可用技术的性能。具体来说，(1)他们将扩展卷积混合的盲源分离的现有技术。他们将把新的学习算法应用于移动通信信道的同信道干扰和助听器的降噪。(2)他们将ITL应用于系统识别和动态建模，并将其与我们之前使用均方误差的结果进行比较。(3)稀疏表示问题对于理解大脑和设计智能人工系统至关重要。我们将研究如何从数据中学习稀疏表示，使用过完备基和ITL算法来实现独立成分分析。＊＊＊

9900394PrincipeThe focus of this research is the development and evaluation of a new class of algorithms for informationtheoretic learning (ITL). Conventional learning designs are usually based on an effort to minimize either square error or a measure of entropy which can be computationally difficult to handle. This project will attempt to estimate and use a new measure of entropy, based on concepts from Renyi. The appeal of the algorithm is that it can be easily integrated with a Parzen window estimator yielding several practical criteria to adapt universal mappers, either under unsupervised or supervised paradigms.If the research is successful a novel and quite general class of algorithms will be made available tothe scientific community interested in studying and applying learning systems. The natural goal of this research is to further develop the ITL algorithms, an study their application to a variety of important problems in learning. Specifically, the project will study the properties of the estimator for both entropy and mutual information, ways to decrease the computational complexity of the algorithm, extend it to time signals, set its parameter and access its scalability. It will also investigate new distance measures for mutual information optimization and the feasibility of imple-menting an "entropy chip" in analog VLSI which will use the laws of physics to do the computation.The research team/will be investigating issues in information filtering, independent component analysis and blind source separation using the newly developed ITL class of algorithms. These areas are important in their own right, and have a momentum that will be further advanced in this research. But the team will also use the common computational infra-structure of estimating entropy and mutual information from examples to compare the proposed ITL algorithms' performance with the best available techniques in each field. Specifically, (1) They will extend the present state-of-the-art in the blind source separation of convolutive mixtures. They will apply the newlearning algorithm to the co-channel interference in mobile communication channels, and noise reduction in hearing aids. (2) They will apply ITL to system identification and dynamic modeling and com-pare it to our previous results using the mean square error. (3) The problem of sparse representa-tions is crucial to understand the brain and design intelligent artificial systems. We will be researching how to learn sparse representation from data, using both overcomplete bases and the ITL algorithm to implement independent component analysis. ***