Information Theoretic Learning for Pattern Recognition and Signal Processing

模式识别和信号处理的信息论学习

• 批准号: 9900394
• 负责人: Jose Principe
• 金额: $ 21.96万
• 依托单位: University of Florida
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 1999
• 资助国家: 美国
• 起止时间: 1999-05-01 至 2003-04-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=9900394&HistoricalAwards=false
• 关键词: Information; Theoretic; Learning; Pattern; Recognition

9900394PrincipeThe focus of this research is the development and evaluation of a new class of algorithms for informationtheoretic learning (ITL). Conventional learning designs are usually based on an effort to minimize either square error or a measure of entropy which can be computationally difficult to handle. This project will attempt to estimate and use a new measure of entropy, based on concepts from Renyi. The appeal of the algorithm is that it can be easily integrated with a Parzen window estimator yielding several practical criteria to adapt universal mappers, either under unsupervised or supervised paradigms.If the research is successful a novel and quite general class of algorithms will be made available tothe scientific community interested in studying and applying learning systems. The natural goal of this research is to further develop the ITL algorithms, an study their application to a variety of important problems in learning. Specifically, the project will study the properties of the estimator for both entropy and mutual information, ways to decrease the computational complexity of the algorithm, extend it to time signals, set its parameter and access its scalability. It will also investigate new distance measures for mutual information optimization and the feasibility of imple-menting an "entropy chip" in analog VLSI which will use the laws of physics to do the computation.The research team/will be investigating issues in information filtering, independent component analysis and blind source separation using the newly developed ITL class of algorithms. These areas are important in their own right, and have a momentum that will be further advanced in this research. But the team will also use the common computational infra-structure of estimating entropy and mutual information from examples to compare the proposed ITL algorithms' performance with the best available techniques in each field. Specifically, (1) They will extend the present state-of-the-art in the blind source separation of convolutive mixtures. They will apply the newlearning algorithm to the co-channel interference in mobile communication channels, and noise reduction in hearing aids. (2) They will apply ITL to system identification and dynamic modeling and com-pare it to our previous results using the mean square error. (3) The problem of sparse representa-tions is crucial to understand the brain and design intelligent artificial systems. We will be researching how to learn sparse representation from data, using both overcomplete bases and the ITL algorithm to implement independent component analysis. ***

9900394原则这项研究的重点是信息理论学习（ITL）的一类新的算法的开发和评估。传统的学习设计通常基于最小化平方误差或熵的测量的努力，这在计算上可能难以处理。 这个项目将尝试估计和使用一种新的熵的测量方法，基于Renyi的概念。 该算法的吸引力在于，它可以很容易地与Parzen窗口估计器集成，从而产生几个实用的标准来适应通用映射器，无论是在无监督的还是有监督的范式下，如果研究成功，一种新颖的和相当通用的算法将被提供给对学习系统研究和应用感兴趣的科学界。本研究的自然目标是进一步发展ITL算法，研究其在学习中的各种重要问题的应用。具体而言，该项目将研究熵和互信息估计器的性质，降低算法计算复杂度的方法，将其扩展到时间信号，设置其参数并访问其可扩展性。 研究小组亦会研究新的互信息最佳化的距离量度，以及在模拟超大规模集成电路中应用物理定律进行计算的“熵芯片”的可行性。研究小组会研究使用新发展的ITL类算法进行信息过滤、独立成分分析及盲源分离等问题。这些领域本身就很重要，并将在本研究中进一步推进。但该小组还将使用从实例中估计熵和互信息的通用计算基础结构，将拟议的国际交易日志算法的性能与每个领域的最佳现有技术进行比较。 具体而言，（1）它们将扩展卷积混合盲源分离的现有技术。 他们将把新的学习算法应用于移动的通信信道中的同信道干扰和助听器中的降噪。(2)他们将ITL应用于系统辨识和动态建模，并使用均方误差将其与我们以前的结果进行比较。(3)稀疏表示问题是理解大脑和设计智能人工系统的关键。我们将研究如何从数据中学习稀疏表示，使用过完备基和ITL算法来实现独立分量分析。***