A Theory of Learning Based on Pairwise Interactions

本计画将探讨一种新颖的方法，利用资讯熵来建立小样本情形下的学习演算法。它将寻求对新方法及其与均方误差的关系的更深入的数学理解，今天仍然是学习的主力。PI将研究由信息粒子创建的信息场的属性，以及如何使用它来提高泛化能力和构建系统误差的范围。他将从统计学提供的静态场出发，将时间纳入信息力中，以模拟扩散场和振荡场的相互作用。这将打开适应时空系统的大门。他还将研究熵参数估计的鲁棒性，并与SVM进行性能比较。但是，新颖的成对交互学习模型的最大吸引力在于，它为理解、应用和实现自适应系统提供了全新的机会。他将研究成对交互模型的以下应用：1-将树突树中的信息处理建模为估计熵的时空交互场。寻求在线算法，并希望显示它们与Hebbian的关系。我们的初步结果非常令人鼓舞。亨利马克拉姆博士，在动态突触世界专家将帮助建立模型的生物相容性。2-制定状态空间模型的线性和非线性系统与我们的熵估计。这一工作将扩展非高斯过程的最小熵控制领域，并将提供一种新的方法来实现鲁棒控制。3-将熵估计器应用于无线信道的盲解卷积以及多用户检测。PI将扩展他目前在卷积混合盲源分离方面的工作，这是更现实（但也更困难）的情况。特别是他将目标锁定在电话会议中的鸡尾酒会效应。

This project will investigate a novel approach to model and design learning algorithms in the small sample case using entropy. It will seek a deeper mathematical understanding of the new method and its relations to the mean square error, still today the workhorse of learning. The PI will investigate the properties of the information field created by the information particles, and how to use it to improve generalization and construct bounds on system errors. He will depart from the static fields provided by statistics and incorporate time in the information forces to mimic the interactions in diffusion fields and oscillating fields. This will open the door to adapt systems in space-time. He will also study the robustness of entropy for parameter estimation and compare performance with SVMs. But the large appeal of the novel pair wise interaction model for learning is that it opens drastically new opportunities to understand, apply and implement adaptive systems. He will be studying the following applications of the pair wise interaction model:1- Model the information processing in the dendritic tree as an spatio-temporal interaction field that estimates entropy. Seek on-line algorithms and hopefully show their relation to Hebbian. Our preliminary results are very encouraging. Dr. Henry Markram, a world expert in dynamic synapses will help establish the biological plausibility of the model.2- Formulate state space models of linear and nonlinear systems with our entropy estimator. This work will extend the field of minimum entropy control for non-Gaussian processes and will pro-vide a new approach to implement robust control.3- Apply the entropy estimator to blind deconvolution of wireless channels as well as multiuser detection. The PI will extend his current work on blind source separation for convolutive mixtures, the more realistic (but also harder) case. In particular he will target the cocktail party effect in teleconferencing.