EAGER: Theoretic Structures of High Dimensional Data Decomposition

本项目旨在从信息理论的角度研究降维和特征选择问题。特征选择问题是高维数据处理中的一个核心问题。从信息处理的角度来看，这是困难的，主要是因为当将高维数据降维到低维特征空间时，通常不可避免地会产生不可逆的信息损失。在这项工作中，这个问题被表述为一个一般的有损信息处理问题。这个问题的解决方案是有效的算法，可以用来选择与一系列推理任务普遍相关的信息特征。该问题的一般理论框架的目标是对现有的各种实际解决方案进行系统的理解和统一的性能比较。主要的技术优点在于信息度量的新的操作意义，它将大量的信息理论研究与高维数据分析的挑战联系起来。在这项工作中使用了一种新的几何分析方法，这有助于将特征选择问题可视化，并将问题与已经得到充分研究的Hirschfeld-Gebelein-Renyi最大相关概念联系起来。该方法的主要优点是其通用性。它可以应用于任何类型的数据，融合先验知识和侧信息，连接多个平台，遵循计算和存储约束，适应时变等，所有这些都基于相同的理论原理。可以设想，这种通用性将导致数据分析领域的架构变化，使用通用接口将数据科学家在信息提取方面的任务与具有领域知识的专家在收集数据、提供模型和解释结果方面的任务分离开来。

This project aims at developing an information theoretic view of the problems of dimension reduction and feature selection. The problem of feature selection is a core issue in processing high dimensional data. It is difficult from an information processing point-of-view mainly because when reducing high dimensional data into lower dimensional feature space, it is in general inevitable to incur irreversible information losses. In this work, the problem is formulated as a general lossy information processing problem. The solutions to this problem is efficient algorithms that can be used to choose informative features that are relevant universally to a family of inference tasks.The goal of a general theoretic framework to this problem is to develop systematic understanding and uniform performance comparisons to the existing wide variety of practical solutions. The main technical merit lies in a new operational meaning of information metrics, which connects a large body of research on information theory to the challenges of high dimensional data analytics. A new geometric analysis approach is used in this work, which helps to visualize the problem of feature selections, and link the problem to the well-studied concept of the Hirschfeld-Gebelein-Renyi maximal correlation.The key advantage of the proposed approach is its generality. It can be applied to any type of data, incorporate prior knowledge and side information, connect multiple platforms, follow computation and storage constraints, adapt to time-variations, etc., all based on the same theoretic principle. It is envisioned that such universality would lead to architectural changes in the area of data analysis, with a universal interface that separates the task of a data scientist, in information extraction, from the task of a specialist with domain knowledge, in collecting the data, providing the models, and interpreting the result.