EAGER: Theoretic Structures of High Dimensional Data Decomposition

本计画的目的是发展一个资讯论的观点来探讨降维与特徴选择的问题。特征选择问题是高维数据处理中的核心问题。从信息处理的观点来看，这是困难的，主要是因为当将高维数据减少到低维特征空间时，通常不可避免地会招致不可逆的信息损失。在这项工作中，该问题被制定为一个一般的有损信息处理问题。这个问题的解决方案是有效的算法，可以用来选择信息的功能，普遍相关的一个家庭的推理tasks.The目标的一般理论框架，这个问题是发展系统的理解和统一的性能比较现有的各种各样的实际解决方案。主要的技术优势在于信息度量的新的操作意义，它将大量的信息理论研究与高维数据分析的挑战联系起来。在这项工作中使用了一种新的几何分析方法，这有助于可视化的特征选择的问题，并链接到已被研究的概念的Hirschal-Gebelein-Renyi最大相关性的问题。所提出的方法的主要优点是它的通用性。它可以应用于任何类型的数据，结合先验知识和辅助信息，连接多个平台，遵循计算和存储约束，适应时间变化等，都是基于同样的理论原理。据设想，这种普遍性将导致数据分析领域的架构变化，具有通用接口，将数据科学家在信息提取方面的任务与具有领域知识的专家在收集数据，提供模型和解释结果方面的任务分开。

This project aims at developing an information theoretic view of the problems of dimension reduction and feature selection. The problem of feature selection is a core issue in processing high dimensional data. It is difficult from an information processing point-of-view mainly because when reducing high dimensional data into lower dimensional feature space, it is in general inevitable to incur irreversible information losses. In this work, the problem is formulated as a general lossy information processing problem. The solutions to this problem is efficient algorithms that can be used to choose informative features that are relevant universally to a family of inference tasks.The goal of a general theoretic framework to this problem is to develop systematic understanding and uniform performance comparisons to the existing wide variety of practical solutions. The main technical merit lies in a new operational meaning of information metrics, which connects a large body of research on information theory to the challenges of high dimensional data analytics. A new geometric analysis approach is used in this work, which helps to visualize the problem of feature selections, and link the problem to the well-studied concept of the Hirschfeld-Gebelein-Renyi maximal correlation.The key advantage of the proposed approach is its generality. It can be applied to any type of data, incorporate prior knowledge and side information, connect multiple platforms, follow computation and storage constraints, adapt to time-variations, etc., all based on the same theoretic principle. It is envisioned that such universality would lead to architectural changes in the area of data analysis, with a universal interface that separates the task of a data scientist, in information extraction, from the task of a specialist with domain knowledge, in collecting the data, providing the models, and interpreting the result.