EAGER: Nonparametric Machine Learning on Sets, Functions, and Distributions

大多数机器学习算法都是在固定维度的特征向量表示上运行的。然而，在许多应用程序中，数据的自然表示由更复杂的对象组成，例如函数、分布和集合，而不是有限维向量。该项目旨在开发一种新的机器学习算法，可以直接对这些复杂的对象进行操作。关键的创新是有效地估计某些信息理论量，以便从复杂数据中学习预测模型。本研究围绕三个具体目标进行：(a)发展和分析密度的某些重要函数的非参数估计，如熵、互信息、条件互信息和散度；并研究了这些估计量的理论性质，包括相合性、偏差和方差的收敛率以及渐近正态性。(b)使用前面的估计器来设计新的学习算法，用于聚类、分类、回归和异常检测，这些算法直接作用于集合、函数和分布，而不需要任何额外的、手工的特征提取、直方图创建或密度估计步骤，这些步骤可能导致信息丢失。(c)研究这些新的机器学习算法的理论性质（计算时间、样本复杂度、泛化误差），并对这些算法进行实证评估，以解决各种重要的现实问题，包括核探测、天文数据分析和计算机视觉，分别与劳伦斯利弗莫尔大学、华盛顿大学、约翰霍普金斯大学和卡内基梅隆大学的研究人员合作。更广泛的影响。该项目如果成功，将大大推进当前从复杂数据中构建预测模型的最先进技术。研究成果，包括出版物和开放源码软件，将免费传播给更大的科学界。该项目为卡内基梅隆大学以及合作院校的研究生和本科生提供了更多的基于研究的培训机会。

Most machine learning algorithms operate on fixed dimensional feature vector representations. In many applications, however, the natural representation of the data consists of more complex objects, for example functions, distributions, and sets, rather than finite-dimensional vectors. This project aims to develop a new family of machine learning algorithms that can operate directly on these complex objects. The key innovation is efficient estimation of certain information theoretic quantities for learning predictive models from complex data. The research is organized around three specific aims: (a) Development and analysis of nonparametric estimators for certain important functionals of densities, such as entropy, mutual information, conditional mutual information, and divergence; and study of the theoretical properties of these estimators including consistency, convergence rates of the bias and variance, and asymptotic normality. (b) Use of the preceding estimators to design new learning algorithms for clustering, classification, regression, and anomaly detection that work directly on sets, functions, and distributions without any additional, hand-made feature extraction, histogram creation, or density estimation steps that could lead to loss of information. (c) Study of the theoretical properties of these new machine learning algorithms (computation time, sample complexity, generalization error) and empirical evaluation of the algirithms them to a variety of important real-world problems, including nuclear detection astronomical data analysis, and computer vision in collaboration with researchers at Lawrence Livermore, University of Washington and Johns Hopkins University, and Carnegie Mellon University respectively.Broader Impact. The project, if successful, could substantially advance the current state-of-the-art in building predictive models from complex data. The results of research, including publications and open source software, will be freely disseminated to the larger scientific community. The project provides enhanced research-based training opportunities for graduate and undergraduate students at Carnegie Mellon University as well as the collaborating institutions.