EAGER: Nonparametric Machine Learning on Sets, Functions, and Distributions

大多数机器学习算法在固定维度的特征向量表示上操作。然而，在许多应用中，数据的自然表示由更复杂的对象组成，例如函数，分布和集合，而不是有限维向量。该项目旨在开发一系列新的机器学习算法，可以直接对这些复杂对象进行操作。关键的创新是有效地估计某些信息理论量，用于从复杂数据中学习预测模型。该研究围绕三个具体目标组织：（a）发展和分析非参数估计的某些重要泛函的密度，如熵，互信息，条件互信息和分歧;和研究这些估计的理论性质，包括一致性，收敛速度的偏差和方差，渐近正态性。(b)使用上述估计器来设计新的学习算法，用于聚类，分类，回归和异常检测，这些算法直接作用于集合，函数和分布，而无需任何额外的手工特征提取，直方图创建或密度估计步骤，这些步骤可能会导致信息丢失。(c)研究这些新的机器学习算法的理论特性（计算时间，样本复杂性，泛化误差）和经验评估的算法，他们对各种重要的现实世界的问题，包括核探测天文数据分析，和计算机视觉与研究人员合作，在劳伦斯利弗莫尔，华盛顿大学和约翰霍普金斯大学，和卡内基梅隆大学。更广泛的影响。该项目如果成功，将大大推进从复杂数据中构建预测模型的当前最先进技术。研究成果，包括出版物和开放源码软件，将免费向广大科学界传播。该项目为卡内基梅隆大学的研究生和本科生以及合作机构提供了增强的研究型培训机会。

Most machine learning algorithms operate on fixed dimensional feature vector representations. In many applications, however, the natural representation of the data consists of more complex objects, for example functions, distributions, and sets, rather than finite-dimensional vectors. This project aims to develop a new family of machine learning algorithms that can operate directly on these complex objects. The key innovation is efficient estimation of certain information theoretic quantities for learning predictive models from complex data. The research is organized around three specific aims: (a) Development and analysis of nonparametric estimators for certain important functionals of densities, such as entropy, mutual information, conditional mutual information, and divergence; and study of the theoretical properties of these estimators including consistency, convergence rates of the bias and variance, and asymptotic normality. (b) Use of the preceding estimators to design new learning algorithms for clustering, classification, regression, and anomaly detection that work directly on sets, functions, and distributions without any additional, hand-made feature extraction, histogram creation, or density estimation steps that could lead to loss of information. (c) Study of the theoretical properties of these new machine learning algorithms (computation time, sample complexity, generalization error) and empirical evaluation of the algirithms them to a variety of important real-world problems, including nuclear detection astronomical data analysis, and computer vision in collaboration with researchers at Lawrence Livermore, University of Washington and Johns Hopkins University, and Carnegie Mellon University respectively.Broader Impact. The project, if successful, could substantially advance the current state-of-the-art in building predictive models from complex data. The results of research, including publications and open source software, will be freely disseminated to the larger scientific community. The project provides enhanced research-based training opportunities for graduate and undergraduate students at Carnegie Mellon University as well as the collaborating institutions.