Reproducing Kernel Hilbert Space Embedding of Measures: Theory and Applications to Statistical Learning

在这个项目中考虑的统计估计和推理问题出现在依赖统计研究的科学和工程的许多领域。其中一些具有高度社会影响的领域包括社会和行为研究、法医学、秘密通信和数据安全漏洞的早期发现、确定传染病爆发的早期预警系统、儿童认知发展研究和药物发现等。在许多这些领域中，常用的统计方法对数据生成分布做出了强有力的假设。这种简单化的做法可能是不合理的；此外，即使这些假设有道理，它们也需要检验。本研究项目调查了这些现有方法的一个强大的替代方案，旨在发展对这些方法的基本理论理解，从而导致新的统计应用。该项目产生的算法代码将公开提供，可供随时使用。核方法是一类统计方法，由于其处理高维和非欧几里得数据的能力，在统计学习中得到了普及。该方法的核心思想是将观察到的数据映射到称为再现核希尔伯特空间（RKHS）的函数空间，该函数空间允许捕获数据中的非线性关系。该项目涉及通过在RKHS中嵌入概率度量来推广该方法的理论和方法研究。这种泛化在统计学习问题中具有广泛的适用性，例如非参数假设检验、密度估计和分布回归，这些将在本项目中进行探讨。在理论方面，将考虑核嵌入的注入性表征。虽然对于在局部紧阿贝尔群和紧非阿贝尔群上定义的核来说，这样的特征已经被很好地理解了，但本项目将研究核嵌入在非标准空间（如核空间、图空间和正定锥）中的注入性。已知嵌入的注入性与RKHS在逼近某一类函数时的丰富度有关。该研究将调查这种近似的速率，这对于分析基于核的回归和密度估计器的收敛速率以及假设检验中的分离速率至关重要。单射嵌入在概率空间上产生一个度量，称为核距离，它被定义为两个概率测度的核嵌入之间的RKHS距离。研究者将研究核距离与其他概率度量的关系，如能量距离、距离协方差、f散度和积分概率度量，以了解与这些距离相关的统计/计算（dis）优势。这些理论研究有相应的应用，其中RKHS嵌入在无限维指数族的概率测度回归和密度估计问题中起着关键作用。对于这些问题，研究者计划开发具有理论保证的计算效率估计器。总体而言，该项目旨在开发RKHS嵌入概率度量的综合理论，并将其应用于统计学习中的问题。

The statistical estimation and inference questions considered in this project appear in many areas of science and engineering that rely on statistical research. Some of these areas of high societal impact include social and behavioral research, forensic sciences, early detection of covert communications and breaches in data security, early-warning systems for identifying outbreaks of infectious diseases, cognitive development studies in children, and drug discovery, among many others. In many of these areas, commonly-used statistical methods make strong assumptions about the data-generating distribution. Such simplistic approaches may be unjustified; moreover, even if these assumptions make sense, they need to be tested. This research project investigates a powerful alternative to these existing methods and aims to develop a fundamental theoretical understanding of the same, leading to novel statistical applications. Code for algorithms that result from this project will be made publicly available for ready use.The kernel method is a class of statistical methodology that has gained popularity in statistical learning due to its ability to handle both high-dimensional and non-Euclidean data. The core idea of the method is to map observed data to a function space, called the reproducing kernel Hilbert space (RKHS), which allows capture of non-linear relationships in the data. This project concerns theoretical and methodological research on a generalization of this method by embedding probability measures in an RKHS. This generalization has wide applicability in statistical learning problems such as nonparametric hypothesis testing, density estimation, and regression on distributions, which will be explored in this project. On the theoretical front, the characterization of injectivity of kernel embedding will be considered. While such a characterization is well understood for kernels defined on locally compact Abelian groups and compact non-Abelian groups, this project will investigate the injectivity of the kernel embedding for non-standard spaces such as nuclear spaces, the space of graphs, and the positive definite cone. The injectivity of the embedding is known to be related to the richness of the RKHS in approximating a certain class of functions. The research will investigate the rate of this approximation, which turns out to be critical in analyzing the convergence rates of kernel-based regression and density estimators and separation rates in hypothesis testing. An injective embedding induces a metric, called the kernel distance on the space of probabilities, which is defined as the RKHS distance between the kernel embeddings of two probability measures. The investigator will study the relation of kernel distance to other probability metrics such as the energy distance, distance covariance, f-divergence, and integral probability metrics in order to understand the statistical/computational (dis)advantages associated with these distances. These theoretical studies have an applied counterpart, wherein the RKHS embedding plays a critical role in the problems of regression on probability measures and density estimation in infinite dimensional exponential families. For these problems, the investigator plans to develop computationally efficient estimators with theoretical guarantees. Overall, the project aims to develop a comprehensive theory of RKHS embedding of probability measures with applications to problems in statistical learning.