CAREER: Statistical Learning, Inference and Approximation with Reproducing Kernels

许多现代科学领域，如天体物理、生物信息学、金融、法医学、社会科学等，都产生了大量的高维和非标准数据。例如，数据可以具有图形、函数、字符串和集合等结构，但不是欧几里得结构。为了分析这些数据集并处理这些领域中出现的各种统计应用，需要能够处理高维和非标准数据的有效学习和推理程序。涉及再生核的泛函分析范式，也被称为核方法，提供了一个处理此类数据的统一框架，并已被机器学习社区应用于各种非参数统计问题，取得了巨大的经验成功。然而，它在统计最优性方面的理论理解一直是有限的，而且在计算上它对大数据的可扩展性很差。该项目的重点是探索与核方法相关的各种基础研究问题，以实现能够处理高维非标准数据的统计最优和计算效率高的范例。这项研究将对与统计学交叉的所有科学和工程领域的科学发展产生重大影响，并将与国际学生联合会的指导学生、开发新课程和建立新合作的教育活动相结合。在该项目下开发的方法和代码将公开提供，以供随时使用。核方法的核心思想是将观测数据(可以是高维的和非标准的)映射到奇异函数空间，称为再生核希尔伯特空间(RKHS)，并将针对欧几里得数据开发的标准方法应用于映射后的数据。具有讽刺意味的是，RKHS通常比观测数据的维度更高(甚至是无限维)，并且由称为再生核的核函数来表征。核方法的主要优点是它能够通过核函数简单地探索RKHS中映射元素之间的线性关系，从而探索数据中的非线性关系。尽管基于核方法的学习算法具有优越的经验性能，但除了在分类、非参数最小二乘回归、主成分分析和拟合优度检验等少数情况下，学习算法的统计理论还没有得到很好的理解。在TITE项目中，PI将探索与核方法和相关学习算法相关的各种基础研究问题，以解决这一差距。该项目由四个相关的研究主题组成，总体上寻求加深对核心方法的数学理解，以便充分发挥其在构建能够有效处理非标准数据的推理程序方面的力量。该项目还将阐明核方法相对于文献中其他非参数方法的优势和局限性。其目的是(I)为基于核的假设检验和非线性典型相关分析开发统计最优性结果，(Ii)使用诸如Nystrom方法、随机特征及其变体的近似方案来开发各种核学习过程的计算与统计权衡分析，以加速这些过程，(Iii)使用核方法来开发具有具体数学保证的新方法，用于函数和概率分布的学习和推理，并在函数数据分析中应用，以及(Iv)使用多尺度核来推广核方法以获得小波样表示，并研究其在各种学习过程中的统计和计算行为。总体而言，该项目将为计算效率高的基于核的学习算法开发一个全面的数学理论，并将其应用于统计学习。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Many modern scientific fields, such as astrophysics, bio-informatics, finance, forensics, social science, and others generate massive amounts of data that are both high-dimensional and non-standard. For example, the data may have structure such as graphs, functions, strings, and sets, but is not Euclidean. To analyze these data sets and address various statistical applications arising in these fields, efficient learning and inference procedures that can handle high-dimensional and non-standard data are needed. The functional analytic paradigm involving reproducing kernels, also known as the kernel method, provides a unified framework to handle such data and has been applied to a variety of non-parametric statistical problems with great empirical success by the machine learning community. However, its theoretical understanding in terms of statistical optimality has been limited, and computationally it scales poorly to large data. The key focus of this project is to explore various foundational research questions associated with the kernel method to achieve a statistically optimal and computationally efficient paradigm that can handle high-dimensional non-standard data. This research will significantly impact scientific development in all areas of science and engineering that intersect with statistics, and will be integrated with the PI's educational activities of mentoring students, developing new courses and forging new collaborations. Methods and code developed under this project will be made publicly available for ready use. The core idea behind the kernel method is to map the observed data (could be high-dimensional and non-standard) to exotic function space, called the reproducing kernel Hilbert space (RKHS) and apply the standard methods developed for Euclidean data on the mapped data. Ironically, the RKHS is usually higher dimensional (even infinite-dimensional) than the dimensionality of the observed data, and is characterized by a kernel function called the reproducing kernel. The main advantage of the kernel method is its ability to explore nonlinear relationships in data by simply exploring linear relationships between the mapped elements in the RKHS through the kernel function. Despite its superior empirical performance, the statistical theory of learning algorithms based on the kernel method is not well understood except in a few cases such as classification, non-parametric least square regression, principal component analysis and goodness-of-fit testing. In tise project, the PI will explore various foundational research questions associated with the kernel method and associated learning algorithms to address this gap. The project consists of four related research themes that overall seek to deepen the mathematical understanding of the kernel method so as to exploit its full power in constructing inference procedures that can efficiently handle non-standard data. The project will also shed light on the advantages and limitations of the kernel method over other non-parametric methods in the literature. The aims are to (i) Develop statistical optimality results for kernel-based hypothesis tests and non-linear canonical correlation analysis, (ii) Develop computational vs. statistical trade-off analysis for various kernel learning procedures using approximation schemes such as Nystrom method, random features and their variations, that speed up these procedures, (iii) Develop new methodologies with concrete mathematical guarantees using the kernel method for learning and inference on functions and probability distributions, with applications in functional data analysis, and (iv) Generalize the kernel method using multi-scale kernels to obtain wavelet-like representations and investigate its statistical and computational behaviors in various learning procedures. Overall, the project will develop a comprehensive mathematical theory for computationally efficient kernel-based learning algorithms with applications in statistical learning.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.