Collaborative Research: Smoothing Spline Semiparametric Density Models

多变量的概率密度函数描述了变量可以联合取的不同值的可能性，因此包含了关于单个变量的分布及其相互作用的全部信息。给定随机变量的观测数据，密度估计是统计学和机器学习的核心，回归、变量选择、聚类和降维等经典问题都可以归结为密度估计问题。因此，先进的密度估计方法对于从数据中提取尽可能多的信息至关重要。对于高维数据或复杂数据(如聚集数据)的灵活密度估计，一直缺乏系统的研究。这个项目的总体目标是开发一个基于平滑样条线的系统框架，允许对复杂和高维数据进行灵活的密度建模。由于这些数据来自广泛的应用，这项拟议研究的结果对来自广泛领域的研究人员是有用的。特别是，拟议的方法将与这些领域的研究人员合作，用于分析卫生和医学、言语、环境变化、食品和计算机科学方面的数据。作为这项研究的结果，将开发高性能计算工具并公开可用。本项目采用半参数方法，结合了参数方法和非参数方法的优点。将开发和研究适用于独立和聚集数据的灵活和通用的半参数密度和条件密度模型。发展用于自适应密度估计、高维条件密度估计的变量选择和半参数图形模型的交互选择的正则化方法。非参数分量将使用再生核Hilbert空间来建模，该空间可以统一地处理不同区域上不同密度的模型，具有不同的惩罚。本课题中考虑的半参数密度模型包含了大多数已有的半参数密度模型作为特例，以及许多新的有趣的模型。本项目中的许多自适应估计、模型/变量选择、模型诊断和推理方法都是新的。这些新的方法构成了密度估计的进步。

A probability density function of multiple variables describes the likelihood of different values the variables can jointly take, therefore, contains full information regarding the distribution of individual variables and their interactions. Given observed data of the random variables, density estimation is at the heart of Statistics and machine learning, where the classical problems such as regression, variable selection, clustering, and dimension reduction, can all be cast into a density estimation problem. Advanced density estimation methods are therefore essential for the extraction of as much information as possible from the data. There has been lack of systematic research in flexible density estimation with high dimensional data or complex data such as clustered data. The overall goal of this project is to develop a smoothing spline based systematic framework that allows for flexible density model building for complex and high dimensional data. As such data arise from a wide range of applications, the results of this proposed research are useful for researchers from a wide range of fields. In particular, the proposed methods will be applied to analyze data in health and medicine, speech, environmental change, food and computer sciences, in collaboration with researchers in these areas. High-performance computing tools will be developed as a result of this research and made publicly available.This project adopts a semi-parametric approach that combines advantages of parametric and nonparametric methods. Flexible and general semi-parametric density and conditional density models for independent and clustered data will be developed and studied. Regularization methods for adaptive density estimation, variable selection in high dimensional conditional density estimation and interaction selection in semi-parametric graphical models will be developed. Nonparametric components will be modeled using reproducing kernel Hilbert spaces which can deal with different density models on different domains with different penalties in a unified fashion. The semiparametric density models considered in this project contain most existing semiparametric density models as special cases as well as many new interesting models. Many methods in this project for adaptive estimation, model/variable selection, model diagnostics and inference are new. These novel methodologies constitute advances in density estimation.