Collaborative Research: Smoothing Spline Semiparametric Density Models

多个变量的概率密度函数描述了变量可以共同采取的不同值的可能性，因此，包含有关单个变量及其相互作用的分布的全部信息。给定随机变量的观测数据，密度估计是统计学和机器学习的核心，其中回归，变量选择，聚类和降维等经典问题都可以转化为密度估计问题。 因此，先进的密度估计方法对于从数据中提取尽可能多的信息至关重要。对于高维数据或复杂数据（如聚类数据）的柔性密度估计，目前还缺乏系统的研究。该项目的总体目标是开发一个基于平滑样条的系统框架，允许为复杂和高维数据建立灵活的密度模型。由于这些数据来自广泛的应用，因此这项研究的结果对来自各个领域的研究人员都很有用。特别是，所提出的方法将应用于分析健康和医学，语音，环境变化，食品和计算机科学的数据，与这些领域的研究人员合作。高性能的计算工具将作为这项研究的结果开发出来，并公布于众。本项目采用了半参数方法，结合了参数和非参数方法的优点。灵活和一般的半参数密度和条件密度模型的独立和集群数据将开发和研究。将发展自适应密度估计的正则化方法、高维条件密度估计中的变量选择和半参数图模型中的交互作用选择。非参数分量将使用再生核希尔伯特空间建模，该空间可以以统一的方式处理不同域上的不同密度模型，并具有不同的惩罚。在这个项目中考虑的半参数密度模型包含了大多数现有的半参数密度模型的特殊情况，以及许多新的有趣的模型。在这个项目中的自适应估计，模型/变量选择，模型诊断和推理的许多方法是新的。这些新的方法构成了密度估计的进步。

A probability density function of multiple variables describes the likelihood of different values the variables can jointly take, therefore, contains full information regarding the distribution of individual variables and their interactions. Given observed data of the random variables, density estimation is at the heart of Statistics and machine learning, where the classical problems such as regression, variable selection, clustering, and dimension reduction, can all be cast into a density estimation problem. Advanced density estimation methods are therefore essential for the extraction of as much information as possible from the data. There has been lack of systematic research in flexible density estimation with high dimensional data or complex data such as clustered data. The overall goal of this project is to develop a smoothing spline based systematic framework that allows for flexible density model building for complex and high dimensional data. As such data arise from a wide range of applications, the results of this proposed research are useful for researchers from a wide range of fields. In particular, the proposed methods will be applied to analyze data in health and medicine, speech, environmental change, food and computer sciences, in collaboration with researchers in these areas. High-performance computing tools will be developed as a result of this research and made publicly available.This project adopts a semi-parametric approach that combines advantages of parametric and nonparametric methods. Flexible and general semi-parametric density and conditional density models for independent and clustered data will be developed and studied. Regularization methods for adaptive density estimation, variable selection in high dimensional conditional density estimation and interaction selection in semi-parametric graphical models will be developed. Nonparametric components will be modeled using reproducing kernel Hilbert spaces which can deal with different density models on different domains with different penalties in a unified fashion. The semiparametric density models considered in this project contain most existing semiparametric density models as special cases as well as many new interesting models. Many methods in this project for adaptive estimation, model/variable selection, model diagnostics and inference are new. These novel methodologies constitute advances in density estimation.