Variable Selection, Instrument Search and Estimation in Problems with Nonignorable Missing Data

缺失数据在许多应用中普遍存在，包括抽样调查、临床试验和医学/健康研究。 当缺失与未观察到的数据有关时，处理不完整的数据特别具有挑战性。 该项目旨在开发用于分析具有不可重复缺失值的数据集的统计模型和方法。 研究主题的动机是调查机构遇到的问题，以及从生物医学/健康研究的数据集。 结果将有显着的影响，无论是在处理不完整的数据，以及在应用程序的真实的数据集从一些科学领域的新方法的发展。当缺失数据机制依赖于未观测数据时，缺失数据被称为不可观测数据。 处理不可观测的缺失数据是一个具有挑战性的问题，因为未观测值遵循与观测数据不同的分布，导致潜在未知参数的可识别性和可估计性问题。 本文的研究主要集中在以下三个方面：（1）工具搜索和模型选择。最近的一种处理不可重复缺失数据的方法是建立在协变量（称为工具）的基础上的，它使研究人员能够识别和估计总体参数。 然而，在应用中，这样的工具必须从一组给定的观测协变量构造。 拟议的研究将开发在不同情况下寻找工具的方法，包括半参数倾向模型，伪似然方法，以及缺失响应和协变量值的问题。 与仪器搜索一起，还将研究关于参数分量的模型选择。 (2)高度降维和变量选择。在许多大数据应用中，协变量向量的维数通常非常大，然而，只有少数协变量是有用的。 虽然在过去的二十年里有大量的研究与降维和变量选择有关，但在存在不可解释的缺失响应的情况下没有结果。 建议的研究重点是协变量的选择或降维两个半参数框架下。 (3)具有不可重复缺失值的多变量数据。该研究计划将开发针对缺失响应和协变量问题的方法、针对生存依赖缺失协变量的生存分析方法、针对有脱落的纵向数据的个性化医疗方法。该奖项反映了NSF的法定使命，通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

Missing data are ubiquitous in many applications including sample surveys, clinical trials and medical/health studies. Handling incomplete data is particularly challenging when the missingness is related to the unobserved data. This project seeks to develop statistical models and methods for analyzing datasets with nonignorable missing values. The research topics are motivated by problems encountered by survey agencies as well as in data sets from biomedical/health studies. The results will have significant impact both in terms of new methodological development for handling incomplete data as well as in applications to real datasets from a number of scientific fields. When the missing data mechanism depends on unobserved data, the missing data are referred to as nonignorable. Handling nonignorable missing data is a challenging problem as the unobserved values follow a distribution different from that of the observed data leading to issues of identifiability and estimability of the underlying unknown parameters. The proposed research focuses on the following three areas: (1) Instrument search and model selection. A recent method for handling nonignorable missing data is built on the use of a covariate (called an instrument) that enables researchers to identify and estimate population parameters. In applications, however, such an instrument must be constructed from a given set of observed covariates. The proposed research will develop methods for finding instruments in different situations, including semiparametric propensity models, pseudo likelihood methods, and problems with both missing responses and covariate values. Together with instrument search, model selection regarding the parametric component will be also investigated. (2) High dimension reduction and variable selection. In many big data applications, the dimension of the covariate vector is often very large, however, only a few covariates are useful. While there is extensive research related to dimension reduction and variable selection over the past two decades, there are no results available in the presence of nonignorable missing responses. The proposed research focuses on covariate selection or dimension reduction under two semiparametric frameworks. (3) Multivariate data with nonignorable missing values. The proposed research will develop methods for problems with both missing responses and covariates, survival analysis with survival-dependent missing covariates, and personalized medicine with longitudinal data having dropouts.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.