Handling incomplete observations with supplementary information in event history data analysis

点击翻译按钮获取中文摘要

Incomplete observations arise in numerous settings. Examples include missing, censored, and truncated data; problems such as measurement errors and latent variables may be viewed as special cases, if we take incomplete observations more generally to mean that the available data lack desired information. Various types of incomplete data motivated many major methodology advances in event history data analysis. In reality, the incompleteness mechanism that gives rise to a particular set of incomplete observations is usually unknown and random, and rarely of scientific interest. To account for it properly in an effort to draw valid and desired inferences, statistical methods for analyzing incomplete data require strong assumptions on the mechanism, such as missing at random and non-informative censoring, and/or estimation of the mechanism's distribution. However, the assumptions cannot be verified in general with the current incomplete data, and are regularly criticized as implausible in practice. Plus, estimating the distribution of the mechanism often complicates the primary inference. Our objective is the development of new theory and methods for incomplete data analysis. We propose a novel formulation for incomplete observations, which extends the notion of coarsening and incorporates truncation. Notice that additional information, which may be in the form of knowledge and/or data on the underlying distribution or the incompleteness mechanism, is often readily available or easily collected by design from the same study or from different information sources. We explore the usefulness of supplementary information, and develop treatments to address particular challenges in identifying the incompleteness mechanism, improving inference efficiency and reducing computational intensity arising from real data sets in epidemiology, marketing and reliability. We focus on event history analysis but the methodologies have broader applications; this research program exemplifies it by extensions in longitudinal and spatial analysis. The research outcomes will shed new light on the theory, provide feasible alternative approaches to handling incomplete data, and help to advance the practice.