Statistical Challenges and Methods in the Analysis of High Dimensional and Complex Structured Data

点击翻译按钮获取中文摘要

With the advancement of modern technologies, complex structured data such as dependent data and data with measurement error and high dimensionality arise in various application areas, including medical and cancer research, genetic studies, and industries. New challenges emerge when extracting information from the available data. Statistical methods therefore play a vital role, and are urgently in demand for enabling more efficient use of the rich source of data. One of the most challenging statistical issues is to address the complex structures in data to effectively identify significant covariates and to utilize the obtained covariates to construct powerful predictive models. Extensive research attention has been directed to this area and there has been a rapid growth in the literature in recent years. However, the assumptions of existing methods are often too stringent and there is a lack of methodology to deal with various specific features of the data. The primary objective of this research is to develop coherent and novel methodology to address the complex structures of data, including ultra-high dimensionality of covariates, imbalanced observations, measurement error in covariates, measurement error in response and hence, to better understand the underlying statistical structure and to efficiently extract helpful information. Specifically, I plan to develop methodologies in the following directions.I plan to construct predictive models for outcomes with the accommodation of specific features, such as imbalanced observations and measurement errors. Widely used predictive models includes a variety of classification methods perform well if the observations are balanced, but in real life it is often the case that observations are imbalanced and the observed data contain measurement error, especially when data are obtained through complex experiments such as those in genomic studies. This work aims to broaden the scope of existing methods on this topic and offer useful complement tools.Another area I plan to develop concerns new methods for ultra-high dimensional variable screening by incorporating random, deterministic or a mix of design matrices. It is common that the majority of the high dimensional variables do not have relevance to the response, but the existing variable selection methods are not usually valid or efficient for handling ultra high dimensional variables. It is crucial to reduce the number of variables by applying screening methods before invoking variable selection procedures to identify the final statistical models. Furthermore, it is important to investigate the measurement error effects on variable screening which I plan to explore in depth.The proposed methodology will lead to valuable new insights into many aspects of statistical research and will be of great importance for the development of areas including medical, computer and defence sciences.