Statistical Challenges and Methods in the Analysis of High Dimensional and Complex Structured Data

高维和复杂结构化数据分析中的统计挑战和方法

• 批准号: RGPIN-2018-05475
• 负责人: He, Wenqing
• 金额: $ 1.46万
• 依托单位: University of Western Ontario
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2019
• 资助国家: 加拿大
• 起止时间: 2019-01-01 至 2020-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=677292
• 关键词: Statistical; Challenges; Methods; Analysis; Dimensional

With the advancement of modern technologies, complex structured data such as dependent data and data with measurement error and high dimensionality arise in various application areas, including medical and cancer research, genetic studies, and industries. New challenges emerge when extracting information from the available data. Statistical methods therefore play a vital role, and are urgently in demand for enabling more efficient use of the rich source of data. One of the most challenging statistical issues is to address the complex structures in data to effectively identify significant covariates and to utilize the obtained covariates to construct powerful predictive models. Extensive research attention has been directed to this area and there has been a rapid growth in the literature in recent years. However, the assumptions of existing methods are often too stringent and there is a lack of methodology to deal with various specific features of the data. The primary objective of this research is to develop coherent and novel methodology to address the complex structures of data, including ultra-high dimensionality of covariates, imbalanced observations, measurement error in covariates, measurement error in response and hence, to better understand the underlying statistical structure and to efficiently extract helpful information. Specifically, I plan to develop methodologies in the following directions.******I plan to construct predictive models for outcomes with the accommodation of specific features, such as imbalanced observations and measurement errors. Widely used predictive models includes a variety of classification methods perform well if the observations are balanced, but in real life it is often the case that observations are imbalanced and the observed data contain measurement error, especially when data are obtained through complex experiments such as those in genomic studies. This work aims to broaden the scope of existing methods on this topic and offer useful complement tools.******Another area I plan to develop concerns new methods for ultra-high dimensional variable screening by incorporating random, deterministic or a mix of design matrices. It is common that the majority of the high dimensional variables do not have relevance to the response, but the existing variable selection methods are not usually valid or efficient for handling ultra high dimensional variables. It is crucial to reduce the number of variables by applying screening methods before invoking variable selection procedures to identify the final statistical models. Furthermore, it is important to investigate the measurement error effects on variable screening which I plan to explore in depth.******The proposed methodology will lead to valuable new insights into many aspects of statistical research and will be of great importance for the development of areas including medical, computer and defence sciences.**

随着现代技术的进步，医学和癌症研究、基因研究和工业等各个应用领域中出现了复杂的结构化数据，例如相关数据、具有测量误差和高维的数据。从可用数据中提取信息时出现了新的挑战。因此，统计方法发挥着至关重要的作用，并且迫切需要更有效地利用丰富的数据源。最具挑战性的统计问题之一是解决数据中的复杂结构，以有效识别重要的协变量，并利用获得的协变量构建强大的预测模型。近年来，该领域受到广泛的研究关注，文献数量迅速增长。然而，现有方法的假设往往过于严格，缺乏处理数据各种具体特征的方法论。这项研究的主要目标是开发连贯且新颖的方法来解决数据的复杂结构，包括协变量的超高维度、不平衡观察、协变量的测量误差、响应的测量误差，从而更好地理解潜在的统计结构并有效地提取有用的信息。具体来说，我计划在以下方向开发方法。******我计划构建适应特定特征（例如不平衡观察和测量误差）的结果预测模型。广泛使用的预测模型包括多种分类方法，如果观察结果是平衡的，则表现良好，但在现实生活中，经常会出现观察结果不平衡，观察到的数据包含测量误差的情况，特别是当数据是通过复杂的实验（例如基因组研究中的实验）获得时。这项工作旨在扩大该主题现有方法的范围，并提供有用的补充工具。 *****我计划开发的另一个领域涉及通过结合随机、确定性或混合设计矩阵来进行超高维变量筛选的新方法。 通常，大多数高维变量与响应没有相关性，但现有的变量选择方法通常对于处理超高维变量来说并不有效或高效。在调用变量选择程序以确定最终统计模型之前，通过应用筛选方法来减少变量数量至关重要。此外，研究测量误差对变量筛选的影响也很重要，我计划深入探讨这一点。******所提出的方法将为统计研究的许多方面带来有价值的新见解，并对医学、计算机和国防科学等领域的发展具有重要意义。**