Collaborative Research: Integrative Large-Scale Data Analysis and Statistical Inference

最近在数据收集和处理方面的技术进步导致了大量数字信息的积累，其中包括各种类型的辅助信息，如先验数据、外部协变量、领域知识和专家见解。然而，由于可用的分析工具很少，在目前的大多数研究中，许多相关数据和辅助信息都没有得到充分利用。复杂结构的大数据分析提出了重大挑战，需要新的信息集成理论和方法。这项合作研究旨在开发新的程序、计算算法和统计软件，为各个科学领域的研究人员提供强大的工具，帮助他们日常收集和分析高维数据，从而有效地将分散和异构的数据源转化为新的知识。该项目旨在为综合大规模数据分析和统计推断发展新的原理、理论基础和方法。一个重要的主题是研究如何将来自多个来源的信息组合在一个统一的框架中。该项目主要关注四类问题：(i)两个稀疏对象的推理；（ii）结构化同时推理；（iii）同时集合推理和多阶段推理；（四）在基因组学和网络分析中的应用。新的集成框架为从海量数据集的各个部分提取和汇集信息提供了一种强大的方法，并且可以通过提供更准确、信息丰富和可解释的结果来改进传统方法。

Recent technological advancements in data collection and processing have led to the accumulation of vast amount of digital information with various types of auxiliary information such as prior data, external covariates, domain knowledge and expert insights. However, with few analytical tools available, much of the relevant data and auxiliary information have been severely underexploited in most current studies. The analysis of big data with complex structures poses significant challenges and calls for new theory and methodology for information integration. This collaborative research aims to develop new procedures, computational algorithms and statistical software to provide powerful tools for researchers in various scientific fields who routinely collect and analyze high dimensional data, which would help translate dispersed and heterogeneous data sources into new knowledge effectively.This NSF project aims to develop new principles, theoretical foundations and methodologies for integrative large-scale data analysis and statistical inference. An important theme is to study how to combine the information from multiple sources in a unified framework. The project focuses on four types of problems: (i) inference of two sparse objects; (ii) structured simultaneous inference; (iii) simultaneous set-wise inference and multi-stage inference; and (iv) applications in genomics and network analysis. The new integrative framework provides a powerful approach for extracting and pooling information from various parts of massive data sets, and can improve conventional methods by delivering more accurate, informative and interpretable results.