Graphical Multi-Resolution Scanning for Cross-Sample Variation

识别数据集之间的差异是最常见的统计推理任务之一，它位于从天体物理和生物学到经济和政治学等广泛领域的众多应用的核心。最近“大数据”的爆炸性增长在检测跨样本差异方面提出了几个关键挑战，这使得现有的方法不够充分，迫切需要新的方法。最显著和最普遍的挑战包括复杂的分布结构、高度局部的变异性质、各种无关的变异来源、数据稀疏和大量的计算需求。这项研究项目的总体目标是开发一个包括理论、方法、算法和软件的总体框架，以有效识别现代大数据集中的变异，以应对这些挑战。本研究项目要解决的具体推理问题包括：(I)识别多个数据集之间的差异，特别是高度局部的差异；(Ii)将内部(即科学上感兴趣的)跨样本差异与外部差异分开；(Iii)将跨样本差异分解为来自多个来源的贡献；以及(Iv)识别一般随机对象中的跨样本差异和方差分量，包括各种过程和功能观察。多尺度推理和贝叶斯非参数建模的使用导致了基于概率模型的通用框架的发展，该框架集成了两种强大的推理策略--多分辨率扫描和图形建模。多分辨率扫描是使用不同大小的窗口扫描样本空间，在每个窗口上对感兴趣的结构--交叉样本变化--进行测试或估计的策略。然后，设计了一类图形模型，以结合跨扫描窗口和数据样本的各种相关性结构，从而允许在窗口和相关样本之间借用强度，以在识别跨样本变化时实现高统计效率。该项目旨在构建一套计算高效和理论上合理的推理方法和算法，并研究它们的统计特性。

Identifying variation across data sets is one of the most commonly encountered statistical inferential tasks, and it lies at the heart of numerous applications in a wide range of fields from astrophysics and biology to economics and political science. The recent explosion of "big data" has raised several critical challenges in detecting cross-sample variation, which render existing methods inadequate and entail an urgent need for new methodologies. The most notable and prevalent challenges include complex distributional structures, the highly local nature of variation, various extraneous sources of variation, data sparsity, and massive computational demand. The overarching aim of this research project is to develop a general framework including theory, methods, algorithms, and software for effectively identifying variation in modern big data sets to address these challenges. Specific inference problems to be addressed in this research project include: (i) identifying differences, especially highly local variations, across multiple data sets; (ii) separating intrinsic (i.e., scientifically interesting) cross-sample variation from extraneous variation; (iii) decomposing cross-sample variation into contributions from multiple sources; and (iv) identifying cross-sample variation and variance components in general random objects, including a variety of processes and functional observations. The use of multi-scale inference and Bayesian nonparametric modeling has led to development of a general probabilistic model-based framework for detecting cross-sample variation that integrates two powerful inference tactics -- multi-resolution scanning and graphical modeling. Multi-resolution scanning is the strategy of scanning through the sample space using windows of various sizes, carrying out testing or estimation for the structure of interest -- the cross-sample variation -- on each window. A class of graphical models is then designed to incorporate various dependency structures across scanning windows and data samples, thereby allowing borrowing strength among windows and related samples to achieve high statistical efficiency in identifying cross-sample variation. The project aims to construct a suite of computationally efficient and theoretically justifiable inferential methods and algorithms and to investigate their statistical properties.