CAREER: New Change-Point Problems in Analyzing High-Dimensional and Non-Euclidean Data

在这个大数据时代，在各个科学领域收集大量的数据序列，用于研究复杂的时空现象，包括神经科学、流行病学、社会科学、计算机视觉、天文学等。变化点分析是对这些数据序列进行分析的关键早期步骤，例如在在线数据监测中出现异常事件时发出警报，将长序列分割成更均匀的部分以供后续研究。为了适应现代应用程序，处理高吞吐量数据和具有复杂结构的数据的能力正在成为一种必要。参数方法通常不能应用于非常高的维度，除非做出强有力的假设以避免估计大量的干扰参数。该项目侧重于开发非参数变化点检测方法，该方法不需要强假设，并且在计算上可扩展到高维和复杂数据。该项目为学生和研究人员提供了令人兴奋的具有统计学和科学重要性的新研究问题。本科生和研究生的培训部分将为跨学科教育培养新的研究人员。这个项目将通过一种新的基于图形的方法来开发一个新的扫描统计框架。PI已经表明，基于图的方法适用于高维和非欧几里得数据，并允许与特定应用程序建模解耦的通用分析排列p值近似，从而促进了它们在大型复杂数据集上的应用。尽管基于图的方法具有良好的特性，但其当前版本与许多现代应用程序之间仍然存在一些差距。这个项目旨在填补这些重要的空白。特别是，该项目将(1)开发新的基于图形的方法来有效地集成来自多个来源的信息，这在许多应用领域（如智能家居和智能城市）中很常见，并寻求将新方法分发到本地中心的方法，以避免分布式系统中原始数据的过度传输；(2)从构造图的层面发展处理依赖数据的方法，比PI先前开发的圆块排列框架更有效；(3)开发一个新的框架，为基于图的方法提供分析功率近似，这些方法适用于数百甚至数千个样本大小，甚至适用于高维数据和非欧几里得数据，从而促进研究人员在实际应用中做出更好的决策。这些方法和理论的发展将提供对来自不同领域的现代复杂数据序列的更好理解，这将进一步促进对这些领域的重大科学问题的理解。在这个项目中开发的工具将作为带有详细文档的开源软件包发布。这将加强统计界与来自更广泛科学领域的研究人员之间的合作，并使数据分析程序更加透明。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

In this big data era, massive data sequences are collected in various scientific fields for studying complicated phenomena over time and space, including neuroscience, epidemiology, social science, computer vision, and astronomy. Change-point analysis is a crucial early step in analyzing these data sequences, such as, to raise an alarm when an abnormal event happens in online data monitoring, and to segment a long sequence into more homogeneous parts for follow-up studies. To accommodate modern applications, the ability to deal with high throughput data and data with complicated structures is becoming a necessity. Parametric methods usually cannot be applied to very high dimensions unless strong assumptions are made to avoid the estimation of a large number of nuisance parameters. This project focuses on developing non-parametric change-point detection methods that are free of strong assumptions and computationally scalable to high dimensional and complex data. This project provides students and researchers with exciting new research problems that have both statistical and scientific importance. The training component for undergraduate and graduate students will prepare new researchers with inter-disciplinary education.This project will develop a new scan statistic framework through a novel adaptation of graph-based methods. The PI has shown that the graph-based approaches scale to high-dimensional and non-Euclidean data, and allow for universal analytic permutation p-value approximations that is decoupled from application-specific modeling, facilitating their applications to large and complicated data sets. Despite the good properties of the graph-based methods, there are still some gaps between its current versions and many modern applications. This project aims to fill those important gaps. In particular, this project will (1) develop new graph-based approaches to effectively integrate information from multiple sources, which is common in many application areas, such as smart homes and smart cities, and seek ways to distribute the new approaches to local centers to avoid the excessive transmission of raw data in a distributed system; (2) develop treatments from the level of constructing the graph to deal with dependent data, which is more effective than a circular block permutation framework developed by the PI earlier; and (3) develop a new framework to provide analytic power approximations to the graph-based methods that kick in for sample sizes in hundreds and thousands even for high-dimensional data and non-Euclidean data, facilitating researchers to make better decisions in real applications. These methodological and theoretical developments will provide better understandings of modern complicated data sequences from diverse fields, which will further advance the understanding of major scientific problems in these fields. The tools developed in this project will be distributed as open source software packages with detailed documentations. This will enhance the collaboration between the statistics community and researchers from broader scientific fields, and make data analysis procedures more transparent.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.