CAREER: New Change-Point Problems in Analyzing High-Dimensional and Non-Euclidean Data

在这个大数据时代，各个科学领域收集了大量数据序列，用于研究时间和空间上的复杂现象，包括神经科学、流行病学、社会科学、计算机视觉和天文学。 变点分析是分析这些数据序列的关键早期步骤，例如在线数据监测中发生异常事件时发出警报，以及将长序列分割成更同质的部分以供后续研究。 为了适应现代应用，处理高吞吐量数据和复杂结构数据的能力正在成为必需。 参数方法通常不能应用于非常高的维度，除非做出强有力的假设以避免估计大量有害参数。 该项目专注于开发非参数变点检测方法，该方法无需强假设，并且可在计算上扩展到高维和复杂数据。 该项目为学生和研究人员提供了令人兴奋的新研究问题，这些问题具有统计和科学重要性。 本科生和研究生的培训部分将为接受跨学科教育的新研究人员做好准备。该项目将通过基于图形的方法的新颖改编来开发新的扫描统计框架。 PI 表明，基于图的方法可以扩展到高维和非欧几里德数据，并允许与特定于应用程序的建模分离的通用分析排列 p 值近似，从而促进其应用于大型复杂数据集。 尽管基于图的方法具有良好的特性，但其当前版本与许多现代应用程序之间仍然存在一些差距。该项目旨在填补这些重要空白。 具体来说，该项目将（1）开发新的基于图的方法来有效地整合来自多个来源的信息，这在智能家居和智慧城市等许多应用领域很常见，并寻求将新方法分发到本地中心的方法，以避免分布式系统中原始数据的过度传输； （2）从构建图的层面开发处理相关数据的处理方法，这比PI早期开发的循环块排列框架更有效； （3）开发一个新的框架，为基于图的方法提供分析功效近似值，即使对于高维数据和非欧几里得数据，该方法也适用于成百上千的样本量，从而促进研究人员在实际应用中做出更好的决策。 这些方法和理论的发展将有助于更好地理解来自不同领域的现代复杂数据序列，从而进一步推进对这些领域的重大科学问题的理解。 该项目中开发的工具将作为带有详细文档的开源软件包分发。 这将加强统计界与更广泛科学领域的研究人员之间的合作，并使数据分析程序更加透明。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

In this big data era, massive data sequences are collected in various scientific fields for studying complicated phenomena over time and space, including neuroscience, epidemiology, social science, computer vision, and astronomy. Change-point analysis is a crucial early step in analyzing these data sequences, such as, to raise an alarm when an abnormal event happens in online data monitoring, and to segment a long sequence into more homogeneous parts for follow-up studies. To accommodate modern applications, the ability to deal with high throughput data and data with complicated structures is becoming a necessity. Parametric methods usually cannot be applied to very high dimensions unless strong assumptions are made to avoid the estimation of a large number of nuisance parameters. This project focuses on developing non-parametric change-point detection methods that are free of strong assumptions and computationally scalable to high dimensional and complex data. This project provides students and researchers with exciting new research problems that have both statistical and scientific importance. The training component for undergraduate and graduate students will prepare new researchers with inter-disciplinary education.This project will develop a new scan statistic framework through a novel adaptation of graph-based methods. The PI has shown that the graph-based approaches scale to high-dimensional and non-Euclidean data, and allow for universal analytic permutation p-value approximations that is decoupled from application-specific modeling, facilitating their applications to large and complicated data sets. Despite the good properties of the graph-based methods, there are still some gaps between its current versions and many modern applications. This project aims to fill those important gaps. In particular, this project will (1) develop new graph-based approaches to effectively integrate information from multiple sources, which is common in many application areas, such as smart homes and smart cities, and seek ways to distribute the new approaches to local centers to avoid the excessive transmission of raw data in a distributed system; (2) develop treatments from the level of constructing the graph to deal with dependent data, which is more effective than a circular block permutation framework developed by the PI earlier; and (3) develop a new framework to provide analytic power approximations to the graph-based methods that kick in for sample sizes in hundreds and thousands even for high-dimensional data and non-Euclidean data, facilitating researchers to make better decisions in real applications. These methodological and theoretical developments will provide better understandings of modern complicated data sequences from diverse fields, which will further advance the understanding of major scientific problems in these fields. The tools developed in this project will be distributed as open source software packages with detailed documentations. This will enhance the collaboration between the statistics community and researchers from broader scientific fields, and make data analysis procedures more transparent.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.