CAREER: New Change-Point Problems in Analyzing High-Dimensional and Non-Euclidean Data

在这个大数据时代，海量数据序列被收集在各个科学领域，用于研究时间和空间上的复杂现象，包括神经科学、流行病学、社会科学、计算机视觉和天文学。变点分析是分析这些数据序列的关键早期步骤，例如，在在线数据监测中发生异常事件时发出警报，并将长序列分割为更均匀的部分，以便后续研究。为了适应现代应用，处理高吞吐量数据和具有复杂结构的数据的能力变得必不可少。参数方法通常不能应用于非常高的维度，除非做出强有力的假设以避免估计大量的有害参数。这个项目的重点是开发非参数变点检测方法，这些方法不需要强烈的假设，并且可以计算扩展到高维和复杂的数据。这个项目为学生和研究人员提供了令人兴奋的具有统计学和科学重要性的新研究问题。针对本科生和研究生的培训部分将为具有跨学科教育的新研究人员做准备。该项目将通过一种新的基于图表的方法的改编来开发一个新的扫描统计框架。PI表明，基于图的方法可扩展到高维和非欧几里德数据，并允许从特定应用的建模中分离出来的通用解析置换p值近似，便于将其应用于大型和复杂的数据集。尽管基于图的方法具有很好的性质，但它的现有版本与许多现代应用程序之间仍有一些差距。该项目旨在填补这些重要空白。特别是，本项目将(1)开发新的基于图的方法来有效地整合来自多个来源的信息，这在许多应用领域是常见的，如智能家居和智能城市，并寻求将新方法分发到本地中心，以避免分布式系统中原始数据的过度传输；(2)从构建图的层面开发处理依赖数据的方法，这比PI早先开发的圆形块置换框架更有效；以及(3)开发一种新的框架，为基于图的方法提供分析能力近似，即使对于高维数据和非欧几里德数据，这些方法也会产生成百上千的样本大小，帮助研究人员在实际应用中做出更好的决策。这些方法和理论的发展将使人们更好地理解来自不同领域的现代复杂数据序列，这将进一步促进对这些领域重大科学问题的理解。在这个项目中开发的工具将作为开放源码软件包分发，并附有详细的文档。这将加强统计界和更广泛科学领域的研究人员之间的合作，并使数据分析程序更加透明。这一奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

In this big data era, massive data sequences are collected in various scientific fields for studying complicated phenomena over time and space, including neuroscience, epidemiology, social science, computer vision, and astronomy. Change-point analysis is a crucial early step in analyzing these data sequences, such as, to raise an alarm when an abnormal event happens in online data monitoring, and to segment a long sequence into more homogeneous parts for follow-up studies. To accommodate modern applications, the ability to deal with high throughput data and data with complicated structures is becoming a necessity. Parametric methods usually cannot be applied to very high dimensions unless strong assumptions are made to avoid the estimation of a large number of nuisance parameters. This project focuses on developing non-parametric change-point detection methods that are free of strong assumptions and computationally scalable to high dimensional and complex data. This project provides students and researchers with exciting new research problems that have both statistical and scientific importance. The training component for undergraduate and graduate students will prepare new researchers with inter-disciplinary education.This project will develop a new scan statistic framework through a novel adaptation of graph-based methods. The PI has shown that the graph-based approaches scale to high-dimensional and non-Euclidean data, and allow for universal analytic permutation p-value approximations that is decoupled from application-specific modeling, facilitating their applications to large and complicated data sets. Despite the good properties of the graph-based methods, there are still some gaps between its current versions and many modern applications. This project aims to fill those important gaps. In particular, this project will (1) develop new graph-based approaches to effectively integrate information from multiple sources, which is common in many application areas, such as smart homes and smart cities, and seek ways to distribute the new approaches to local centers to avoid the excessive transmission of raw data in a distributed system; (2) develop treatments from the level of constructing the graph to deal with dependent data, which is more effective than a circular block permutation framework developed by the PI earlier; and (3) develop a new framework to provide analytic power approximations to the graph-based methods that kick in for sample sizes in hundreds and thousands even for high-dimensional data and non-Euclidean data, facilitating researchers to make better decisions in real applications. These methodological and theoretical developments will provide better understandings of modern complicated data sequences from diverse fields, which will further advance the understanding of major scientific problems in these fields. The tools developed in this project will be distributed as open source software packages with detailed documentations. This will enhance the collaboration between the statistics community and researchers from broader scientific fields, and make data analysis procedures more transparent.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.