Offline and Online Change-point Analysis for Large-scale Time Series Data

大规模时间序列数据的离线和在线变点分析

• 批准号: 1916239
• 负责人: Jun Li
• 金额: $ 10万
• 依托单位: Kent State University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-09-01 至 2023-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1916239&HistoricalAwards=false
• 关键词: Offline; Online; Change; point; Analysis

Offline or online time series data often involve change points due to the dynamic behavior of the monitored systems. Identifying change points from offline time series data makes parameter estimation and statistical inference efficient by pooling homogeneous observations. Detection of change points from online time series data provides timely snapshots of the monitored system and allows for real-time anomaly detection. Despite its importance, methods available for detecting change points in large-scale offline and online time series data are scarce. This is because a large number of parameters cannot be estimated accurately with a limited number of observations, and parametric models do not fully capture multifarious aspects of data dependence. This project will develop new non-parametric change-point detection methods that incorporate both spatial and temporal dependence without imposing restrictive structural assumptions on large-scale time series data. The proposed methods will span a wide range of topics in applications, including identifying significant genes associated with certain diseases, studying dynamic functional connectivity in resting-state functional magnetic resonance imaging data, and detecting abrupt events such as dissociation of communities, or formation of new communities from social networking platforms. This project will integrate research and education by involving students at different levels, including those from underrepresented groups, and by training the pre-college and high school teachers to improve their knowledge in statistics through new developed courses. The developed methods will be disseminated to biomedical and social scientists through interdisciplinary collaborations and the analysis of first-hand datasets. This project will develop a general factor model framework for spatial and temporal dependence of large-scale time series data. By integrating the framework, this project will provide hypothesis testing and offline change-point estimation of specific parameters, including the population mean and covariance matrix. The proposed methods can be readily modified to incorporate the advantages of both sum-of-squares-norm and max-norm statistics for hypothesis testing. They can be extended from regular binary segmentation methods to other popular change-point estimation methods, such as circular binary segmentation and wild binary segmentation. This project will also provide new stopping rules for online change-point detection of large-scale time series data. An explicit expression for the average run length (ARL) will be derived, so that the level of threshold in stopping rules can be easily obtained with no need to run time-consuming Monte Carlo simulations. The proposed research will derive an upper bound for the expected detection delay (EDD), the expression of which clearly demonstrates the impact of data dimensionality and dependence. This project will extend the current knowledge about change-point detection. For offline change-point detection, the PI will study the possibility of estimating the change point near the boundary in high dimensional settings. For online change-point detection, a comparison will be made between the stopping rule based on the sum-of-squares-norm statistic and the one based on the max-norm statistic, through the derived ARLs and EDDs.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

由于受监控系统的动态行为，离线或在线时间序列数据通常涉及变化点。从离线时间序列数据中识别变化点，通过汇集同类观测数据，使参数估计和统计推断变得高效。从在线时间序列数据中检测变化点可提供被监控系统的及时快照，并允许实时异常检测。尽管它很重要，但可用于检测大规模离线和在线时间序列数据中的变化点的方法很少。这是因为大量的参数不能用有限的观测值来准确估计，而且参数模型不能完全捕捉数据相关性的各种方面。这个项目将开发新的非参数变点检测方法，既包括空间相关性，也包括时间相关性，而不会对大规模时间序列数据施加限制性的结构性假设。提出的方法将涵盖广泛的应用主题，包括识别与某些疾病相关的重要基因，研究静息状态功能磁共振成像数据中的动态功能连通性，以及从社交网络平台检测突然事件，如社区分离或新社区的形成。该项目将把研究和教育结合起来，让不同级别的学生参与进来，包括那些来自代表性不足的群体的学生，并通过新开发的课程培训大学预科和高中教师提高他们在统计学方面的知识。开发的方法将通过跨学科合作和对第一手数据集的分析传播给生物医学和社会科学家。该项目将为大规模时间序列数据的空间和时间相关性开发一个通用因素模型框架。通过整合该框架，该项目将提供具体参数的假设检验和离线变点估计，包括总体均值和协方差矩阵。所提出的方法可以很容易地进行修改，以结合平方和范数和最大范数统计量用于假设检验的优点。它们可以从常规的二值分割方法扩展到其他流行的变点估计方法，如圆形二值分割和野二值分割。该项目还将为大规模时间序列数据的在线变点检测提供新的停止规则。推导出平均行程长度(ARL)的显式表达式，从而无需运行耗时的蒙特卡罗模拟，即可容易地获得停止规则中的阈值水平。该研究将推导出期望检测延迟(EDD)的上界，该上界的表达式清楚地表明了数据维度和相关性的影响。该项目将扩展目前关于变点检测的知识。对于离线变化点检测，PI将研究在高维设置中估计边界附近变化点的可能性。对于在线变点检测，将通过推导出的ARL和EDD，在基于平方和范数统计量的停止规则和基于最大范数统计量的停止规则之间进行比较。这一裁决反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Finite sample t-tests for high-dimensional means

• DOI: 10.1016/j.jmva.2023.105183
• 发表时间: 2022-03
• 期刊: Journal of multivariate analysis
• 影响因子: 1.6
• 作者: Jun Li

Online Change-Point Detection in High-Dimensional Covariance Structure with Application to Dynamic Networks

• 发表时间: 2019-11
• 期刊: J. Mach. Learn. Res.
• 作者: Lingjun Li;Jun Li

Multivariate analysis of variance and change points estimation for high‐dimensional longitudinal data

• DOI: 10.1111/sjos.12460
• 发表时间: 2020-04
• 期刊: Scandinavian Journal of Statistics
• 影响因子: 1
• 作者: Pingshou Zhong;Jun Li;P. Kokoszka