Collaborative Research:CIF:Small:Fisher-Inspired Approach to Quickest Change Detection for Score-Based Models

检测在线数据流的潜在统计特征的突变是许多科学和工程应用中经常遇到的一个重要问题。示例包括使用视频流的异常检测、电网线路中断检测、流行病发作检测以及网络攻击检测。虽然传统技术假设变化前后的概率分布是已知的或可以找到的，但这种假设在大多数实际情况下是不现实的。作为传统变更检测方法的替代方案，本项目考虑使用深度神经网络来实现变更检测。然而，该项目不是试图直接学习概率分布，而是利用最近展示的深度神经网络从数据中学习“分数”（即概率密度的对数梯度）的能力，旨在开发基于分数的变化检测算法。这些分数可以通过使用现代人工智能工具来学习大量高维数据模型，并将开发的算法应用于广泛的变化检测问题。该项目将发展基础数学理论，以确定所提出方法的有效性和效率，并将在几个公开可用的机器学习和异常检测数据集上验证所开发的算法。该项目的更广泛影响方面包括为更广泛的社区提供算法，以解决跨多个不同领域的变化和异常检测问题，以及将研究整合到研究生课程中的活动，并为未被充分代表的学生提供参与项目的机会。项目中开发的算法将基于数据的分数；该分数可以为已知的非归一化模型显式导出，也可以使用人工神经网络使用分数匹配来学习，并且开发的算法将被优化，以尽可能小的延迟检测变化，同时避免误报。该项目分为四个技术重点。第一个重点将发展基于分数的快速变化检测的基本理论，用于贝叶斯、广义贝叶斯和极大极小问题公式下的独立和相同分布的单流数据。经典的变化检测方法的性能取决于变化前后分布之间的Kullback-Leibler距离，而基于分数的方法的性能取决于分布之间的Fisher距离。第二个推力将开发健壮的方法来检测建模不确定性下的变化，使用不确定性类元素之间的费雪距离。第三个要点将为相关数据序列定义分数的概念，并获得检测变化的最佳算法，在这种情况下，分数基于条件密度的对数梯度。第四个也是最后一个重点将开发用于分布式变更检测的算法，其中多个代理可能对分布有部分了解，并且可能只与地理区域内的邻居通信。 该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Detecting abrupt changes in the underlying statistical characteristics of online data streams is an important problem commonly encountered in many science and engineering applications. Examples include anomaly detection using video streams, line-outage detection in power grids, onset detection of a pandemic, and detection of cyber-attacks. While traditional techniques assume that probability distributions for both before and after the change are known or can be found, this assumption is unrealistic in most scenarios of practical interest. As an alternative to such traditional change-detection approaches, this project considers the use of deep neural networks to effectuate change detection. However, rather than attempting to learn probability distributions directly, the project leverages the recently-demonstrated ability of deep neural networks to learn the "score" (i.e., the gradient of logarithm of the probability density) from the data and aims to develop score-based algorithms for change detection. These scores can be learned for a large class of high-dimensional data models using modern tools of artificial intelligence and rendering the developed algorithms applicable to a broad class of change-detection problems. Fundamental mathematical theories will be developed in the project to establish the efficacy and efficiency of the proposed methods, and the developed algorithms will be validated on several publicly available machine-learning and anomaly-detection datasets. Broader-impact aspects of the project include providing algorithms to the wider community for solving change- and anomaly-detection problems across multiple, disparate fields as well as activities centered on integrating research into graduate coursework and providing opportunities for underrepresented students to participate in the project. The algorithms developed in the project will be based on the score of the data; this score can be explicitly derived for known unnormalized models or can be learned using score matching using an artificial neural network, and developed algorithms will be optimized to detect the changes with the minimum possible delay while avoiding false alarms. The project is divided into four technical thrusts. The first thrust will develop the fundamental theory for score-based quickest change detection for independent and identically distributed single-stream data under Bayesian, generalized Bayesian, and minimax problem formulations. While the performance of classical change-detection methods depends on the Kullback-Leibler distance between the distributions before and after the change, it will be established that the performance of the score-based methods depends on the Fisher distance between distributions. The second thrust will develop robust methods for detecting changes under modeling uncertainty, using the Fisher distance between the elements of the uncertainty classes. The third thrust will define the notion of scores for dependent data sequences and obtain optimal algorithms for detecting changes, with the scores in this case being based on the gradient of the logarithm of the conditional densities. The fourth and final thrust will develop algorithms for distributed change detection wherein multiple agents may have partial knowledge of the distributions and may only communicate with their neighbors in a geographical area. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.