Collaborative Research: Learning and forecasting high-dimensional extremes: sparsity, causality, privacy

这个研究项目的主要目标是学习如何预测未来的极端观测并评估其影响。公众几乎每天都被淹没在与极端气候事件有关的新闻报道中，从持续严重的干旱到异常的降水记录，再到以这样或那样的形式到达美国几乎每个地区的热浪记录。这些极端事件的出现出乎意料，可能是危险的，并且可能是巧合，也可能不是巧合。随着全球气温上升，热带风暴是否会变得更加致命？随着经济状况的恶化，极端暴力是否变得更加普遍？这类问题分别由气候科学家和社会科学家研究，但极端值的统计和概率分析是任何分析中不可或缺的组成部分。现代极端数据的统计分析既受到海量可用数据的祝福，也受到海量数据的诅咒。可用的数据通常是高维的和受污染的。对未来极端事件的快速预测和相应的政策更新的必要性要求对极端事件进行在线分析。本研究旨在评估各种因素的因果影响，这些因素来自潜在的大量变量，包括不断变化的环境条件、美国境内的人口流动、不断变化的景观和不断变化的经济条件，对极端事件的频率和幅度产生影响。从许多变量中，希望产生一种方法来提取数据中的重要特征，这些特征对描述和预测极端事件有直接影响。本研究还围绕差异隐私的概念展开，旨在开发工具，在不泄露个人层面信息的情况下发布数据集的全局特征。本研究的重点将是开发针对大型数据集的极值特征量身定制的差异隐私程序，这是具有挑战性的，因为极值观察恰恰是揭示最个体信息的观察。这个研究项目的首要目标是使现代统计学习工具适应预测极端的问题。由于极值数据的数量有限和极值标签的稀缺性，学习极值结构提出了困难的挑战。一种方法是开发检测更小维度的非线性集的方法，这些方法可以提供对高维极值的充分描述。这项研究的主要目的是开发强大的现代学习技术（如基于图的学习方法和核主成分分析），使人们能够从数据中确定极端支持。本研究的第二个主要推力集中在小维度和大维度问题的因果关系问题上。在最基本的形式中，如果X的某些变化（有时本身是极端的，但并不总是如此）影响Y的尾部行为，则一组变量X被认为是依赖向量Y的尾部因果关系。极端事件因果关系的潜在结果框架将是本提案研究议程的主要焦点。本研究的第三个重点是关于极端背景下的差异隐私，它提供了在不泄露个人层面信息的情况下发布数据集的全局特征的工具。这是通过在释放数据之前修改数据，特别是随机化数据来实现的，这样一来，过程的输出不会过多地依赖于任何特定的观察结果，同时仍然允许对原始数据集的某些特征进行统计推断。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The principal goal of this research project is to learn how to forecast future extreme observations and to assess their impact. On an almost daily occurrence, the public is inundated with news accounts related to extreme observations arising from extraordinary climatic events from extended and severe droughts to extraordinary precipitation records, to record heat waves that have reached virtually every region of the US in one form or another. These extreme events appear unexpectedly, can be dangerous and occur in combinations that may or may not be coincidental. Do tropical storms become more deadly as global temperatures rise? Does extreme violence become more widespread as the economic conditions worsen? Questions of this type are studied by climate scientists and social scientists respectively, but statistical and probabilistic analysis of extreme values is an indispensable ingredient in any analysis. Modern statistical analysis of extremes is both blessed by the deluge of the amount of available data and cursed by this deluge. The available data are often high dimensional and contaminated. The necessity of quick forecast of future extremes and corresponding policy updates require online analysis of extremes. This research aims to evaluate causal impacts of various factors from a potentially large array of variables including changing environmental conditions, demographic movements within the US, changing landscapes, and changing economic conditions, on the frequency and magnitude of extreme events. From many variables, the hope is to produce methodology to extract the important features in the data that have a direct impact on describing and predicting extremes. This research also revolves around the notion of differential privacy and aims to develop tools for releasing global characteristics of a data set without revealing individual level information. The focus of this research will be related to developing differential privacy procedures that are tailored to extreme value characteristics of large data sets, which is challenging because extreme observations are precisely the ones that reveal the most individual information. An overarching objective of this research project is to adapt modern statistical learning tools to the problem of forecasting extremes. Learning the structure of extremes presents difficult challenges due to both a limited number of extreme data and to the scarcity of extremal labels. One approach is to develop methods for detecting nonlinear sets of much smaller dimension that can provide an adequate description of extremes in high dimensions. A main thrust of this research is to develop powerful modern learning techniques (such as graph-based learning methods and kernel principal component analysis) that allow one to determine the extremal support from the data. A second main thrust of this research centers on the issue of causality in both small and large dimensional problems. In the most basic form, a set of variables X is said to be tail causal to a dependent vector Y if certain changes in X (sometimes themselves extreme but not always so) impact the tail behavior of Y. The potential outcomes framework for causality of extreme events will be a major focus in this proposal’s research agenda. A third main thrust of this research is about differential privacy in the context of extremes, which provides tools for releasing global characteristics of a data set without revealing individual level information. This is achieved by modifying the data before releasing it and, in particular, randomizing it, in such a way that the output of the procedure does not depend too much on any specific observation while still allowing for statistical inference for certain characteristics of the original data set.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.