Collaborative Research: Learning and forecasting high-dimensional extremes: sparsity, causality, privacy

该研究项目的主要目标是学习如何预测未来的极端观测并评估其影响。几乎每天都有，公众都被淹没在与极端观测相关的新闻报道中，这些极端观测来自于从长期严重干旱到异常降水记录的异常气候事件，以记录以某种形式到达美国几乎每个地区的热浪。这些极端事件出乎意料地出现，可能是危险的，可能是巧合，也可能不是巧合。随着全球气温的上升，热带风暴会变得更加致命吗？随着经济状况的恶化，极端暴力是否会变得更加普遍？这类问题分别由气候科学家和社会科学家研究，但对极端值的统计和概率分析是任何分析中不可或缺的组成部分。现代的极端统计分析既受到大量可用数据的祝福，也受到这种洪水的诅咒。现有的数据往往是高维和污染。快速预测未来极端事件和相应政策更新的必要性要求对极端事件进行在线分析。这项研究旨在评估各种因素的因果影响，这些因素来自一系列潜在的变量，包括不断变化的环境条件，美国境内的人口流动，不断变化的景观和不断变化的经济条件，对极端事件的频率和幅度。从许多变量中，希望能够产生方法来提取数据中对描述和预测极端情况有直接影响的重要特征。这项研究还围绕着差异隐私的概念，旨在开发工具，发布全球的数据集的特征，而不透露个人层面的信息。这项研究的重点将与开发针对大型数据集的极值特征量身定制的差分隐私程序有关，这是具有挑战性的，因为极端观测恰恰是揭示最多个人信息的观测。该研究项目的总体目标是使现代统计学习工具适应极端预测问题。学习极端的结构提出了困难的挑战，由于数量有限的极端数据和极端标签的稀缺性。一种方法是开发用于检测小得多的维度的非线性集合的方法，所述非线性集合可以提供对高维度中的极端的充分描述。这项研究的一个主要目标是开发强大的现代学习技术（如基于图的学习方法和核主成分分析），允许从数据中确定极值支持。本研究的第二个主要重点集中在小维度和大维度问题中的因果关系问题上。在最基本的形式中，如果X的某些变化（有时本身极端，但不总是如此）影响Y的尾部行为，则一组变量X被称为是依赖向量Y的尾部因果。极端事件因果关系的潜在结果框架将是本提案研究议程的一个主要重点。本研究的第三个主要目标是关于极端情况下的差异隐私，它提供了在不泄露个人信息的情况下发布数据集的全局特征的工具。这是通过在发布数据之前修改数据来实现的，特别是将其随机化，该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的知识价值进行评估，被认为值得支持。影响审查标准。

The principal goal of this research project is to learn how to forecast future extreme observations and to assess their impact. On an almost daily occurrence, the public is inundated with news accounts related to extreme observations arising from extraordinary climatic events from extended and severe droughts to extraordinary precipitation records, to record heat waves that have reached virtually every region of the US in one form or another. These extreme events appear unexpectedly, can be dangerous and occur in combinations that may or may not be coincidental. Do tropical storms become more deadly as global temperatures rise? Does extreme violence become more widespread as the economic conditions worsen? Questions of this type are studied by climate scientists and social scientists respectively, but statistical and probabilistic analysis of extreme values is an indispensable ingredient in any analysis. Modern statistical analysis of extremes is both blessed by the deluge of the amount of available data and cursed by this deluge. The available data are often high dimensional and contaminated. The necessity of quick forecast of future extremes and corresponding policy updates require online analysis of extremes. This research aims to evaluate causal impacts of various factors from a potentially large array of variables including changing environmental conditions, demographic movements within the US, changing landscapes, and changing economic conditions, on the frequency and magnitude of extreme events. From many variables, the hope is to produce methodology to extract the important features in the data that have a direct impact on describing and predicting extremes. This research also revolves around the notion of differential privacy and aims to develop tools for releasing global characteristics of a data set without revealing individual level information. The focus of this research will be related to developing differential privacy procedures that are tailored to extreme value characteristics of large data sets, which is challenging because extreme observations are precisely the ones that reveal the most individual information. An overarching objective of this research project is to adapt modern statistical learning tools to the problem of forecasting extremes. Learning the structure of extremes presents difficult challenges due to both a limited number of extreme data and to the scarcity of extremal labels. One approach is to develop methods for detecting nonlinear sets of much smaller dimension that can provide an adequate description of extremes in high dimensions. A main thrust of this research is to develop powerful modern learning techniques (such as graph-based learning methods and kernel principal component analysis) that allow one to determine the extremal support from the data. A second main thrust of this research centers on the issue of causality in both small and large dimensional problems. In the most basic form, a set of variables X is said to be tail causal to a dependent vector Y if certain changes in X (sometimes themselves extreme but not always so) impact the tail behavior of Y. The potential outcomes framework for causality of extreme events will be a major focus in this proposal’s research agenda. A third main thrust of this research is about differential privacy in the context of extremes, which provides tools for releasing global characteristics of a data set without revealing individual level information. This is achieved by modifying the data before releasing it and, in particular, randomizing it, in such a way that the output of the procedure does not depend too much on any specific observation while still allowing for statistical inference for certain characteristics of the original data set.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.