Collaborative Research: Learning and forecasting high-dimensional extremes: sparsity, causality, privacy

这项研究项目的主要目标是学习如何预测未来的极端观测并评估其影响。几乎每天都会发生这样一种情况，公众被与极端观测有关的新闻报道淹没，这些极端观测来自异常气候事件，从长期和严重的干旱到异常降水记录，再到记录热浪，这些热浪以这样或那样的形式几乎到达了美国的每一个地区。这些极端事件出人意料地出现，可能是危险的，可能是巧合也可能不是巧合。随着全球气温的升高，热带风暴是否会变得更加致命？随着经济状况的恶化，极端暴力是否会变得更加普遍？气候科学家和社会科学家分别研究这类问题，但对极值的统计和概率分析是任何分析中不可或缺的组成部分。现代的极端统计分析既因可用数据量的泛滥而受益，也因这种泛滥而受到诅咒。可用的数据往往是高维的和受污染的。需要对未来的极端情况进行快速预测和相应的政策更新，这就需要对极端情况进行在线分析。这项研究旨在从一系列潜在的变量中评估各种因素对极端事件的频率和规模的因果影响，这些变量包括不断变化的环境条件、美国境内的人口变动、不断变化的地形和不断变化的经济状况。从许多变量中，希望能产生方法来提取数据中对描述和预测极端情况有直接影响的重要特征。这项研究还围绕差异隐私的概念，旨在开发工具来发布数据集的全局特征，而不会泄露个人级别的信息。这项研究的重点将是开发针对大型数据集的极值特征量身定做的差异隐私程序，这是具有挑战性的，因为极端观察恰恰是揭示最个人化信息的观察。这一研究项目的首要目标是使现代统计学习工具适用于预测极端情况的问题。由于极值数据的数量有限和极值标签的稀缺，学习极值的结构是一个困难的挑战。一种方法是开发用于检测更小维度的非线性集的方法，该方法可以提供对高维中的极值的充分描述。这项研究的一个主要目的是发展强大的现代学习技术(如基于图的学习方法和核主成分分析)，使人们能够从数据中确定最大支持度。这项研究的第二个主旨集中在小维度和大维度问题中的因果关系问题上。在最基本的形式中，如果X的某些变化(有时本身是极端的，但并不总是如此)影响Y的尾部行为，则一组变量X被称为对依赖向量Y的尾部因果关系。极端事件因果关系的潜在结果框架将是本提案研究议程中的主要焦点。这项研究的第三个主旨是关于极端情况下的差异隐私，它提供了在不泄露个人级别信息的情况下释放数据集的全局特征的工具。这是通过在发布之前修改数据来实现的，特别是通过随机化的方式，使程序的输出不太依赖于任何特定的观察，同时仍然允许对原始数据集的某些特征进行统计推断。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

The principal goal of this research project is to learn how to forecast future extreme observations and to assess their impact. On an almost daily occurrence, the public is inundated with news accounts related to extreme observations arising from extraordinary climatic events from extended and severe droughts to extraordinary precipitation records, to record heat waves that have reached virtually every region of the US in one form or another. These extreme events appear unexpectedly, can be dangerous and occur in combinations that may or may not be coincidental. Do tropical storms become more deadly as global temperatures rise? Does extreme violence become more widespread as the economic conditions worsen? Questions of this type are studied by climate scientists and social scientists respectively, but statistical and probabilistic analysis of extreme values is an indispensable ingredient in any analysis. Modern statistical analysis of extremes is both blessed by the deluge of the amount of available data and cursed by this deluge. The available data are often high dimensional and contaminated. The necessity of quick forecast of future extremes and corresponding policy updates require online analysis of extremes. This research aims to evaluate causal impacts of various factors from a potentially large array of variables including changing environmental conditions, demographic movements within the US, changing landscapes, and changing economic conditions, on the frequency and magnitude of extreme events. From many variables, the hope is to produce methodology to extract the important features in the data that have a direct impact on describing and predicting extremes. This research also revolves around the notion of differential privacy and aims to develop tools for releasing global characteristics of a data set without revealing individual level information. The focus of this research will be related to developing differential privacy procedures that are tailored to extreme value characteristics of large data sets, which is challenging because extreme observations are precisely the ones that reveal the most individual information. An overarching objective of this research project is to adapt modern statistical learning tools to the problem of forecasting extremes. Learning the structure of extremes presents difficult challenges due to both a limited number of extreme data and to the scarcity of extremal labels. One approach is to develop methods for detecting nonlinear sets of much smaller dimension that can provide an adequate description of extremes in high dimensions. A main thrust of this research is to develop powerful modern learning techniques (such as graph-based learning methods and kernel principal component analysis) that allow one to determine the extremal support from the data. A second main thrust of this research centers on the issue of causality in both small and large dimensional problems. In the most basic form, a set of variables X is said to be tail causal to a dependent vector Y if certain changes in X (sometimes themselves extreme but not always so) impact the tail behavior of Y. The potential outcomes framework for causality of extreme events will be a major focus in this proposal’s research agenda. A third main thrust of this research is about differential privacy in the context of extremes, which provides tools for releasing global characteristics of a data set without revealing individual level information. This is achieved by modifying the data before releasing it and, in particular, randomizing it, in such a way that the output of the procedure does not depend too much on any specific observation while still allowing for statistical inference for certain characteristics of the original data set.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.