BIGDATA: F: Statistical Foundation of Predictivity: A Novel Architecture for Big Data Learning

识别有利于预测的变量，特别是在大数据的背景下，是一个重要的挑战。科学文献目前缺乏直接将变量集的潜在预测能力（称为“预测性”）作为需要估计的参数的研究。该项目旨在为可预测性的测量奠定统计基础，并提出一个在大数据学习中最大化可预测性的新框架。该研究包括将大数据应用于城市规划，解决纽约市“零愿景”项目中的预测问题。与纽约市交通部合作，PI及其团队将识别与交通事故及其后果相关的风险因素及其组合，并改进事故预防和受害者结果预测。提出了一种新的基于样本的预测性度量，即I-score，它可以有效地区分大数据中的噪声变量和预测变量。这一措施可以与正确预测率的下界有关。在这个I-score的指导下，可以识别出具有高潜在预测性的变量集。这种高预测性通常存在于变量之间复杂的相互作用中。为了充分利用已识别变量集的预测性，将构建基于深度体系结构的强大分类器。提出了新的策略来实现所提出的框架的可扩展计算。将使用模拟和基准真实数据集对所提出的方法进行系统评估，并与当前策略进行比较。

Identifying variables that are good for prediction, especially in the context of BIG DATA, is an important challenge. The scientific literature currently lacks research that directly considers a variable set's potential ability to predict, referred to as "predictivity", as a parameter to be estimated. This project sets out to lay down statistical foundations for measures of predictivity, and proposes a novel framework for maximizing predictivity in big data learning. The research includes an application to big data in urban planning, addressing prediction problems in New York City's Vision Zero project. In collaboration with the NYC Department of Transportation, the PI and his team will identify risk factors and their combinations that are associated with traffic accidents and their outcomes, and improve accident prevention and victim outcome prediction. A novel sample-based measure of predictivity, the I-score, that is effective in differentiating between noisy and predictive variables in big data is proposed. This measure can be related to a lower bound for the correct prediction rate. Guided by this I-score, variable sets of high potential predictivity can be identified. This high predictivity often resides within complex interactions among the variables. To fully leverage the predictivity in an identified variable set, powerful classifiers based on deep architectures will be constructed. Novel strategies are proposed for scalable computational implementation of the proposed framework. Systematic evaluation of the proposed methods, comparing with current strategies, will be carried out using simulations and benchmark real data sets.