A Novel Statistical Framework for Big Data Prediction

全基因组关联研究（GWAS）的最新进展导致可用遗传数据的规模增加，并确定了导致各种疾病的重要遗传变异。 对这些遗传疾病的预测也变得至关重要。 然而，对GWAS等大数据的预测并不是微不足道的。 大数据预测中的一个关键障碍是识别（可能是少量的）变量集，当变量维度非常大时，这些变量集可以导致良好的预测。 该项目探讨了为什么一种常见的预测方法往往无法提供强大的预测率。 一种新的，基于交互和面向预测的方法来提取隐藏在大数据中的信息将被调查。 为了改进预测，将制定一个新的标准来指导选择变量集，优先考虑预测性，而不是重要性，需要使用正确的预测率估计数，并制定基于预测性的标准来评价变量集。 该项目提供了一个新的理论框架，描述了高度预测的变量集的特征，并为识别这些集的新标准提供了基础性工作。 在本研究项目的框架内，变量集具有理论（真实）水平的预测性，可以通过适当设计的基于样本的措施进行估计。 这个框架是第一个寻求开发特定于预测性标准的估计量的框架。 此外，将调查包含边际效应和联合效应的方法，并研究预测性的候选指标。 四个真实的数据的例子进行了分析，以说明如何通过新的方法相比，在目前的文献中的其他方法找到最终的预测。

Recent advances in genome-wide association studies (GWAS) have led to both an increase in the size of genetic data available and identification of important genetic variants responsible for a variety of diseases. Prediction for these genetic diseases has also become of paramount importance. However, prediction for big data such as GWAS is not trivial. A key obstacle in big data prediction is identifying (perhaps a small number of) variable sets that lead to good prediction when variable dimensionality can be extremely large. The project explores why a common approach towards prediction can often fail to deliver strong prediction rates. A novel, interaction-based and prediction-oriented approach to extracting hidden information contained in big data will be investigated. To improve prediction, a new criterion to guide the selection of variable sets will be developed.Prioritizing predictivity, not significance, requires using the correct estimates of prediction rates and developing predictivity-based criteria to evaluate variable sets. The project offers a novel theoretical framework by characterizing what makes for highly predictive variable sets, and providing fundamental work towards a new criterion to identify these sets. In the framework of this research project, variable sets have theoretical (true) levels of predictivity, which can be estimated with appropriately designed sample-based measures. This framework is the first that seeks to develop estimators specific to a criterion of predictivity. Additionally, methods that encompass both marginal and joint effects will be investigated, and a candidate measure of predictivity will be studied. Four real data examples are analyzed to illustrate how final predictors found via the new approach compare to other approaches in the current literature.