Significance Based Procedures for Mining and Prediction of Large Data Sets

基于显着性的大数据集挖掘和预测程序

• 批准号: 0907177
• 负责人: Andrew Nobel
• 金额: $ 21万
• 依托单位: University of North Carolina at Chapel Hill
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2009
• 资助国家: 美国
• 起止时间: 2009-09-01 至 2013-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=0907177&HistoricalAwards=false
• 关键词: Significance; Based; Procedures; Mining; Prediction

Exploratory methods play a critical role in the understanding of large data sets, regardless of their origin, and are typically the first step in their analysis. The investigator is studying the development and use of exploratory, data-mining methods that identify patterns or regularities in high-dimensional data. The specific focus of his research is the problem of identifying sample-variable associations in large data sets that may arise from multiple measurement technologies. In the typical case where the data from an experiment are represented in the form of a rectangular matrix, sample-variable associations correspond to distinguished submatrices of the data matrix. The investigator is developing a statistically principled, significance-based approach to the problem of finding large average submatrices of a data matrix, using a simple iterative algorithm. The algorithm is applicable to real-valued and categorical data matrices. In addition to the basic method, the investigator is developing several extensions, including data-driven null models that incorporate dependence between variables, data arising from the simultaneous application of multiple measurement technologies, and application of the basic method to prediction problems such as classification, regression and survival analysis. In addition, the investigator is developing basic theory to support the use of the algorithm, and to assess the structure of data matrices under the different null models. The development and application of the methods is being carried out in close collaboration with several groups of biomedical researchers. In particular, the new data mining methodology is being incorporated into software that is used by collaborating scientists to identify and assess significant sample-variable associations in ongoing experiments involving breast, brain and lung cancer.Large data sets are now common in many experimental areas of science, and in particular gene-level studies of human diseases such as cancer. In such studies it is not unusual to encounter experiments containing from hundreds to thousands of samples, and tens of thousands to millions of measurements on each sample. Large data sets are part of a trend away from traditional hypothesis-driven scientific research towards data-driven research, in which researchers explore large data sets for patterns or regularities that, in conjunction with subject matter expertise, yield hypotheses that can be tested by more traditional means. The investigator is studying an exploratory method that identifies statistically significant associations between samples and variables in large data sets, associations that can yield testable scientific hypotheses. The methods being developed by the investigator are computationally efficient, and are based on established statistical principles, in particular the notion of statistical significance. The investigator is also studying ways in which the basic exploratory method can be applied to data arising from multiple measurement technologies, and application of the basic method to statistical problems such as classification and survival analysis. These activities are being carried out as part of a collaborative research program involving the sustained interactions of faculty and students from the statistical, biological, and medical sciences. The exploratory method developed by the investigator is being integrated into the basic exploratory tools of the collaborating scientists, and is a component in the analysis of several new, previously unanalyzed, data sets.

探索性方法在理解大型数据集方面发挥着关键作用，无论其来源如何，并且通常是其分析的第一步。研究人员正在研究探索性数据挖掘方法的开发和使用，这些方法可以识别高维数据中的模式或规律。他的研究的具体重点是在可能由多种测量技术产生的大数据集中识别样本变量关联的问题。在实验数据以矩形矩阵的形式表示的典型情况下，样本变量关联对应于数据矩阵的区别子矩阵。研究人员正在开发一种统计原理的、基于重要性的方法来寻找数据矩阵的大平均子矩阵，使用一种简单的迭代算法。该算法适用于实值数据矩阵和分类数据矩阵。除了基本方法外，研究人员还在开发几个扩展，包括包含变量之间相关性的数据驱动零模型、同时应用多种测量技术产生的数据，以及将基本方法应用于分类、回归和生存分析等预测问题。此外，研究人员正在发展基础理论，以支持算法的使用，并评估不同零模型下的数据矩阵结构。这些方法的开发和应用是与几个生物医学研究小组密切合作进行的。特别是，新的数据挖掘方法正被纳入合作科学家使用的软件中，以在涉及乳腺癌、脑癌和肺癌的正在进行的实验中识别和评估重要的样本变量关联。现在，大型数据集在许多科学实验领域很常见，特别是对癌症等人类疾病的基因水平研究。在这样的研究中，遇到包含数百到数千个样本的实验，以及每个样本上的数万到数百万个测量值的实验并不少见。大数据集是从传统的假设驱动型科学研究转向数据驱动型研究的趋势的一部分，在这种趋势中，研究人员探索大数据集以寻找模式或规律，与主题专业知识一起产生可以用更传统的方法检验的假设。这位研究人员正在研究一种探索性方法，该方法可以在大型数据集中确定样本和变量之间具有统计学意义的关联，这种关联可以产生可检验的科学假设。研究人员正在开发的方法在计算上是有效的，并且基于既定的统计原则，特别是统计意义的概念。研究人员还在研究如何将基本探索性方法应用于来自多种测量技术的数据，以及将基本方法应用于分类和生存分析等统计问题。这些活动是作为合作研究计划的一部分进行的，该计划涉及统计、生物和医学科学的教职员工和学生之间的持续互动。研究人员开发的探索方法正在被整合到合作科学家的基本探索工具中，并是对几个以前未分析的新数据集进行分析的组成部分。