III: Small: Mining Local Correlations in Extremely High-Dimensional Data: Models, Algorithms, and Applications

该项目开发了在极高维度数据中挖掘局部潜在相关性的计算和统计原理。实验技术的最新进展使收集极高维度的数据成为可能。例子包括基因表达、遗传变异、蛋白质和DNA序列数据。分析此类数据的一个关键问题是发现特征之间潜在的局部相关性。这种局部相关性只存在于特征子空间中，并且可能涉及两个以上的特征。数据的大量特征和噪声特征使得建模、识别和评估这种局部相关性的统计显著性成为一个具有挑战性的研究问题。该项目旨在开发工具，使用户能够高效地挖掘和探索本地相关性。它寻求开发(1)有效的模型来捕捉特征之间的局部相关性；(2)从极高维度数据中识别局部相关性的可扩展算法；(3)稳健的方法来评估识别相关性的统计显著性。所提出的方法结合了降维、固有维数估计、信息论方法和假设检验的优点，用于建模和识别显著的局部相关性。由此产生的工具将帮助许多学科的科学家，包括研究基因功能的生物学家和了解疾病进展并寻找新的有效治疗方法的医生。研究成果将发表在同行评议的数据挖掘和生物信息学期刊和会议上，并纳入CWRU的教育和推广计划。项目网站（http://engr.case.edu/zhang_xiang）将用于传播研究成果，包括出版物、数据和软件。

This project develops the computational and statistical principles of mining local latent correlations in extremely high-dimensional data. Recent advances in experimental technologies have rendered it possible to collect data of extremely high dimensionality. Examples include gene expression, genetic variation, and protein and DNA sequence data. A key problem in analyzing such data is finding latent local correlations among features. Such local correlations only exist in feature subspaces and may involve more than two features. The large number of features and the noisy characteristics of the data make modeling, identifying, and assessing the statistical significance of such local correlations a challenging research problem. The project aims to develop tools that enable users to mine and explore local correlations efficiently and effectively. It seeks to develop (1) effective models to capture the local correlations among features; (2) scalable algorithms to identify local correlations from extremely high-dimensional data; (3) robust methods to assess the statistical significance of the identified correlations. The proposed methods combine the advantages of dimension reduction, intrinsic dimensionality estimation, information theoretic approach, and hypothesis testing for modeling and identifying significant local correlations. The resuling tools will assist scientists in many disciplines including biologists in their study of gene function and medical doctors in their understanding of disease progression and searching for new and effective treatments. The research results will be published in peer reviewed data mining and bioinformatics journals and conferences and integrated into the educational and outreach programs at CWRU. The project Web site (http://engr.case.edu/zhang_xiang) will be used for dissemination of research results including publications, data, and software.