EAGER: Efficient Algorithms for Dimensionality Reduction and Clustering Using Disk-Based Matrices

EAGER：使用基于磁盘的矩阵进行降维和聚类的高效算法

• 批准号: 0937562
• 负责人: Carlos Ordonez
• 金额: $ 10万
• 依托单位: University of Houston
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2009
• 资助国家: 美国
• 起止时间: 2009-07-15 至 2011-06-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=0937562&HistoricalAwards=false
• 关键词: EAGER; Efficient; Algorithms; Dimensionality; Reduction

EAGER: Efficient Algorithms for Dimensionality Reduction and Clustering Using Disk-Based Matrices Carlos Ordonez1. Research and Education Proposal Linear Gaussian models on large data sets computation is characterized by heavy matrix manipulation and iterative methods with slow convergence. Efficiency issues become worse considering the fact that large data sets are stored and retrieved from disk and that in a database system models are manipulated on disk as well. Despite their importance there is scarce research work that attempts to adapt this big family of Gaussian models exploiting database systems techniques. This proposal studies how to improve algorithms for linear Gaussian models to analyze large, high dimensional, data sets, manipulating matrices on secondary storage (i.e. disk), using a small amount of primary storage (i.e. RAM memory). The models studied herein include maximum likelihood factor analysis for dimensionality reduction and mixtures of Gaussian distributions to perform clustering.The educational component of this proposal involves two main activities. The first activity is to develop a plan to expose disadvantaged and minority high school students to data mining research and practice in order to encourage them to study computer science. The second activity involves enhancing current research and teaching of data mining at the University of Houston.2. Intellectual MeritThis research project requires the discovery of common algorithmic principles to perform incremental matrix computations for a family of statistical models, understanding how to summarize large data sets, preserving their statistical properties required by multiple models and proposing new database techniques tailored for such models, capable of performing efficient matrix manipulation on secondary storage. Incremental computations are difficult to attain because methods for linear Gaussian models require iterations on the entire data set. Summarization requires transforming complex matrix equations considering high dimensionality, large data set size and numerical stability, preserving model accuracy. Developing matrix optimizations combining primary and secondary storage is quite different from optimizing a matrix algorithm that works only on primary storage.This research work requires mathematical knowledge to generalize, optimize and transform the computation of linear Gaussian models. On the other hand, it needs database systems expertise on how to organize and index diverse matrices on secondary storage for efficient reading and writing.3. Broader ImpactThis proposal will have a broad impact on the analysis of large, complex, high dimensional scientific data sets and enhancing database systems with incremental model computation capabilities. We plan to apply and test our proposed algorithms and techniques on scientific data sets, including geographical, medical and biological data sets, among others.

EAGER：Efficient Algorithms for Decisionality Reduction and Clustering Using Escherichia-Based Matrices卡洛斯奥多涅斯1.研究和教育建议大型数据集上的线性高斯模型计算的特点是繁重的矩阵运算和收敛缓慢的迭代方法。考虑到大型数据集是从磁盘存储和检索的，并且在数据库系统中模型也是在磁盘上操作的，效率问题变得更糟。尽管它们的重要性，有稀缺的研究工作，试图适应这个大家庭的高斯模型，利用数据库系统技术。该提案研究如何改进线性高斯模型的算法，以分析大型高维数据集，在二级存储器（即磁盘）上操作矩阵，使用少量的主存储器（即RAM存储器）。本文研究的模型包括用于降维的最大似然因子分析和用于执行聚类的高斯分布的混合物。第一项活动是制定一项计划，使处境不利和少数民族高中学生接触数据挖掘研究和实践，以鼓励他们学习计算机科学。第二项活动涉及加强休斯顿大学目前的数据挖掘研究和教学。智力MeritThis研究项目需要发现共同的算法原则，以执行增量矩阵计算的一个家庭的统计模型，了解如何总结大型数据集，保留其所需的多个模型的统计特性，并提出新的数据库技术，为这些模型量身定制，能够执行有效的矩阵操作二级存储。增量计算很难实现，因为线性高斯模型的方法需要对整个数据集进行迭代。 总结需要转换复杂的矩阵方程，考虑到高维，大数据集大小和数值稳定性，保持模型的准确性。开发结合主存储和辅助存储的矩阵优化与优化仅在主存储上工作的矩阵算法有很大不同，这项研究工作需要数学知识来推广、优化和转换线性高斯模型的计算。另一方面，它需要数据库系统的专业知识，如何组织和索引二级存储上的各种矩阵，以实现高效的阅读和写入。更广泛的影响该提案将对大型、复杂、高维科学数据集的分析产生广泛的影响，并增强具有增量模型计算能力的数据库系统。我们计划在科学数据集上应用和测试我们提出的算法和技术，包括地理、医学和生物数据集等。