EFFICIENT SPECTRAL APPROACHES FOR FINDING UNDERLYING STRUCTURES IN BIG DATA

 最近，我们开发了几种光谱方法，用于分析落入大数据（BD）类别的非常大的基因组学数据集或完整的数据库。第一种方法被设计为基于随机化来执行SVD或PCA，相对于在所有常见软件包中实现的标准Lanczos算法，该方法可以显着加速其特征向量和特征值的计算。更有效地计算PCA和SVD可以彻底改变基于PCA和SVD的无数生物医学应用，例如非常大的GWAS中的人口分层。这些算法比经典（确定性）方法产生更高的准确性，使处理的数据流太大，存储，并易于并行化，在多核微处理器中使用。我们的第二种新方法是无监督光谱学习方法。它提供了新的数学见解的惊人的概念简单的排名多个竞争的算法，而无需访问验证数据，并优化组合这种合奏的算法，以获得改进的预测在地面真理的情况下。构建一种工具，该工具为最终用户提供了一个选项，以最佳地排名或联合收割机算法分析基因组数据是一个实用和有效的解决方案，以消除最终用户或生物信息学家之间的困惑，他们面临着需要决定选择哪种工具进行他们的研究，因为大量的生物结果推断的不同的工具往往是不一致的。最佳执行算法或流水线的选择是必不可少的，因为它通常可以导致大规模并行测序实验的读出质量的实质性改善。此外，组合这些工具通常导致性能优于最佳性能算法的性能上级。我们的目标是建立一个团队，其重点是提供和传播光谱BD工具和方法的全面实施，这些工具和方法在生物医学科学，临床研究和医疗保健提供领域具有广泛的适用性。具体来说，我们将为基因组学和生物医学应用开发可扩展的PCA和SVD，进一步推进我们的光谱方法，用于对竞争管道的性能进行排名，并将它们联合收割机结合起来，以实现更好的预测，而无需访问验证数据。此外，我们将开发可扩展的降维技术，用于从生物医学应用中组织BD。

 DESCRIPTION (provided by applicant)l Recently we developed several spectral approaches for analyzing very large genomics datasets or complete databases that fall into the category of big data (BD). The first approach is designed to perform SVD or PCA based on randomization that can dramatically accelerate the computation of their eigenvectors and eigenvalues relative to the standard Lanczos algorithm implemented in all common software packages. Computing PCA and the SVD more efficiently could revolutionize the innumerable biomedical applications based on PCA and the SVD, e.g. population stratification in very large GWAS. These algorithms produce higher accuracy than classical (deterministic) methods, enable the processing of data streams that are too large to store, and parallelize easily to be used in multicore microprocessors. Our second novel approach is an unsupervised spectral learning method. It provides new mathematical insights of striking conceptual simplicity for ranking multiple competing algorithms without access to validation data and for optimally combining this ensemble of algorithms to obtain improved predictions in the absence of ground truth. Constructing a tool that provides end users an option to optimally rank or combine algorithms for analysis of genomics data is a practical and efficient solution to remove the confusion among end-users or bioinformaticians who are faced with the need to decide which tool to choose for their study, as a large number of biological results inferred by the different tools are often in disagreement. The choice of the best performing algorithm or pipeline is essential as it can often lead to substantial improvement in quality of the readout from massively parallel sequencing experiments. Moreover, combining these tools typically results in performance superior to the best performing algorithm. Our goal is to establish a team whose focus is to provide and disseminate full-blown implementations of spectral BD tools and methods that have broad applicability across the spectrum of biomedical sciences, clinical research, and healthcare delivery. Specifically we will develop scalable PCA and SVD for Genomics and biomedical applications, further advance our spectral method for ranking the performance of competing pipelines and combine them to achieve better predictions without access to validation data. Moreover, we will develop scalable dimensional reduction techniques for organizing BD from biomedical applications.