CAREER: Enabling Distributed and In-Situ Analysis for Multidimensional Structured Data

现代科学的进步导致了所有科学、技术、工程和数学(STEM)学科的数据爆炸式增长。从这个庞大的信息库中提取有意义的知识变得既复杂又昂贵。在基因组学和天文学等每天产生大量数据的领域，有必要将存储库存储在多个地理上不同的位置。这种类型的数据分配会导致昂贵的计算和不完整的分析。对于健康信息学和财务，由于隐私、安全或成本问题，数据通常在研究中心之间隔离。同样，无法获得数据的全局视图会在计算时产生不准确的结果。传统的集中式数据分析方法不再产生最佳结果；它已成为一个主要瓶颈，阻碍了大数据所能提供的优势。当前的分布式分析解决方案仍然缺乏通用性、可伸缩性或准确性。该项目旨在改善分布式数据管理中的问题，同时实现可伸缩和准确的分析。该项目提供了一种全面的方法来处理数据到知识的提取、表示和大规模学习。这项研究的产品包括：(1)一套语义投影和可伸缩学习方法的算法套件，用于有效的数据降维、模式识别、异常检测和聚类；(2)用于将分布式数据获取过程与现场分析和众包相结合的开源中间件。这些产品将通过https://github.com/distributedreasoningatunm.上的GitHub存储库提供此外，众包扩展还兼具教育平台的作用，旨在吸引人们对STEM领域的兴趣。

Advances in modern science have led to explosions of data across all science, technology, engineering, and mathematics (STEM) disciplines. Extracting meaningful knowledge from this large pool of information has become both complicated and costly. In fields like genomics and astronomy, where very large volumes of data are produced daily, it is necessary to store repositories throughout multiple, geographically distinct locations. This type of data allocation results in expensive computations and incomplete analyses. For health informatics and finances, data is typically isolated between research centers due to privacy, security, or cost issues. Again, the inability to have a global view of the data yields inaccurate outcomes at computation time. The classic centralized approach to analyzing data no longer produces optimal results; it has become a major bottleneck, hindering the advantages Big Data has to offer. Current solutions for distributed analysis still lack generality, scalability, or accuracy. This project aims to ameliorate problems in the management of distributed data while enabling scalable and accurate analyses. The project provides a comprehensive approach to handle data-to-knowledge extraction, representation, and learning at scale. Products of this research include: (1) an algorithmic suite of semantic projections and scalable learning methods for efficient data dimensionality reduction, pattern recognition, anomaly detection, and clustering, and (2) an open source middleware for coupling distributed data acquisition processes with in-situ analytics and crowd sourcing. These products will be made available through a GitHub repository at https://github.com/distributedreasoningatunm. Moreover, the crowd sourcing extension doubles as an educational platform, which aims to attract interest in the STEM fields.