SHF: Medium: Collaborative Research: Scalable Algorithms for Spatio-temporal Data Analysis

超级计算机计算能力的加速以及大型仪器的开发和部署，如望远镜、对撞机、传感器和设备，提出了一个基本问题。“洞察和发现知识的时间能以同样的指数速度减少吗？”目前的答案显然是否定的，因为结合分析、挖掘和从海量数据集中发现知识的关键一步远远落后于软件、模拟和数据生成方面的进步。数据分析需要“数据驱动”的计算和分析。这需要可扩展的软件来进行数据缩减、近似、分析、统计和自下而上的发现。为了在科学发现方面取得重大飞跃，需要可扩展的并行分析软件来处理大量数据。该项目开发创新的、可扩展的、可持续的数据分析算法，以在高性能并行计算机上分析和挖掘海量数据，包括(1)适用于时空数据、海量图分析、社区计算的自下而上和无监督的数据聚类算法，以及时变图、不同类型数据和不同数据特征中的模式检测；(2)时空数据中的变化检测和异常检测；(3)在一定的时间和空间约束下跟踪移动数据和集群动态。这些并行算法利用天体物理、宇宙学模拟、气候模拟和社会网络分析等科学应用中产生的海量数据，在现代高性能并行计算机上进行结果验证和性能评估。该项目通过大数据并行分析软件直接满足大规模科学发现的时空数据分析、性能可扩展性和编程生产率的关键需求。这项工作将通过加速科学发现，影响巨大的社会效益和科学重要性的应用，如气候理解、环境可持续性、天体物理学、生物学和医学。此外，所开发的软件基础设施可用于商业应用，如商业、社会、安全、药物研发等。源代码向公众开放，供所有社区适应、构建、定制和贡献，从而使其价值和用途成倍增加。

Acceleration of computing power of supercomputers along with development and deployment of large instruments such as telescopes, colliders, sensors and devices raises one fundamental question. "Can the time to insight and knowledge discovery be reduced at the same exponential rate?" The answer currently is clearly "NO", because a critical step that combines analytics, mining and discovering knowledge from the massive datasets has lagged far behind advances in software, simulation and generation of data. Analysis of data requires "data-driven" computing and analytics. This entails scalable software for data reduction, approximations, analysis, statistics, and bottom-up discovery. Scalable and parallel analytics software for processing large amount of data is required in order to make a significant leap forward in scientific discoveries. This project develops innovative, scalable, and sustainable data analytics algorithms to enable analysis and mining of massive data on high-performance parallel computers, which include (1) bottom-up and unsupervised data clustering algorithms that are suitable for spatio-temporal data, massive graph analytics, community computations, and detection of patterns in time-varying graphs, different types of data, and different data characteristics; (2) change detection and anomaly detection in spatio-temporal data; and (3) tracking moving data and cluster dynamics within certain time and space constraints. These parallel algorithms use the massive amount of data generated from scientific applications, such as astrophysics, cosmology simulations, climate modeling, and social networking analysis, for result verification and performance evaluation on modern high-performance parallel computers.This project directly addresses the critical needs for spatio-temporal data analysis, performance scalability, and programming productivity of large-scale scientific discovery via parallel analytics software for big data. This work will impact applications of enormous societal benefits and scientific importance such as climate understanding, environmental sustainability, astrophysics, biology and medicine by accelerating scientific discoveries. Furthermore, the developed software infrastructure can be used and adopted in commercial applications, such as commerce, social, security, drug discovery, and so on. The source codes are open to the public for all community to adapt, build-upon, customize and contribute to, thereby multiplying its value and usage.