Where the Ocean Meets the Cloud: Ad Hoc Longitudinal Analysis of Massive Mesh Data

科学正在成为一个数据管理问题。传感和计算建模的进步极大地提高了数据采集率，建立了查询——“在铁”实验——作为科学发现的基本方法，与原位、体外和硅实验一起。不幸的是，在大量数据集上设计和进行铁实验的基础设施跟不上我们创建这些数据集的集体能力。长期以来，计算建模者一直享受着创建更大更快的CPU农场的研究带来的好处，现在他们面临着tb级的模拟结果，其中对地球健康状况的深刻见解仍然是锁定的。关键是无缝性：交互式地定量和定性地分析这些数据，而不考虑时间和空间领域、物理位置、硬件架构、存储介质或文件组织的边界。该项目正在构建一个新的基础设施，该基础设施使用CluE平台，允许以交互速度对大量海洋模拟结果进行临时、纵向查询和可视化。该基础设施利用并扩展了两个现有系统：GridFields，一个用于通用和有效操作仿真结果的库；以及VisTrails，一个用于科学工作流、协作、可视化和溯源的综合平台。通过云计算这些系统，拟议的基础设施提供：(1)以交互速度无缝访问10年的模拟结果历史；(2)利用远程云和本地桌面资源的架构和执行策略；(3)支持可重复性、代码重用、取证和协作的来源捕获和操作平台。

Science is becoming a data management problem. Advancements in sensing and computational modeling have dramatically increased data acquisition rates, establishing queries -- "in ferro" experiments -- as an essential method of scientific discovery alongside in situ, in vitro, and in silico experiments. Unfortunately, the infrastructure to design and conduct in ferro experiments over massive datasets has not kept pace with our collective ability to create these datasets. Computational modelers, who have long enjoyed the benefits of a research focus on creating larger and faster CPU farms, now face terabytes of simulation results in which deep insights into the health of the planet remain locked. The key is seamlessness: to interactively analyze these data, quantitatively and qualitatively, without regard to boundaries manifesting from time and space domains, physical location, hardware architecture, storage medium, or file organization.This project is building a new infrastructure that uses the CluE platform to allow ad hoc, longitudinal query and visualization of massive ocean simulation results at interactive speeds. This infrastructure leverages and extends two existing systems: GridFields, a library for general and efficient manipulation of simulation results; and VisTrails, a comprehensive platform for scientific workflow, collaboration, visualization, and provenance. By cloud-enabling these systems, the proposed infrastructure provides: (1) seamless access to the 10 year history of simulation results at interactive speeds; (2) an architecture and execution strategy that exploit both remote cloud and local desktop resources; and (3) a provenance capture and manipulation platform that enables repeatability, code reuse, forensics, and collaboration.