A Real-Time Machine Learning and Visualization Framework for Scientific Workflows
A Real-Time Machine Learning and Visualization Framework for Scientific Workflows
复制标题
用于科学工作流程的实时机器学习和可视化框架
DOI：
10.1145/3093338.3093380
复制
发表时间：
2017
期刊：
影响因子：
--
通讯作者：
Fengguang Song
中科院分区：
文献类型：
--
作者：
Feng Li;Fengguang Song
高性能计算资源目前在科学和工程领域得到了广泛的应用。典型的事后处理方法使用持久存储来保存模拟生成的数据，因此数据分析任务需要从存储读取到内存。对于大规模的科学模拟，这样的I/O操作将产生巨大的开销。原位/在途方法通过直接访问和处理内存模拟结果来绕过I/O，这表明模拟和分析应用应该更紧密地耦合。本文构建了一个灵活且可扩展的框架，将科学模拟与多步骤机器学习过程和现场可视化工具连接起来，从而在复杂的工作流程中实时提供插入式分析和可视化功能。提出了一种分布式模拟时间聚类方法来检测真实湍流的异常。
High-performance computing resources are currently widely used in science and engineering areas. Typical post-hoc approaches use persistent storage to save produced data from simulation, thus reading from storage to memory is required for data analysis tasks. For large-scale scientific simulations, such I/O operation will produce significant overhead. In-situ/in-transit approaches bypass I/O by accessing and processing in-memory simulation results directly, which suggests simulations and analysis applications should be more closely coupled. This paper constructs a flexible and extensible framework to connect scientific simulations with multi-steps machine learning processes and in-situ visualization tools, thus providing plugged-in analysis and visualization functionality over complex workflows at real time. A distributed simulation-time clustering method is proposed to detect anomalies from real turbulence flows.