CAREER: Reliable and Efficient Data Encoding for Extreme-Scale Simulation and Analysis

科学和工程中的变革性研究以应对我们这个时代的挑战，如设计新的燃烧系统，依赖于在高性能计算系统上运行的日益复杂的计算模型和模拟。这些模拟和分析越来越受到它们必须使用、生成和分析的海量数据的限制。为了管理这些海量数据，该项目探索了创新的机制，通过减少数据移动和最大限度地利用计算能力，同时最大限度地减少错误和信息损失，来优化这些模拟的性能。这样的性能改进支持了NSF推进新兴的、数据密集型科学发现的使命，并为解决世界上最紧迫和最复杂的当代科学和工程问题做出了贡献。该项目实施了全面的外展和教育，以培训下一代专业工作者和研究人员最新的计算架构和编程方法，并为学生参与、研究和就业提供丰富的机会。它利用多个校园和国家资源，并实施经过验证的、基于研究的干预措施，以吸引、留住和教育计算机工程方面的女性和代表性较低的少数群体，这进一步推动了美国增加工程参与度的国家目标。本项目的研究目标是将视频数据的压缩技术和格式应用于研究新的数据编解码方案，以优化数据密集型模拟和分析中的数据移动和计算。创新的新机制有可能有效地减少生成和传输的数据量，同时还能够使用压缩数据快速执行各种分析内核，并允许在当前和未来的极端规模平台上无缝扩展其性能。研究目标是研究科学数据集的数据编码/解码和利用编码数据，在当前极端规模的科学工作流中无缝地使用和扩展编码数据集，并优化机器学习和数据挖掘算法，目标是在最大限度地利用计算能力的同时将误差降至最低。这些新机制被应用于评估框架，并在多种极端规模的数据驱动的科学应用中得到验证，包括气候、多物理和流体动力学。这种方法预计将改变数据表示和编码，同时对现有应用程序造成最小的干扰，以响应硬件体系结构和数据集特征的趋势。预计它将通过降低防御性I/O成本和生产性I/O成本来提高计算科学家工作负载的整体性能，分别在空间和时间上将数据减少到原来的1/100和1/200，潜在地将整体I/O成本提高到原来的1/50。该项目利用多种协作来建立系统共同设计的指导原则和可伸缩的系统软件层，以便在世界级计算基础设施内进行更好的数据编码。该项目加强了马萨诸塞大学洛厄尔分校的计算机工程课程，扩大了对计算机工程的参与，并创建了一个协作性、跨学科的研究计划，旨在利用不断发展的计算范例。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Transformative research in science and engineering to address challenges of our time, such as designing new combustion systems, depends on progressively sophisticated computational models and simulations that operate on high performance computing systems. These simulations and analyses are increasingly constrained by the massive volumes of data that they must use, generate, and analyze. To manage this enormous amount of data, this project explores innovative mechanisms to optimize the performance of these simulations by reducing data movement and maximizing the use of computing power, while minimizing errors and information loss. Such performance improvements support NSF's mission to advance emerging, data-intensive science discovery and contribute to solving the world's most pressing and complex contemporary science and engineering problems. This project implements comprehensive outreach and education to train the next-generation of professional workers and researchers in the latest computing architectures and programming methodologies, and provides rich opportunities for student engagement, research, and employment. It leverages multiple campus and national resources and implements proven, research-based interventions to attract, retain, and educate female and underrepresented minority populations in computer engineering, which furthers the US national goal of increased participation in engineering. The research goal of this project is to adapt techniques and formats for compressing video data to the investigation of novel data encoding and decoding schemes to optimize data movement and computation in data-intensive simulation and analyses. Innovative new mechanisms have the potential to efficiently reduce the volume of data generated and transferred while also enabling rapid execution of various analysis kernels using compressed data, and permitting seamless scaling of their performance on current and future extreme-scale platforms. The research objectives are to investigate data encoding/decoding of scientific datasets and harness encoded data, employ and scale encoded datasets seamlessly within current extreme-scale scientific workflows, and optimize machine learning and data mining algorithms with the goal of maximizing the use of computing power while minimizing errors. These new mechanisms are applied to an evaluation framework and validated on multiple extreme-scale data-driven scientific applications, including climate, multiphysics, and fluid dynamics. This approach is expected to transform data representation and encoding while incurring minimal disturbance to existing applications, responding to the trends in hardware architecture and dataset characteristics. It is anticipated to improve the overall performance of computational scientists' workloads by reducing defensive and productive I/O costs, respectively, up to 100x and 200x data reduction spatially and temporally, potentially resulting in up to an overall 50x I/O cost improvement. The project leverages multiple collaborations in order to establish the governing principles for system co-design and scalable system software layers for better data encoding within world-class computational infrastructures. This project strengthens the University of Massachusetts Lowell computer engineering curriculum, broadens participation in computer engineering, and creates a collaborative, interdisciplinary research program geared toward exploiting ever-evolving computing paradigms.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.