CSR: Small: Collaborative Research:System Research on Persistent High-Dimensional Data Access and Its Application to Semiclassical Molecular Dynamics Simulation

CSR：小：协作研究：持久高维数据访问系统研究及其在半经典分子动力学模拟中的应用

• 批准号: 1526190
• 负责人: Weijun Xiao
• 金额: $ 23.15万
• 依托单位: Virginia Commonwealth University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2015
• 资助国家: 美国
• 起止时间: 2015-09-01 至 2021-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1526190&HistoricalAwards=false
• 关键词: CSR; Small; Collaborative; Research; System

High dimensional data arise in many scientific studies. High-frequency accesses of large disk-resident high dimensional datasets pose both I/O and algorithmic challenges for high access efficiency. Investigators of this proposal are faced with high dimensional data from semiclassical molecular dynamics studies that need to be accessed repeatedly at a very high frequency during the simulation, and in some simulation studies the datasets can be far larger than the memory capacity. The project addresses the challenge of large high-dimensional datasets by proposing a middleware system for fast access of storage-resident high dimensional data, with easy-to-use APIs for data organizing, indexing, search, and I/O latency hiding. In order to enable fast searches with low memory footprint, various indexing structures are explored and a new locality preserving tree (LPT) is proposed. Furthermore, the project exploits specific data access patterns in dynamics studies for I/O latency hiding through overlapping of computations with I/O?s, enabled by a small set of easy-to-use programming constructs for non-blocking read/write operations that are under development for further reduction of data access time. The developed non-blocking APIs are compatible with hard disk drives (HDDs) and hybrid solid state drives (SSDs) and HDDs. In addition, an SSD-based I/O accelerator is developed and integrated with non-blocking APIs.This project tackles important and challenging problems for accessing large high-dimensional data sets, which has applications beyond semiclassical molecular dynamics simulations. The investigators will leverage the research efforts to motivate and educate students. To broaden the impact beyond project participants, courses will be enhanced with knowledge, insights, and materials obtained from the research, with careful adaptation as course materials and course projects.

高维数据出现在许多科学研究中。大型磁盘驻留的高维数据集的高频率访问对高访问效率提出了I/O和算法挑战。 研究人员面临的是来自半经典分子动力学研究的高维数据，这些数据需要在模拟过程中以非常高的频率重复访问，并且在某些模拟研究中，数据集可能远远大于内存容量。该项目通过提出一个用于快速访问存储驻留的高维数据的中间件系统，以及用于数据组织、索引、搜索和I/O延迟隐藏的易于使用的API，来解决大型高维数据集的挑战。为了实现低内存占用的快速搜索，探索了各种索引结构，并提出了一种新的局部保持树（LPT）。此外，该项目利用特定的数据访问模式的动态研究I/O延迟隐藏通过重叠的计算与I/O？s，由一小部分易于使用的编程结构实现，用于正在开发的非阻塞读/写操作，以进一步减少数据访问时间。开发的非阻塞API与硬盘驱动器（HDD）和混合固态驱动器（SSD）和HDD兼容。此外，我们还开发了一个基于SSD的I/O加速器，并将其与非阻塞API集成。该项目解决了访问大型高维数据集的重要和具有挑战性的问题，其应用范围超出了半经典分子动力学模拟。研究人员将利用研究工作来激励和教育学生。 为了扩大项目参与者之外的影响，课程将通过从研究中获得的知识，见解和材料来增强，并仔细调整为课程材料和课程项目。