Elements: Streaming Molecular Dynamics Simulation Trajectories for Direct Analysis: Applications to Sub-Picosecond Dynamics in Microsecond Simulations

计算能力的大幅提高增加了数值模拟中生成的数据量，而数据存储和传输能力却没有以同样的速度增长。这对于全原子分子动力学（MD）模拟来说尤其如此，这些模拟用于研究生物分子和新材料在微秒级时间尺度上在包含数十万到数百万原子的系统中的功能。这样的模拟评估了109次不同的原子位置和速度，这相当于2 pb的数据，超过了合理的数据存储和传输能力。结果，模拟数据仅以粗糙的时间间隔存储，尽管这种方法失去了对计算许多实验观测值至关重要的快速分子过程的信息。本项目创建的软件基础设施通过数据生成和分析的融合，避免了分子动力学模拟中的此类信息丢失。不是存储数据进行后处理（现状），而是通过流协议直接传递到并行软件平台。将流协议实现到具有大量用户基础的现有软件包中，可以直接集成到已建立的模拟协议中。因此，分子动力学模拟产生更多可用的信息，并更有效地利用相关的计算资源。MD模拟的流协议使用TCP/IP套接字应用程序编程接口（API）将数据直接从运行的模拟传输到分析软件。这种方法可以同时分析分子模拟中的快速（亚皮秒）和慢速（微秒）过程，而不会产生瓶颈或需要大量的轨迹输出文件。分析是在开源的MDAnalysis平台上进行的，该平台的工具是作为MDAKit插件实现的，这些插件明显受益于流接口，例如3D-2PT（空间分辨两相热力学）和用于分析空间分辨蛋白质溶剂化图的相关工具。该奖项由美国国家科学基金会高级网络基础设施办公室颁发，并得到化学部门的联合支持。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Massive increases in computing capabilities are increasing the data volume generated in numerical simulations, while data storage and transfer capabilities are not growing at the same pace. This is especially true for all-atom molecular dynamics (MD) simulations that are used to study the function of biomolecules and novel materials on timescales of microseconds in systems containing hundreds of thousands to millions of atoms. Such simulations evaluate atom positions and velocities for 109 distinct times, which corresponds to 2 petabytes of data and exceeds reasonable data storage and transfer capacities. As a result, simulation data is stored only at coarse time intervals even though this approach loses information on fast molecular processes that is essential to computing many experimental observables. The software infrastructure created in this project avoids such information loss in molecular dynamics simulations via convergence of data generation and analysis. Instead of storing data for post-processing (status quo), it is passed directly to a parallel software platform via a streaming protocol. The implementation of the streaming protocol into existing software packages with a large user base allows for a direct integration into established simulation protocols. As a result, molecular dynamics simulations produce more usable information, and associated computational resources are used more efficiently.The streaming protocol for MD simulations uses a TCP/IP socket application programming interface (API) to transfer data directly from a running simulation to the analysis software. This approach enables the simultaneous analysis of fast (sub-picosecond) and slow (microsecond) processes in molecular simulations without creating bottlenecks or requiring massive trajectory output files. The analysis is performed with the open-source MDAnalysis platform for which tools are implemented as MDAKit plugins that significantly benefit from the streaming interface, such as 3D-2PT (spatially resolved two-phase thermodynamics) and related tools for the analysis of spatially resolved protein solvation maps.This award by the NSF Office of Advanced Cyberinfrastructure is jointly supported by the Division of Chemistry.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.