Effectiveness and predictability of in-network storage cache for Scientific Workflows
Effectiveness and predictability of in-network storage cache for Scientific Workflows
复制标题
科学工作流程网络内存储缓存的有效性和可预测性
DOI：
10.1109/icnc57223.2023.10074058
复制
发表时间：
2023
期刊：
影响因子：
--
通讯作者：
J. Balcas
中科院分区：
文献类型：
--
作者：
Caitlin Sim;K. Wu;A. Sim;I. Monga;C. Guok;F. Würthwein;Diego Davila;Harvey B. Newman;J. Balcas
大型科学合作通常会有多名科学家在进行不同分析时访问同一组文件，这会重复访问位于远处的大量共享数据。这些数据访问由于距离而具有长延迟，并且占用广域网上可用的有限带宽。为了减少广域网流量和数据访问延迟，已安装区域数据存储缓存作为新的网络服务。为了研究这种高速缓存系统在科学应用中的有效性，我们研究了南加州PB级高速缓存的高能物理实验。通过检查约3TB的操作日志，我们表明，该缓存删除了67.6%的文件请求从广域网和减少了12广域网上的流量。3TB（或35.4%）。通信量的减少（35.4%）小于文件数量的减少（67.6%），因为较大的文件不太可能被重用。由于数据访问模式的这种差异，该高速缓存系统已经实现了一种策略，以避免在处理较大文件时驱逐较小的文件。我们还建立了一个机器学习模型来研究该高速缓存行为的可预测性。测试表明，该模型能够准确预测该高速缓存访问、缓存未命中和网络吞吐量，为今后的资源配置和规划研究提供了参考。
Large scientific collaborations often have multiple scientists accessing the same set of files while doing different analyses, which create repeated accesses to the large amounts of shared data located far away. These data accesses have long latency due to distance and occupy the limited bandwidth available over the wide-area network. To reduce the wide-area network traffic and the data access latency, regional data storage caches have been installed as a new networking service. To study the effectiveness of such a cache system in scientific applications, we examine the Southern California Petabyte Scale Cache for a high-energy physics experiment. By examining about 3TB of operational logs, we show that this cache removed 67.6% of file requests from the wide-area network and reduced the traffic volume on wide-area network by 12. 3TB (or 35.4%) an average day. The reduction in the traffic volume (35.4%) is less than the reduction in file counts (67.6%) because the larger files are less likely to be reused. Due to this difference in data access patterns, the cache system has implemented a policy to avoid evicting smaller files when processing larger files. We also build a machine learning model to study the predictability of the cache behavior. Tests show that this model is able to accurately predict the cache accesses, cache misses, and network throughput, making the model useful for future studies on resource provisioning and planning.
登录
查看更多内容
StashCache：开放科学网格的分布式缓存联盟
DOI：
10.1145/3332186.3332212
发表时间：
2019
期刊：
7 pages.
影响因子：
--
作者：
Weitzel, Derek;Zvada, Marian;Vukotic, Ilija;Gardner, Rob;Bockelman, Brian;Rynge, Mats;Hernandez, Edgar Fajardo;Lin, Brian;Selmeci, Mátyás
通讯作者：
Selmeci, Mátyás
分析网络内数据缓存的科学数据共享模式
DOI：
10.1145/3452411.3464441
发表时间：
2021
期刊：
SNTA '21: Proceedings of the 2021 on Systems and Network Telemetry and Analytics
影响因子：
--
作者：
Copps, Elizabeth;Zhang, Huiyi;Sim, Alex;Wu, Kesheng;Monga, Inder;Guok, Chin;Würthwein, Frank;Davila, Diego;Fajardo, Edgar
通讯作者：
Fajardo, Edgar
DOI：
10.1145/3526064.3534110
发表时间：
2022
期刊：
SNTA '22: Fifth International Workshop on Systems and Network Telemetry and Analytics
影响因子：
--
作者：
Han, Ruize;Sim, Alex;Wu, Kesheng;Monga, Inder;Guok, Chin;Würthwein, Frank;Davila, Diego;Balcas, Justas;Newman, Harvey
通讯作者：
Newman, Harvey
使用 XCache 在互联网骨干网上创建普通科学内容交付网络
影响因子：
--
作者：
Fajardo, Edgar;Weitzel, Derek;Rynge, Mats;Zvada, Marian;Hicks, John;Selmeci, Mat;Lin, Brian;Paschos, Pascal;Bockelman, Brian;Hanushevsky, Andrew
通讯作者：
Hanushevsky, Andrew