RIVA: Robust Integrity Verification Algorithm for High-Speed File Transfers
RIVA: Robust Integrity Verification Algorithm for High-Speed File Transfers
复制标题
DOI：
10.1109/tpds.2020.2966616
复制
发表时间：
2020-01
影响因子：
5.3
通讯作者：
Batyr Charyyev;Engin Arslan
中科院分区：
文献类型：
--
作者：
Batyr Charyyev;Engin Arslan
端到端完整性验证旨在通过使用MD5和SHA1等加密散列函数比较源和目标端点上的文件的校验和，来保护文件传输免受静默数据损坏。然而，用于文件传输的端到端完整性验证的现有实现不足以检测导致盘和高速缓冲存储器之间不一致的未检测到的盘错误。在本文中，我们提出了健壮的完整性验证算法(RIVA)，通过强制校验和计算任务直接从磁盘读取文件来加强文件传输的完整性。Riva通过在文件页面传输后使其内存映射无效来实现这一点，以便在再次读取文件以进行校验和计算时，将从磁盘中获取该文件，并捕获静默磁盘错误。我们设计并进行了大量的容错实验，以评估完整性验证算法对未检测到的磁盘写入错误的健壮性。结果表明，虽然最新的完整性验证算法无法检测到几乎所有文件大小的注入错误，但Riva借助缓存失效捕获了所有注入错误。我们进一步运行统计分析来评估丢失静默磁盘错误的概率，并发现与现有方法相比，Riva将可能性降低了10到15个数量级。最后，强制执行磁盘读入完整性验证会带来不可避免的开销，以换取针对静默磁盘错误增强的健壮性，但在大多数情况下，Riva通过对同一文件的不同部分同时运行传输、缓存失效和校验和计算过程，将其开销保持在15%以下。
End-to-end integrity verification is designed to protect file transfers against silent data corruption by comparing checksum of files at source and destination end points using cryptographic hash functions such as MD5 and SHA1. However, existing implementations of end-to-end integrity verification for file transfers fall short to detect undetected disk errors that causes inconsistency between disk and cache memory. In this article, we propose Robust Integrity Verification Algorithm (RIVA) to strengthen the integrity of file transfers by forcing checksum computation tasks to read files directly from disk. RIVA achieves this by invalidating memory mappings of file pages after their transfer such that when the file is read again for checksum calculation, it will be fetched from disk and silent disk errors will be captured. We design and conduct extensive fault resilience experiments to evaluate the robustness of integrity verification algorithms against undetected disk write errors. The results indicate that while the state-of-the-art integrity verification algorithms fail to detect the injected errors for almost all file sizes, RIVA captures all of them with the help of cache invalidation. We further run statistical analysis to assess the probability of missing silent disk errors and find that RIVA reduces the likelihood by 10 to 15 orders of magnitude compared to the existing approaches. Finally, enforcing disk read in integrity verification introduces an inevitable overhead in exchange of increased robustness against silent disk errors, but RIVA keeps its overhead below 15 percent in most cases by running transfer, cache invalidation, and checksum computation processes concurrently for different portions of the same file.