Enhancing DataRaceBench for Evaluating Data Race Detection Tools
Enhancing DataRaceBench for Evaluating Data Race Detection Tools
复制标题
增强 DataRaceBench 以评估数据竞争检测工具
DOI：
--
复制
发表时间：
2020
期刊：
影响因子：
--
通讯作者：
Yonghong Yan
中科院分区：
文献类型：
--
作者：
Gaurav Verma;Yaying Shi;C. Liao;B. Chapman;Yonghong Yan
DataRaceBench是一个专用的基准测试套件，用于评估旨在发现OpenMP程序中数据竞争错误的工具。自2017年首次发布以来，DataRaceBench已被工具开发人员广泛用于发现其工具的优势和局限性。结果还提供了一个苹果到苹果的最先进的数据竞争检测工具的比较。在本文中，我们将讨论我们为增强DataRaceBench所做的最新努力。特别是，我们增加了对Fortran语言和一些最新的OpenMP 5.0语言功能的支持。我们还添加了代表来自文献和其他基准测试的新模式的新内核（例如，NAS并行基准测试）。为了减少基准测试套件中重复的代码模式，我们设计了一个基于距离的代码相似性分析，结合静态和动态代码功能。最后，我们将工具对接并简化整个基准测试流程，以快速生成一个仪表板，显示最先进的OpenMP程序数据竞争检测。增强的DataRaceBench作为1.3.0版发布，新增了222个基准测试。其中56个是C语言，其余166个是Fortran语言，再现了C程序的本质。我们的实验表明，这个新版本可以发现当前数据竞争检测工具的更多局限性，大大减少了用户运行实验所需的工作量。
DataRaceBench is a dedicated benchmark suite to evaluate tools aimed to find data race bugs in OpenMP programs. Since its initial release in 2017, DataRaceBench has been widely used by tool developers to find the strengths and limitations of their tools. The results also provide an apple-to-apple comparison of the state-of-the-art of data race detection tools. In this paper, we discuss our latest efforts to enhance DataRaceBench. In particular, we have added support for Fortran language and some of the newest OpenMP 5.0 language features. We have also added new kernels representing new patterns from literature and other benchmarks (e.g., NAS Parallel Benchmark). To reduce duplicated code patterns in the benchmark suite, we have designed a distance-based code similarity analysis, combining both static and dynamic code features. Finally, we dockerize tools and streamline the entire benchmarking process to quickly generate a dashboard showing the state-of-the-art of data race detection of OpenMP programs. The enhanced DataRaceBench is released as v 1.3.0, with 222 newly added benchmarks. 56 of them are in C, and the remaining 166 are in Fortran, reproducing the C programs’ nature. Our experiments show that this new version can spot more limitations of the current data race detection tools, with significantly reduced user efforts needed to run experiments.