CONCEPTUAL: a network correctness and performance testing language
CONCEPTUAL: a network correctness and performance testing language
复制标题
概念：网络正确性和性能测试语言
DOI：
10.1109/ipdps.2004.1303014
复制
发表时间：
2004
期刊：
影响因子：
--
通讯作者：
S. Pakin
中科院分区：
文献类型：
--
作者：
S. Pakin
仅提供摘要形式。我们引入了一种新的，特定于领域的规范语言，称为概念。CONCEPTUAL允许在相对较少的代码行中表达复杂的通信基准和网络验证测试。除了帮助程序员节省编写和调试代码的时间外，CONCEPTUAL还解决了基准不透明的重要但很大程度上未被认识到的问题。基准不透明是指目前以促进结果的可重复性和独立评估的方式呈现性能测量是不切实际的。例如，声明性能图是由“带宽”测试产生的，并没有说明该测试是测量往返传输期间的数据速率还是测量多个背靠背单向消息上的平均数据速率;基准测试是否预注册缓冲区，发送预热消息，和/或在启动时钟之前预发布异步接收;执行了多少次运行，以及这些运行是通过取平均值、中值还是最大值来聚合的;或者，甚至诸如“MB/s”之类的数据单位是否指示每秒10/sup 6/或2/sup 20/字节。由于CONCEPTUAL程序简洁，基准测试的完整源代码可以与性能结果一起列出，从而明确了基准测试程序中的所有设计决策。因为CONCEPTUAL的语法类似于英语，所以CONCEPTUAL程序很容易被非专家理解。由于CONCEPTUAL是一种高级语言，它可以针对各种消息传递层和网络，从而实现公平准确的性能比较。
Summary form only given. We introduce a new, domain-specific specification language called CONCEPTUAL. CONCEPTUAL enables the expression of sophisticated communication benchmarks and network validation tests in comparatively few lines of code. Besides helping programmers save time writing and debugging code, CONCEPTUAL addresses the important-but largely unrecognized-problem of benchmark opacity. Benchmark opacity refers to the current impracticality of presenting performance measurements in a manner that promotes reproducibility and independent evaluation of the results. For example, stating that a performance graph was produced by a "bandwidth" test says nothing about whether that test measures the data rate during a round-trip transmission or the average data rate over a number of back-to-back unidirectional messages; whether the benchmark preregisters buffers, sends warm-up messages, and/or preposts asynchronous receives before starting the clock; how many runs were performed and whether these were aggregated by taking the mean, median, or maximum; or, even whether a data unit such as "MB/s" indicates 10/sup 6/ or 2/sup 20/ bytes per second. Because CONCEPTUAL programs are terse, a benchmark's complete source code can be listed alongside performance results, making explicit all of the design decisions that went into the benchmark program. Because CONCEPTUAL's grammar is English-like, CONCEPTUAL programs can easily be understood by nonexperts. And because CONCEPTUAL is a high-level language, it can target a variety of messaging layers and networks, enabling fair and accurate performance comparisons.