REPETITA: Repeatable Experiments for Performance Evaluation of Traffic-Engineering Algorithms
REPETITA: Repeatable Experiments for Performance Evaluation of Traffic-Engineering Algorithms
复制标题
DOI：
--
复制
发表时间：
2017-10
期刊：
影响因子：
--
通讯作者：
S. Gay;P. Schaus;Stefano Vissicchio
中科院分区：
文献类型：
--
作者：
S. Gay;P. Schaus;Stefano Vissicchio
在本文中，我们提出了一种实用的方法来提高交通工程（TE）算法的实验分析的可重复性，其实现，评估和比较目前难以复制。我们设想的目标是实现现有和未来TE算法的普遍可检查实验。我们描述了repeta的设计和实现，这是一个软件框架，实现了常见的TE功能，自动化实验设置，并简化了TE算法的比较（在解决方案质量，执行时间等方面）。在其当前版本中，REPETITA包括(i)可重复实验的数据集，由250多个真实网络拓扑组成，具有完整的带宽和延迟信息以及相关的流量矩阵；（ii）实现具有IGP权重调整和分段路由优化的域内TE的最先进算法。我们展示了我们的框架如何成功地重现文献中描述的结果，并简化了对定性多样化TE算法的新分析。我们公开发布了我们的repeta实现，希望社区将其视为可行性的演示、激励和提高实验再现性的初始代码基础：其面向插件的架构确实使repeta易于扩展新的数据集、算法、TE原语和分析。因此，我们邀请研究社区使用并贡献我们发布的代码和数据集。
In this paper, we propose a pragmatic approach to improve reproducibility of experimental analyses of traffic engineering (TE) algorithms, whose implementation, evaluation and comparison are currently hard to replicate. Our envisioned goal is to enable universally-checkable experiments of existing and future TE algorithms. We describe the design and implementation of REPETITA, a software framework that implements common TE functions, automates experimental setup, and eases comparisons (in terms of solution quality, execution time, etc.) of TE algorithms. In its current version, REPETITA includes (i) a dataset for repeatable experiments, consisting of more than 250 real network topologies with complete bandwidth and delay information as well as associated traffic matrices; and (ii) the implementation of state-of-the-art algorithms for intra-domain TE with IGP weight tweaking and Segment Routing optimization. We showcase how our framework can successfully reproduce results described in the literature, and ease new analyses of qualitatively-diverse TE algorithms. We publicly release our REPETITA implementation, hoping that the community will consider it as a demonstration of feasibility, an incentive and an initial code basis for improving experiment reproducibility: Its plugin-oriented architecture indeed makes REPETITA easy to extend with new datasets, algorithms, TE primitives and analyses. We therefore invite the research community to use and contribute to our released code and dataset.