New ideas track: testing mapreduce-style programs
New ideas track: testing mapreduce-style programs
复制标题
DOI：
10.1145/2025113.2025204
复制
发表时间：
2011-09
期刊：
影响因子：
--
通讯作者：
Christoph Csallner;L. Fegaras;Chengkai Li
中科院分区：
文献类型：
--
作者：
Christoph Csallner;L. Fegaras;Chengkai Li
MapReduce已成为处理大量数据的常见编程模型，这是在现代计算应用程序中所需的。如今，已经存在一些MAPREDUCE实施和执行系统，并且正在开发和部署许多MAPREDUCE程序。但是，开发MapReduce程序并不总是一件容易的事。编程模型使程序容易发生几个特定于MapReduce的错误。也就是说，要产生确定的结果，MapReduce计划需要满足某些高级正确性条件。违反程序可能会基于低级基础架构事件（例如网络延迟，调度决策等）在相同的输入数据上产生不同的输出值。当前的MapReduce系统和工具缺乏支持检查这些条件和报告违规行为的支持。本文提出了一种新型技术，该技术系统地在MapReduce应用程序中搜索此类错误并生成相应的测试用例。该技术通过编码高级MAPREDUCUS正确性条件作为符号程序的约束，并检查它们是否正在测试的程序中进行检查。据我们所知，这是解决MapReduce式编程问题的第一种方法。
MapReduce has become a common programming model for processing very large amounts of data, which is needed in a spectrum of modern computing applications. Today several MapReduce implementations and execution systems exist and many MapReduce programs are being developed and deployed in practice. However, developing MapReduce programs is not always an easy task. The programming model makes programs prone to several MapReduce-specific bugs. That is, to produce deterministic results, a MapReduce program needs to satisfy certain high-level correctness conditions. A violating program may yield different output values on the same input data, based on low-level infrastructure events such as network latency, scheduling decisions, etc. Current MapReduce systems and tools are lacking in support for checking these conditions and reporting violations. This paper presents a novel technique that systematically searches for such bugs in MapReduce applications and generates corresponding test cases. The technique works by encoding the high-level MapReduce correctness conditions as symbolic program constraints and checking them for the program under test. To the best of our knowledge, this is the first approach to addressing this problem of MapReduce-style programming.