Stratosphere Data and Processing Model, its Optimization and Parallelization

点击翻译按钮获取中文摘要

The goal of Project A within the Stratosphere research unit is to optimize and parallelize the declarative data flow programs on a massively parallel, fault-tolerant, adaptive computing architecture. As foundation for the overall research unit, the project will coordinate the design of a programming model, where text data and uncertainty are first class citizens, and data-intensive operations, e.g., information extraction, and data cleansing operators, can be specified and optimally executed. The project will research how to effectively and efficiently optimize the robustness and parallelize the execution of these data flow programs by providing performance metrics, optimization algorithms, and an abstraction for parallel data flow programming. The project will also demonstrate the overall effectiveness of the Stratosphere approach by analyzing and implementing a real-world use-case from climate research and by evaluating and benchmarking the system performance.