XPS: DSD: Adaptive Stream-Processing Compilers for a Messy World

流处理是一个越来越重要的应用领域;数据泛滥的社会中有很大一部分是实时数据，从科学数据到推文。 幸运的是，在过去的十年中，各种流处理语言如雨后春笋般涌现，包括WaveScope、StreamIt、Feldspar等。 虽然这些语言使流处理程序（其采取流操作符的流图的形式）在多核和小型机器集群上自动且高效地执行，它们在假设流工作负载不变的情况下进行优化，并且不能处理在许多现实流情况中发现的动态条件，诸如在现代网络内部。快速、增量地编译和重新编译流处理流程图的子图，以支持动态放置和优化策略，同时保持高性能。 其目标是允许流媒体应用程序立即并行启动，或者在程序更改时重新启动，同时随着时间的推移适应环境的可预测特征，包括恒定速率的流。 用于诸如JavaScript的语言的现有即时（JIT）编译器是成熟的技术，但是需要新的技术主体来在动态上下文中应用流编译器的更激进的优化。 该项目旨在开发这些技术，并在一个特定的应用领域进行评估：高速网络内处理。

Stream processing is an increasingly important application domain; a significant portion of the data-deluge beleaguering society takes the form of real-time data, ranging from scientific data to tweets. Fortunately, in the last decade, a variety of stream-processing languages have sprung up, including WaveScope, StreamIt, Feldspar, and others. While these languages enable stream-processing programs (which take the form of flow-graphs of stream operators) to execute automatically and efficiently on multicores and small clusters of machines, they optimize assuming an unchanging streaming workload and cannot handle dynamic conditions found in many realistic streaming situations, such as inside modern networks.This proposal pursues a more adaptive approach: fast, incremental compilation and recompilation of subgraphs of a stream-processing flow-graph to support dynamic placement and optimization policies while retaining high performance. The goal is to allow streaming applications to start instantly and in parallel, or restart if the program changes, while adapting to predictable features of the environment over time including streams of constant rate. Existing just-in-time (JIT) compilers for languages such as JavaScript are a mature technology, but a new body of techniques are needed to apply the more radical optimizations of stream compilers in a dynamic context. This project aims to develop these techniques and evaluate them in a specific application domain: high-speed in-network processing.