XPS: DSD: Adaptive Stream-Processing Compilers for a Messy World

流处理是一个日益重要的应用领域；在困扰社会的数据洪流中，有很大一部分是实时数据的形式，从科学数据到推文。幸运的是，在过去的十年中，各种流处理语言如雨后春笋般涌现，包括WaveScope、StreamIt、Feldspar等。虽然这些语言使得流处理程序(以流运算符的流图的形式)能够在多核和小型机器集群上自动高效地执行，但它们在假设流工作负载不变的情况下进行了优化，并且不能处理许多现实流情况下的动态条件，例如在现代网络中。该方案追求一种更自适应的方法：快速、增量地编译和重新编译流处理流图的子图，以支持动态布局和优化策略，同时保持高性能。其目标是允许流应用程序立即并行启动，或者在程序发生变化时重新启动，同时随着时间的推移适应环境的可预测功能，包括恒定速率的流。现有的针对语言的即时(JIT)编译器是一项成熟的技术，但需要一套新的技术来在动态上下文中应用更激进的流编译器优化。该项目旨在开发这些技术，并在一个特定的应用领域对其进行评估：高速网络内处理。

Stream processing is an increasingly important application domain; a significant portion of the data-deluge beleaguering society takes the form of real-time data, ranging from scientific data to tweets. Fortunately, in the last decade, a variety of stream-processing languages have sprung up, including WaveScope, StreamIt, Feldspar, and others. While these languages enable stream-processing programs (which take the form of flow-graphs of stream operators) to execute automatically and efficiently on multicores and small clusters of machines, they optimize assuming an unchanging streaming workload and cannot handle dynamic conditions found in many realistic streaming situations, such as inside modern networks.This proposal pursues a more adaptive approach: fast, incremental compilation and recompilation of subgraphs of a stream-processing flow-graph to support dynamic placement and optimization policies while retaining high performance. The goal is to allow streaming applications to start instantly and in parallel, or restart if the program changes, while adapting to predictable features of the environment over time including streams of constant rate. Existing just-in-time (JIT) compilers for languages such as JavaScript are a mature technology, but a new body of techniques are needed to apply the more radical optimizations of stream compilers in a dynamic context. This project aims to develop these techniques and evaluate them in a specific application domain: high-speed in-network processing.