ITR: A Language, Compilers and Tools for the Streaming Application Domain

以数据流为中心的应用程序正变得越来越重要和广泛。在分布式领域中，存在传感器网络、位置感知设备和智能设备，它们需要通过流进行分散处理和健壮的通信。在嵌入式领域，有手持计算机和移动电话，它们在严格的资源约束下处理数据流。还有一些高性能应用程序，如软件路由器、手机基站和雷达图像处理，这些应用程序必须将流抽象有效地映射到最新的超级计算机上。总之，我们相信流将成为下一代软件的基石。StreamIt项目的目标是为流领域建立一个高级编程范例。在上面列出的所有应用程序中，对程序员都有很高的要求。StreamIt提供了一种高级流语言，其中域的固有复杂性由流感知编译器处理。一个程序被写成一组自治的过滤器，每个过滤器都有自己的内存，并通过数据通道与相邻的过滤器通信。这暴露了并行性和通信模式，为编译器提供了更强大的功能。我们相信这将在三个不同的社区中显著提高程序员的生产力、程序的健壮性和可移植性。第一个目标是异构分布式计算，其中不可靠的设备在ad-hoc网络中与分布在多个节点上的应用程序进行通信。为了解决跨多个节点划分程序的挑战，StreamIt为程序员提供了“单个机器抽象”。过滤器按照算法的粒度指定，编译器根据运行时资源约束将它们映射到节点。StreamIt还将提供对容错的高级支持。如果不可靠的节点发生故障，轻量级运行时系统可以利用静态编译器分析从备份节点恢复关键数据，自动重新定位计算，并继续处理。StreamIt的第二个重点领域是针对嵌入式和DSP社区的特定领域优化。为了消除昂贵的手工汇编代码，StreamIt将包括特定于领域的代码生成策略，如缓存约束调度、流感知存储管理和流程序的新调用约定。为了降低高级算法设计所需的专业知识，StreamIt还将包括新的算法转换，如线性节点检测和自动转换到频域。StreamIt的第三个推动力与高性能计算机架构有关。StreamIt旨在为新兴的通信暴露机器提供一种通用机器语言（CML）。就像C是冯-诺伊曼架构的CML一样，StreamIt将隐藏不同机器之间的差异，同时捕获对获得性能很重要的属性。这将为面向通信的机器的可移植编程模型提供基础。StreamIt的最后一个重点是与前面所有主题交织在一起的：对流应用程序领域的全面描述和理解。该项目将包括研究和实现广泛的最先进的流算法，从而揭示它们的共同属性，并为流语言和编译的未来研究提供框架。为了展示这个项目的可行性，我们创建了一个StreamIt语言的原型版本，一个用StreamIt编写的大量流媒体程序集合，一个将StreamIt映射到单处理器的编译器，以及一个将StreamIt映射到原始通信暴露架构的优化编译器。

Applications centered around streams of data are becoming increasingly important and widespread. In the distributed realm, there are sensor networks, location-aware devices, and intelligent appliances, which require decentralized processing and robust communication via streams. In the embedded domain, there are handheld computers and cell phones which process data streams under tight resource con-straints. There are also high-performance applications such as software routers, cell phone base stations, and radar image processing, in which the stream abstraction must be mapped efficiently to the latest supercomputers. In all, we believe that streams will be the cornerstone of next-generation software.The goal of the StreamIt project is to establish a high-level programming paradigm for the streaming domain. In all of the applications listed above, there are intense demands on the programmer. StreamIt provides a high-level stream language in which the inherent complexities of the domain are handled by a stream-aware compiler. A program is written as a set of autonomous filters, each of which contains its own memory and communicates with its neighbors through data channels. This exposes the parallelism and communication patterns, giving more power to the compiler. We believe that this will lead to a dramatic improvement in programmer productivity, program robustness, and portability in three distinct communities. The first target is heterogeneous distributed computing, in which unreliable devices are communicating in ad-hoc networks with applications spread across multiple nodes. To address the challenge of partitioning a program across multiple nodes, StreamIt provides a "single machine abstraction" to the programmer. Filters are specified at the granularity of the algorithm, and the compiler maps them to nodes according to run-time resource constraints. StreamIt will also provide high-level support for fault-tolerance. If an unreliable node fails, a lightweight runtime system can leverage static compiler analysis to recover crucial data from a backup node, automatically relocate the computation, and continue processing.The second focus area for StreamIt is domain-specific optimization for the embedded and DSP communities. To eradicate costly hand-written assembly code, StreamIt will include domain-specific code generation strategies such as cache-constrained scheduling, stream-aware storage management, and new calling conventions for stream programs. To lower the expertise needed for high-level algorithm design, StreamIt will also include novel algorithmic transformations such as detection of linear nodes and automated translation to the frequency domain.The third thrust of StreamIt is with regards to high-performance computer architecture. StreamIt aims to provide a common machine language (CML) for the emerging class of communication-exposed machines. Just as C was a CML for von-Neumann architectures, StreamIt will hide the variations from one machine from another while capturing the properties that are important for obtaining performance. This will provide the foundation for a portable programming model for communication-exposed machines.A final focus of StreamIt is one that interweaves through all of the preceding themes: a full characterization and understanding of the streaming application domain. The project will include the study and implementation of a wide range of state-of-the-art stream algorithms, thereby exposing their common properties and providing a framework for future research on stream languages and compilation.To show the feasibility of this project, we have created a prototype version of the StreamIt language, a large collection of streaming programs written in StreamIt, a compiler that maps StreamIt to uniprocessors, and an optimizing compiler that maps StreamIt onto the Raw communication-exposed architecture.