ITR Collaborative Proposal: Aurora - Enabling Stream-Based Monitoring Applications

有许多现有的和新兴的应用程序需要复杂和及时地处理快速、大容量的数据流。这些应用程序的示例包括环境监控、投资组合管理、资产跟踪和交通管理。这些应用程序通常称为监控应用程序，它们跟踪来自大量连续数据流(来自传感器网络和股票馈送)的数据，过滤它们以发现异常活动的迹象，并及时处理它们，以便进行聚合、缩减和关联。目前，监控应用程序通常使用特定于应用程序的自定义代码实现-没有通用的软件基础设施可以高效和有效地满足这些应用程序的处理和性能要求。该项目将通过开发适当的抽象、机制、策略和协议来研究与通用数据流处理系统的设计和体系结构相关的关键问题。特别是，该项目将开发新的面向流的语言、查询执行和优化技术以及资源分配算法。该项目还将通过解决面向流的负载共享、动态操作员放置和高可用性等问题来解决分布式流处理中出现的挑战和机遇。在这两种情况下，算法都将由目标应用程序的服务质量预期驱动。

There is a host of existing and newly-emerging applications that require sophisticated and timely processing of fast, high-volume data streams. Examples of these applications include environmental monitoring, portfolio management, asset tracking, and traffic management. These applications, often referred to as monitoring applications, track data from numerous continuous data streams (coming from such sources as sensor networks and stock feeds), filtering them for signs of abnormal activity, and processing them in a timely fashion for purposes of aggregation, reduction, and correlation.Currently, monitoring applications are commonly implemented using custom, application-specific code - there is no general-purpose software infrastructure that can efficiently and effectively meet the processing and performance requirements of these applications. The project will investigate the key issues related to the design and architecture of general-purpose data-stream processing systems by developing proper abstractions, mechanisms, policies, and protocols. In particular, the project will develop novel stream-oriented languages, query execution and optimization techniques, and resource allocation algorithms. The project will also address the challenges and opportunities that arise in distributed stream processing by addressing issues such as stream-oriented load sharing, dynamic operator placement, and high availability. In both cases, the algorithms will be driven by the Quality-of-Service expectations of the target applications.