ITR Collaborative Proposal: Aurora - Enabling Stream-Based Monitoring Applications

有许多现有的和新出现的应用程序需要对快速、大容量的数据流进行复杂而及时的处理。这些应用的示例包括环境监控、投资组合管理、资产跟踪和交通管理。这些应用程序通常被称为监控应用程序，它们跟踪来自众多连续数据流的数据（来自诸如传感器网络和股票馈送之类的源），针对异常活动的迹象对其进行过滤，并且为了聚合、减少和关联的目的而及时地处理它们。当前，监视应用通常使用自定义，特定于应用程序的代码-没有通用的软件基础设施可以有效地满足这些应用程序的处理和性能要求。该项目将通过开发适当的抽象、机制、策略和协议来研究与通用数据流处理系统的设计和体系结构相关的关键问题。特别是，该项目将开发新的面向流的语言，查询执行和优化技术，以及资源分配算法。该项目还将通过解决面向流的负载共享、动态操作符放置和高可用性等问题来解决分布式流处理中出现的挑战和机遇。在这两种情况下，算法将由目标应用程序的服务质量期望驱动。

There is a host of existing and newly-emerging applications that require sophisticated and timely processing of fast, high-volume data streams. Examples of these applications include environmental monitoring, portfolio management, asset tracking, and traffic management. These applications, often referred to as monitoring applications, track data from numerous continuous data streams (coming from such sources as sensor networks and stock feeds), filtering them for signs of abnormal activity, and processing them in a timely fashion for purposes of aggregation, reduction, and correlation.Currently, monitoring applications are commonly implemented using custom, application-specific code - there is no general-purpose software infrastructure that can efficiently and effectively meet the processing and performance requirements of these applications. The project will investigate the key issues related to the design and architecture of general-purpose data-stream processing systems by developing proper abstractions, mechanisms, policies, and protocols. In particular, the project will develop novel stream-oriented languages, query execution and optimization techniques, and resource allocation algorithms. The project will also address the challenges and opportunities that arise in distributed stream processing by addressing issues such as stream-oriented load sharing, dynamic operator placement, and high availability. In both cases, the algorithms will be driven by the Quality-of-Service expectations of the target applications.