III: Distributed Stream Integration

用于编程分布式应用程序的声明式、数据库风格的模型正被广泛采用，应用于从传感器到发布-订阅再到网络状态管理的各种领域。它们使开发人员可以为感兴趣的特定数据定义高级查询，而不必考虑数据源、通信协议或同步的详细信息。随着这种编程方法的发展，越来越需要在统一的表示（即视图）下抽象低级流数据源的变化；并集成，即连接来自大量来源的不同类型的流数据。与传统的分布式数据库甚至数据流管理系统相比，这些任务涉及到更多的分布式通信和协调。以考虑网络拓扑的方式对查询进行网络内计算，并分别优化每个流（或几个流）的处理变得至关重要。该提案开发了支持数据流集成的技术，包括用于流模式映射的语言，重点关注与组合分布式消息和维护定时信息相关的问题；通过网络快速建立查询计算路径的技术，用于需要连接和聚合在一起的数据流元素集；面向网络分布式计算的离线自适应网络感知查询优化技术。这些技术将在广泛的异构（传感器、无线和传统）网络中进行扩展，并将在环境监测应用中进行评估。智力上的优点是开发了跨大型、高度分布的流生成源网络执行查询的新技术；当对数据项的访问成本不统一且查询处理需要分布式通信时，这增加了对自适应查询处理空间的理解，以及关于离线与自适应优化以及相对于优化粒度的权衡。更广泛的影响包括分布式流集成能力的发展，可以直接解决网络和环境监测领域中许多新兴和众所周知的挑战。教育部分包括培训两名博士生，以及在研究生和高级本科课程中教授流数据集成。项目网址：http://www.cis.upenn.edu/~zives/stream-integration/

Declarative, database-style models for programming distributed applications are becoming widely adopted, in a variety of realms ranging from sensors to publish-subscribe to network state management. They free the developerto define high-level queries for the specific data of interest,without regard to details about data sources, communications protocols,or synchronization.As this approach to programming gains momentum, there is increasing needto abstract low-level stream data source variations away under a uniform representation, i.e., a view; and to integrate, i.e., conjoin, different types of stream data from large numbers of sources. Such tasks involvemuch more distributed communication and coordination than in traditionaldistributed databases or even data stream management systems. It becomes essential to do in-network computation of the query, and to optimize the processing of each stream (or few streams) separately, in a way that considers the topology of the network.This proposal develops the technologies to support integration of data streams, including languages for stream schema mappings, focusing on issues relatingto combining distributed messages and maintaining timing information;techniques for rapidly establishing query computation paths through a network,for sets of data stream elements that need to be joined and aggregated together; offline and adaptive, network-aware query optimization techniques fordistributed computation in the network. These techniques will scaleacross widely heterogeneous (sensor, wireless, andconventional) networks, and will be evaluated in environmental monitoringapplications.The intellectual merit is the development of new techniques for performingqueries across large, highly distributed networks of stream-producing sources;this increases understanding of the adaptive query processing space when accesscosts to data items are non-uniform and query processing requires distributedcommunication, and the trade-offs with respect to offline versus adaptiveoptimization and relative to optimization granularity. The broader impactincludes the development of distributed stream integration capabilities thatcan directly address a number of emerging and well-known challenges in the network and environmental monitoring domains. The educational component includes the training of two PhD students, and the teaching of stream data integration in graduate and advanced undergraduate courses.Project URL: http://www.cis.upenn.edu/~zives/stream-integration/