III: Distributed Stream Integration

点击翻译按钮获取中文摘要

Declarative, database-style models for programming distributed applications are becoming widely adopted, in a variety of realms ranging from sensors to publish-subscribe to network state management. They free the developerto define high-level queries for the specific data of interest,without regard to details about data sources, communications protocols,or synchronization.As this approach to programming gains momentum, there is increasing needto abstract low-level stream data source variations away under a uniform representation, i.e., a view; and to integrate, i.e., conjoin, different types of stream data from large numbers of sources. Such tasks involvemuch more distributed communication and coordination than in traditionaldistributed databases or even data stream management systems. It becomes essential to do in-network computation of the query, and to optimize the processing of each stream (or few streams) separately, in a way that considers the topology of the network.This proposal develops the technologies to support integration of data streams, including languages for stream schema mappings, focusing on issues relatingto combining distributed messages and maintaining timing information;techniques for rapidly establishing query computation paths through a network,for sets of data stream elements that need to be joined and aggregated together; offline and adaptive, network-aware query optimization techniques fordistributed computation in the network. These techniques will scaleacross widely heterogeneous (sensor, wireless, andconventional) networks, and will be evaluated in environmental monitoringapplications.The intellectual merit is the development of new techniques for performingqueries across large, highly distributed networks of stream-producing sources;this increases understanding of the adaptive query processing space when accesscosts to data items are non-uniform and query processing requires distributedcommunication, and the trade-offs with respect to offline versus adaptiveoptimization and relative to optimization granularity. The broader impactincludes the development of distributed stream integration capabilities thatcan directly address a number of emerging and well-known challenges in the network and environmental monitoring domains. The educational component includes the training of two PhD students, and the teaching of stream data integration in graduate and advanced undergraduate courses.Project URL: http://www.cis.upenn.edu/~zives/stream-integration/