III-COR-Small: Towards More Flexible, Expressive and Robust Stream Systems

点击翻译按钮获取中文摘要

This award is funded under the American Recovery and Reinvestment Act of 2009 (Public Law 111-5).Current stream engines are mostly stand-alone systems, whereas in many applications, stream processing will be one component of a larger information system. Coupling with other components, such as user interfaces, transactional data and archives will be increasingly important. For stream queries to be robust over extended execution periods, they must have the means to adapt to both internal and external changes. Changes in input rates, time lags and data distributions can cause shifts in internal memory and processing loads. Operators must adapt to these changes both local and in concert with other operators. Variations in client demands create opportunities for improved resource use to which operators must adapt. For a stream processing system to be robust in the face of changing workloads and possible system faults, the architecture must have levels of flexibility and adaptivity not currently existing. The team proposes three approaches to developing the necessary flexibility. They will use a formal analysis that will provide precise notions of time and progress, in order to provide criteria and metrics for a variety of situations. In addition, they will elaborate operator and architecture design activities that will then be implemented and evaluated in two different data stream systems. In addition to faculty at Portland State University and graduate students there, the team has an ongoing collaboration with AT&T Research, who will provide access for testing the new system. The techniques developed on this project will broaden the number of applications that can be reasonably served with data stream systems, and by working with commercial systems, the adoption of those techniques into next-generation products will be accelerated. In addition to a new course in steam systems, the team works with high school interns each summer, who are recruited through the Saturday Academy program at Portland State.