CAREER: Scalable and Adaptive Edge Stream Processing

职业：可扩展和自适应边缘流处理

• 批准号: 1943071
• 负责人: Liting Hu
• 金额: $ 48.87万
• 依托单位: Florida International University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-09-01 至 2022-01-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1943071&HistoricalAwards=false
• 关键词: CAREER; Scalable; Adaptive; Edge; Stream

Internet-of-Things (IoT) applications such as self-driving cars, augmented reality, interactive gaming, and event monitoring have a tremendous potential to improve our lives. These applications generate a large influx of sensor data at massive scales. Under many time-critical scenarios, these massive data streams must be processed in a very short time to derive actionable intelligence. This CAREER project aims to support time-critical IoT applications by applying the stream processing paradigm to the Edge computing architecture. The success of this research will benefit many time-critical IoT applications in the areas such as factory automation, the tactile internet, autonomous vehicles, and process automation. It will also substantially improve the performance profiles of a variety of data processing systems, such as wide-area data analytics systems, mobile data access systems, event tracking systems, and streaming databases. As an integral part of its research program, this CAREER project involves K-12, undergraduate and graduate level education in partnership with the local Public School system.Specifically, this CAREER project will build a scalable and adaptive Edge stream processing engine, which enables fast stream processing of a large number of concurrent IoT queries in the dynamic, heterogeneous Edge environment. This work includes three primary research directions. First, a new dynamic dataflow graph abstraction will be implemented, which automatically chains, parallelizes and replicates stream operators to adapt to the Edge dynamics and handle failures in a scalable way. Second, a new customizable data shuffling service abstraction will be implemented, which customizes the data shuffling path (e.g., ring shuffle, hierarchical tree shuffle, butterfly wrap shuffle) at runtime for the given network topology and workload. Third, a fully decentralized architecture with many distributed schedulers will be implemented, in which each scheduler operates autonomously to process IoT queries. All three parts of the project will be prototyped and implemented on real-world stream processing systems and validated by performing real-world experiments.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

物联网（IoT）应用，如自动驾驶汽车、增强现实、互动游戏和事件监控，具有改善我们生活的巨大潜力。这些应用产生了大量的传感器数据。在许多时间紧迫的情况下，必须在很短的时间内处理这些大量数据流，以获得可操作的情报。这个CAREER项目旨在通过将流处理范例应用于边缘计算架构来支持时间关键型物联网应用。这项研究的成功将有利于工厂自动化、触觉互联网、自动驾驶汽车和过程自动化等领域的许多时间关键型物联网应用。它还将大大提高各种数据处理系统的性能概况，如广域数据分析系统、移动数据访问系统、事件跟踪系统和流数据库。作为其研究计划的一个组成部分，这个职业项目涉及K-12、本科和研究生水平的教育，与当地公立学校系统合作。具体来说，这个CAREER项目将构建一个可扩展和自适应的Edge流处理引擎，它可以在动态、异构的Edge环境中对大量并发物联网查询进行快速流处理。本工作包括三个主要研究方向。首先，将实现一个新的动态数据流图抽象，它自动链接、并行和复制流操作符，以适应Edge动态，并以可扩展的方式处理故障。其次，将实现一个新的可定制的数据洗牌服务抽象，它在运行时针对给定的网络拓扑和工作负载定制数据洗牌路径（例如，环洗牌、分层树洗牌、蝴蝶包裹洗牌）。第三，将实现具有许多分布式调度器的完全分散的架构，其中每个调度器都自主运行以处理物联网查询。项目的所有三个部分都将在现实世界的流处理系统上进行原型设计和实施，并通过现实世界的实验进行验证。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

SR3: Customizable Recovery for Stateful Stream Processing Systems

• DOI: 10.1145/3423211.3425681
• 发表时间: 2020-12
• 期刊: Proceedings of the 21st International Middleware Conference
• 作者: Hailu Xu;Pinchao Liu;Susana Cruz-Diaz;D. D. Silva-D.;Liting Hu

DART: A Scalable and Adaptive Edge Stream Processing Engine

DART：可扩展的自适应边缘流处理引擎

• 发表时间: 2021
• 期刊: 2021 USENIX Annual Technical Conference (USENIX ATC 21
• 作者: Liu, Pinchao;Silva, Dilma Da;Hu, Liting.
• 通讯作者: Hu, Liting.