CAREER: Scalable and Adaptive Edge Stream Processing

职业：可扩展和自适应边缘流处理

• 批准号: 2313737
• 负责人: Liting Hu
• 金额: $ 48.87万
• 依托单位: University of California-Santa Cruz
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-10-01 至 2025-08-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2313737&HistoricalAwards=false
• 关键词: CAREER; Scalable; Adaptive; Edge; Stream

Internet-of-Things (IoT) applications such as self-driving cars, augmented reality, interactive gaming, and event monitoring have a tremendous potential to improve our lives. These applications generate a large influx of sensor data at massive scales. Under many time-critical scenarios, these massive data streams must be processed in a very short time to derive actionable intelligence. This CAREER project aims to support time-critical IoT applications by applying the stream processing paradigm to the Edge computing architecture. The success of this research will benefit many time-critical IoT applications in the areas such as factory automation, the tactile internet, autonomous vehicles, and process automation. It will also substantially improve the performance profiles of a variety of data processing systems, such as wide-area data analytics systems, mobile data access systems, event tracking systems, and streaming databases. As an integral part of its research program, this CAREER project involves K-12, undergraduate and graduate level education in partnership with the local Public School system.Specifically, this CAREER project will build a scalable and adaptive Edge stream processing engine, which enables fast stream processing of a large number of concurrent IoT queries in the dynamic, heterogeneous Edge environment. This work includes three primary research directions. First, a new dynamic dataflow graph abstraction will be implemented, which automatically chains, parallelizes and replicates stream operators to adapt to the Edge dynamics and handle failures in a scalable way. Second, a new customizable data shuffling service abstraction will be implemented, which customizes the data shuffling path (e.g., ring shuffle, hierarchical tree shuffle, butterfly wrap shuffle) at runtime for the given network topology and workload. Third, a fully decentralized architecture with many distributed schedulers will be implemented, in which each scheduler operates autonomously to process IoT queries. All three parts of the project will be prototyped and implemented on real-world stream processing systems and validated by performing real-world experiments.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

自动驾驶汽车、增强现实、互动游戏和事件监控等物联网(IoT)应用程序具有改善我们生活的巨大潜力。这些应用程序产生了大量大规模的传感器数据。在许多时间关键型场景下，必须在非常短的时间内处理这些海量数据流，以获取可操作的情报。该职业项目旨在通过将流处理范例应用于边缘计算架构来支持时间关键型物联网应用。这项研究的成功将使工厂自动化、触觉互联网、自动驾驶汽车和过程自动化等领域的许多时间关键型物联网应用受益。它还将大大改善各种数据处理系统的性能，如广域数据分析系统、移动数据访问系统、事件跟踪系统和流数据库。作为其研究计划的组成部分，该职业项目涉及与当地公立学校系统合作的K-12、本科生和研究生水平的教育。具体地说，该职业项目将构建一个可扩展和自适应的边缘流处理引擎，该引擎能够在动态、异构的边缘环境中快速处理大量并发的物联网查询。本工作包括三个主要研究方向。首先，实现了一种新的动态数据流图抽象，它自动链接、并行和复制流运算符，以适应Edge的动态变化并以可扩展的方式处理故障。其次，将实现新的可定制的数据洗牌服务抽象，其针对给定的网络拓扑和工作负载在运行时定制数据洗牌路径(例如，环形洗牌、分层树洗牌、蝴蝶包裹洗牌)。第三，将实现具有多个分布式调度器的完全去中心化的体系结构，其中每个调度器自主操作来处理物联网查询。该项目的所有三个部分都将在真实世界的流处理系统上进行原型和实施，并通过执行真实世界的实验进行验证。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Achieving Online and Scalable Information Integrity by Harnessing Social Spam Correlations

• DOI: 10.1109/access.2023.3236604
• 发表时间: 2023
• 期刊: IEEE Access
• 影响因子: 3.9
• 作者: Hailu Xu;Pinchao Liu;Boyuan Guan;Qingyang Wang;Dilma Da Silva;Liting Hu

DART: A Scalable and Adaptive Edge Stream Processing Engine

DART：可扩展的自适应边缘流处理引擎

• 发表时间: 2021
• 期刊: 2021 USENIX Annual Technical Conference (USENIX ATC 21
• 作者: Liu, Pinchao;Silva, Dilma Da;Hu, Liting.
• 通讯作者: Hu, Liting.

Towards low-latency I/O services for mixed workloads using ultra-low latency SSDs

• DOI: 10.1145/3524059.3532378
• 发表时间: 2022-06
• 期刊: Proceedings of the 36th ACM International Conference on Supercomputing
• 作者: Mingzhe Liu;Haikun Liu;Chencheng Ye;Xiaofei Liao;Hai Jin;Yu Zhang;Ran Zheng;Liting Hu

OrcoDCS: An IoT-Edge Orchestrated Online Deep Compressed Sensing Framework

• DOI: 10.1109/icdcsw60045.2023.00007
• 发表时间: 2023-07
• 期刊: 2023 IEEE 43rd International Conference on Distributed Computing Systems Workshops (ICDCSW)
• 作者: Cheng-Wei Ching;Chirag Gupta;Zisen Huang;Liting Hu

Adaptive Fragment-Based Parallel State Recovery for Stream Processing Systems

• DOI: 10.1109/tpds.2023.3251997
• 发表时间: 2023-08
• 期刊: IEEE Transactions on Parallel and Distributed Systems
• 影响因子: 5.3
• 作者: Hailu Xu;Pinchao Liu;Sarker Tanzir Ahmed;Dilma Da Silva;Liting Hu