CAREER: Scalable and Adaptive Edge Stream Processing

物联网（IoT）应用，如自动驾驶汽车、增强现实、交互式游戏和事件监控，具有改善我们生活的巨大潜力。这些应用程序在大规模上产生大量的传感器数据。在许多时间紧迫的情况下，这些海量数据流必须在很短的时间内处理，以获得可操作的情报。这个CAREER项目旨在通过将流处理范式应用于边缘计算架构来支持时间关键型物联网应用。这项研究的成功将使工厂自动化、触觉互联网、自动驾驶汽车和过程自动化等领域的许多时间关键型物联网应用受益。它还将大大提高各种数据处理系统的性能，如广域数据分析系统、移动的数据访问系统、事件跟踪系统和流数据库。作为其研究计划的一个组成部分，该CAREER项目涉及K-12，本科和研究生教育，并与当地公立学校系统合作，具体来说，该CAREER项目将构建一个可扩展和自适应的边缘流处理引擎，该引擎可以在动态，异构的边缘环境中快速处理大量并发的物联网查询。这项工作包括三个主要的研究方向。首先，将实现一个新的动态BLOG图抽象，它自动链接、并行化和复制流操作符，以适应Edge动态并以可扩展的方式处理故障。第二，将实现新的可定制的数据混洗服务抽象，其定制数据混洗路径（例如，环混洗、分层树混洗、蝶形缠绕混洗）。第三，将实现一个具有许多分布式调度器的完全分散的架构，其中每个调度器都自主运行以处理物联网查询。该项目的所有三个部分都将在真实世界的流处理系统上进行原型设计和实现，并通过执行真实世界的实验进行验证。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Internet-of-Things (IoT) applications such as self-driving cars, augmented reality, interactive gaming, and event monitoring have a tremendous potential to improve our lives. These applications generate a large influx of sensor data at massive scales. Under many time-critical scenarios, these massive data streams must be processed in a very short time to derive actionable intelligence. This CAREER project aims to support time-critical IoT applications by applying the stream processing paradigm to the Edge computing architecture. The success of this research will benefit many time-critical IoT applications in the areas such as factory automation, the tactile internet, autonomous vehicles, and process automation. It will also substantially improve the performance profiles of a variety of data processing systems, such as wide-area data analytics systems, mobile data access systems, event tracking systems, and streaming databases. As an integral part of its research program, this CAREER project involves K-12, undergraduate and graduate level education in partnership with the local Public School system.Specifically, this CAREER project will build a scalable and adaptive Edge stream processing engine, which enables fast stream processing of a large number of concurrent IoT queries in the dynamic, heterogeneous Edge environment. This work includes three primary research directions. First, a new dynamic dataflow graph abstraction will be implemented, which automatically chains, parallelizes and replicates stream operators to adapt to the Edge dynamics and handle failures in a scalable way. Second, a new customizable data shuffling service abstraction will be implemented, which customizes the data shuffling path (e.g., ring shuffle, hierarchical tree shuffle, butterfly wrap shuffle) at runtime for the given network topology and workload. Third, a fully decentralized architecture with many distributed schedulers will be implemented, in which each scheduler operates autonomously to process IoT queries. All three parts of the project will be prototyped and implemented on real-world stream processing systems and validated by performing real-world experiments.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.