Collaborative Research:PPoSS:Planning: Streamware - A Scalable Framework for Accelerating Streaming Data Science

在重大挑战的科学应用中，传感和仪器基础设施产生的海量数据往往在一小段时间后失去价值。因此，为了从数据中获取可操作的情报，流分析(即分析动态数据的能力)变得越来越关键。此外，现代计算系统是高度异构性的，由处理器、加速器和大型高带宽外部存储器组成。要开发可扩展的流分析应用程序，需要解决整个系统堆栈--从应用程序到目标平台--的挑战。在这方面，这一规划项目正在确定一套全面的研究挑战、目标、关键创新和时间表，涉及算法和应用、系统软件、软硬件协同设计和计算机体系结构。该项目汇集了一个由应用程序开发人员和用户、计算机科学家和数据科学家组成的社区，他们的兴趣在于构建面向各种可伸缩系统的流数据科学应用程序。该项目正在展示如何使用隐私保护流图学习安全智能电网作为驱动应用程序来实现显著的跨堆栈性能改进的初步结果。现代数据科学应用程序的特点是高度分散、分布式，需要在数千或数百万个边缘平台上的本地化分析和云/数据中心中的海量集中式分析之间进行组合和协调，并要求对流数据进行实时分析。要实现大挑战流数据科学应用程序的可扩展性能，需要一个框架，允许开发人员无缝构建针对各种可扩展系统的这些应用程序。该规划项目正在对开发开源框架StreamWare的大型提案进行初步研究，该框架将使用户能够开发流数据科学应用程序。该项目正在建立一个由应用程序开发人员和用户、计算机科学家和数据科学家组成的社区，他们将成为StreamWare框架的早期采用者和开发人员。在与领域专家协商后，正在生成StreamWare的关键数据科学核心的清单，并正在评估其现有的最先进的算法和硬件IP，以确定性能限制和改进机会。该项目还阐明了能够在不同平台上表示和操作流数据的新颖抽象的要求。该项目使用安全智能电网的隐私保护流图学习作为激励应用程序，使用共生可伸缩性的新概念显示端到端可伸缩性的初步证据，该概念捕捉到StreamWare跨层优化的影响。这一规划项目的预期结果包括一份将在大笔拨款中进行的研究活动的建议，关于调查活动结果的出版物和未来支持流数据科学的研究方向，以及未来研究生和本科生课程的课程。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

In grand-challenge scientific applications, the enormous amount of data produced by the sensing and instrumentation infrastructure often loses its value after a small window of time. Thus, to obtain actionable intelligence from the data, streaming analytics, i.e., the ability to analyze in-motion data, is increasingly becoming critical. Moreover, modern computing systems are highly heterogeneous, consisting of processors, accelerators, and large high-bandwidth external memories. To develop scalable streaming analytics applications, challenges across the full system stack -- from application to target platform -- need to be addressed. In this regard, this planning project is identifying a comprehensive set of research challenges, goals, key innovations and timelines in algorithms and applications, systems software, hardware-software co-design, and computer architecture. This project is bringing together a community of application developers and users, computer scientists, and data scientists, whose interests lie in building streaming data science applications targeting a wide variety of scalable systems. This project is demonstrating preliminary results on how it will achieve significant cross-stack performance improvements using Privacy Preserving Streaming Graph Learning for Secure Smart Grids as the driving application.Modern data-science applications are characterized as being highly decentralized, distributed and requiring composition and orchestration between localized analytics on thousands or millions of edge platforms and massive centralized analytics in cloud/data centers, as well as requiring real-time analytics on streaming data. To enable scalable performance of grand-challenge streaming data-science applications, a framework that allows developers to seamlessly build these applications targeting a wide variety of scalable systems is needed. This planning project is conducting preliminary research towards a large proposal for developing an opensource framework, StreamWare, that will enable users to develop streaming data-science applications. This project is establishing a community of application developers and users, computer scientists, and data scientists who would serve as early adopters and developers of the StreamWare framework. In consultation with domain experts, a list of key data-science kernels for StreamWare is being generated, and their existing state-of-the-art algorithms and hardware IPs are being evaluated to identify performance limitations and opportunities for improvement. This project is also articulating the requirements of novel abstractions that can represent and operate on streaming data on heterogeneous platforms. This project uses Privacy Preserving Streaming Graph Learning for Secure Smart Grids as a motivating application to show preliminary evidence of end-to-end scalability using a novel notion of symbiotic scalability that captures the impact of StreamWare's cross-layer optimizations. The expected outcomes of this planning project include a proposal for the research activities to be carried out in the large grant, publications on the results of the survey activities and future research directions for enabling streaming data science, and curricula for future graduate and undergraduate courses.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.