NeTS: Medium: Streaming Data Analytics over Programmable Datacenter Networks

今天的数据中心扮演着许多重要的角色，其中最重要的是支持大数据分析。尤其是流数据，它是一种来自网络点击流和金融交易等应用程序的普遍工作负载。流数据分析带来了独特的挑战，因为数据以非常高的速度连续到达，这使得获得实时洞察变得困难。然而，数据中心网络正变得越来越可编程，这意味着数据流可能经过的更多常见组件现在可以灵活地进行实时处理。此外，新技术使数据中心网络拓扑能够动态配置，以连接更有效地处理特定流所需的组件和服务器。该项目旨在构建一个框架和工具，利用这种新兴的可编程性来实现实时数据处理。它有潜力极大地提高数据分析的可扩展性、性能和能源效率，并提供关键的洞察力和实时响应。该项目将开发一个新的数据分析框架，旨在通过利用数据中心可编程性来改变执行流数据分析的方式。可编程元件集已经扩展到不仅包括服务器，还包括网络接口组件、现场可编程门阵列、专用集成电路和网络拓扑。该项目的愿景是共同优化所有组件，以在每个应用程序的设计空间中实现“最佳点”。研究人员旨在通过以下活动开发科学基础和实用技术来实现这一愿景：确定可编程数据中心网络可以提供的关键抽象，以支持应用级数据分析，设计实用的流数据分析框架以利用这些抽象，并开发调度算法以跨不同应用程序复用可编程资源。最后，研究人员将探索框架的高效和可重用实现，这些实现将作为案例研究应用于实际工作负载。该项目包括与行业从业者的合作，使研究理念和系统原型能够顺利地转化为实践。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Today's datacenters play many important roles, not the least of which is to support big data analytics. Streaming data, in particular, is a prevalent workload that comes from web clickstreams and applications like financial transactions. Streaming data analytics presents unique challenges, as the data arrives continuously at very high speeds, which makes deriving real-time insight about it difficult. However, datacenter networks are becoming more programmable, meaning that more of the common components that a data stream might pass through now have the flexibility to do real-time processing. In addition, new technologies are enabling the datacenter network topology to be dynamically configured to connect the components and servers needed to process a certain stream more efficiently. This project seeks to build a framework and tools that exploit this emerging programmability to enable real time data processing. It has the potential to drastically improve the scalability, performance and energy efficiency of data analytics, and to deliver critical insight and response in real time. This project will develop a new data analytics framework designed to transform the way of performing streaming data analytics by leveraging datacenter programmability. The set of programmable elements has expanded to include not only servers, but network interface components, field-programmable gate arrays, application-specific integrated circuits, and network topology. The vision of the project is to jointly optimize all components to achieve a "sweetspot" in the design space for each application. The researchers aim to develop the scientific foundations and practical techniques to realize this vision through the following activities: identifying key abstractions that programmable datacenter networks can provide to support application-level data analytics, designing a practical streaming data analytics framework to leverage these abstractions, and developing scheduling algorithms to multiplex the programmable resources across different applications. Finally, the investigators will explore efficient and reusable implementations of the framework, which will be applied to real-world workloads as case studies. The project includes collaboration with industry practitioners to enable research ideas and system prototypes to be smoothly transitioned into practice.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.