权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

CC* Integration-Large: SciStream: Architecture and Toolkit for Data Streaming between Federated Science Instruments

CC* Integration-Large：SciStream：联合科学仪器之间数据流的架构和工具包

基本信息

批准号：
2019073
负责人：
Rajkumar Kettimuthu
金额：
$ 85万
依托单位：
University of Chicago
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2020
资助国家：
美国
起止时间：
2020-10-01 至 2024-09-30
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=2019073&HistoricalAwards=false
关键词：
CC Integration Large SciStream Architecture

项目摘要

Scientific instruments are capable of generating data at very high speeds. However, with traditional file-based data movement and analysis methods, data are often processed at a much lower speed, leading to either operating the instruments at a lower speed or discarding a (significant) portion of the data without processing it. To address this issue, SciStream project will develop software tools to stream data at very high speeds from scientific instruments to supercomputers at a distant location. SciStream hides the complexities in network connections from the end user and provides a high level of security for all the network connections.The data producers (e.g., data acquisition applications on scientific instruments, simulations on supercomputers) and consumers (e.g., data analysis applications on high performance computing systems) may be in different security domains (and thus require bridging of those domains) and may, further, lack external network connectivity (and thus, require traffic forwarding proxies). SciStream establishes necessary bridging and end-to-end authentication between source and destination, while providing efficient memory-to-memory data streaming. Through the exploration of architectural and design choices and addressing issues of control protocols and security, SciStream will advance the understanding of the challenges in supporting high speed memory-to-memory data streaming between remote instruments in federated science environments.SciStream will benefit all scientific applications that require memory-to-memory data streaming between distributed instruments. Recent trends suggest that this is an important and growing requirement for many scientific applications. SciStream will help significantly reduce the time to solution for these applications, resulting in improved scientific productivity and thus far-reaching benefits for society. Key design choices such as application-agnostic streaming and support for best-effort streaming will make SciStream appealing to a broader science community. SciStream will engage with domain scientists, campus computing centers, and a scientific user facility to reach a wider audience. Through on-campus programs at the University of Chicago, SciStream will train under-represented students in networking. Additional details on SciStream can be found here: https://scistream.github.io/This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

科学仪器能够以非常高的速度产生数据。然而，使用传统的基于文件的数据移动和分析方法，数据的处理速度往往要慢得多，导致仪器以较低的速度运行，或者在不处理数据的情况下丢弃(相当大的)部分数据。为了解决这个问题，SciStream项目将开发软件工具，以非常高的速度将数据从科学仪器传输到遥远地点的超级计算机。数据生产者(例如，科学仪器上的数据采集应用、超级计算机上的模拟)和消费者(例如，高性能计算系统上的数据分析应用)可能位于不同的安全域(因此需要那些域的桥接)，并且可能进一步缺乏外部网络连接(并且因此需要流量转发代理)。SciStream在源和目标之间建立必要的桥接和端到端身份验证，同时提供高效的内存到内存数据流。通过探索架构和设计选择以及解决控制协议和安全问题，SciStream将促进对在联合科学环境中支持远程仪器之间的高速内存到内存数据流的挑战的理解。SciStream将使所有需要在分布式仪器之间进行内存到内存数据流的科学应用程序受益。最近的趋势表明，这是许多科学应用的一个重要且不断增长的要求。SciStream将帮助显著减少解决这些应用的时间，从而提高科学生产率，从而为社会带来深远的好处。关键的设计选择，如与应用程序无关的流和对尽力而为的流的支持，将使SciStream吸引更广泛的科学界。SciStream将与领域科学家、校园计算中心和科学用户设施接触，以接触到更广泛的受众。通过芝加哥大学的校园项目，SciStream将培训代表不足的学生建立网络。关于科学流的更多细节可以在这里找到：https://scistream.github.io/This奖反映了美国国家科学基金会的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

期刊论文数量（2）

专著数量（0）

科研奖励数量（0）

会议论文数量（0）

专利数量（0）

SciStream: Architecture and Toolkit for Data Streaming between Federated Science Instruments

DOI：
10.1145/3502181.3531475
发表时间：
2022-06
期刊：
Proceedings of the 31st International Symposium on High-Performance Parallel and Distributed Computing
影响因子：
0
作者：
Joaquín Chung;Wojciech Zacherek;AJ Wisniewski;Zhengchun Liu;Tekin Bicer;R. Kettimuthu;Ian T Foster
通讯作者：
Joaquín Chung;Wojciech Zacherek;AJ Wisniewski;Zhengchun Liu;Tekin Bicer;R. Kettimuthu;Ian T Foster

Evaluating SciStream (Federated Scientific Data Streaming Architecture) on FABRIC

评估 FABRIC 上的 SciStream（联合科学数据流架构）