EAGER: Characterization, Prediction, and Management of Latency Variation in Virtualized HPC and Warehouse Scale Systems
EAGER:虚拟化 HPC 和仓库规模系统中延迟变化的表征、预测和管理
基本信息
- 批准号:1642542
- 负责人:
- 金额:$ 30万
- 依托单位:
- 依托单位国家:美国
- 项目类别:Standard Grant
- 财政年份:2016
- 资助国家:美国
- 起止时间:2016-08-01 至 2020-07-31
- 项目状态:已结题
- 来源:
- 关键词:
项目摘要
Recent advancements in the Linux operating system have enabled widespread use of lightweight host-based virtualization, or containers. Containers can be used to deploy workloads in High Performance Computing (HPC) and warehouse computing environments. However, research has shown that containers can have adverse effects on the latency of the application's network traffic, a metric that many HPC applications are sensitive to. This project addresses limitations of prior research that focus primarily on the mean latency in HPC applications but do not fully analyze variation in network latency as a factor in application performance. The project is to study, predict, and manage variation in network latency and its impact to HPC and warehouse computing application performance. Results will have impact to the sustained performance of HPC and warehouse scale applications, an area of national interest. Results from this project will give insight and direction to new research in the characterization, prediction, and management of variation of latency in highly parallel systems. Research tasks include studying the tradeoffs of host-managed network traffic versus application-managed network interfaces in the context of containers, identifying the granularity of virtualization appropriate for high speed communication between workers in different computing platforms and how these choices affect message latency variation, and studying the effects of zero-copy messaging between Docker containers in combination with other network communication technologies. This project will develop a thorough characterization and analysis of the mechanisms for managing latency that have become available with recent technologies. The project studies technologies that are not yet widely used in HPC applications, including Docker containers in combination with SR-IOV, DPDK, Ethernet, and InfiniBand. Experimental design and testing of environment configurations and application resource allocation strategies will lead to the development and validation of descriptive and predictive models of the variation in network latency for the selected technologies, and will provide guidance for the development of best practice recommendations for configuration of container networking. The project, with a focus on variation in network latency, lays the groundwork for a larger project to develop an experimental platform and framework for the evaluation of the use of emerging technologies in HPC and warehouse computing systems. Funds will support two Ph.D. students at Clemson University in South Carolina, an EPSCoR state.
Linux操作系统的最新进展使得轻量级的基于主机的虚拟化或容器得到广泛使用。 容器可用于在高性能计算(HPC)和仓库计算环境中部署工作负载。 然而,研究表明,容器可能会对应用程序的网络流量延迟产生不利影响,这是许多HPC应用程序敏感的指标。 该项目解决了先前研究的局限性,这些研究主要集中在HPC应用程序的平均延迟上,但没有充分分析网络延迟的变化作为应用程序性能的一个因素。 该项目旨在研究、预测和管理网络延迟的变化及其对HPC和仓库计算应用程序性能的影响。 结果将对HPC和仓库规模应用程序的持续性能产生影响,这是一个国家利益领域。 该项目的结果将为高度并行系统中延迟变化的表征、预测和管理方面的新研究提供见解和方向。 研究任务包括研究容器上下文中主机管理的网络流量与应用程序管理的网络接口之间的权衡,确定适合不同计算平台中的工作者之间高速通信的虚拟化粒度以及这些选择如何影响消息延迟变化,并研究Docker容器之间的零拷贝消息传递与其他网络通信技术的结合。 该项目将开发一个全面的表征和分析的机制,用于管理延迟,已成为可用的最新技术。该项目研究尚未在HPC应用中广泛使用的技术,包括Docker容器与SR-IOV,DPDK,以太网和InfiniBand的组合。 环境配置和应用程序资源分配策略的实验设计和测试将导致开发和验证所选技术的网络延迟变化的描述性和预测性模型,并将为制定容器网络配置的最佳实践建议提供指导。该项目的重点是网络延迟的变化,为一个更大的项目奠定了基础,该项目旨在开发一个实验平台和框架,用于评估HPC和仓库计算系统中新兴技术的使用。 基金将支持两个博士学位。南卡罗来纳州的克莱姆森大学的学生。
项目成果
期刊论文数量(4)
专著数量(0)
科研奖励数量(0)
会议论文数量(0)
专利数量(0)
Random access in nondelimited variable-length record collections for parallel reading with Hadoop
- DOI:10.23919/inm.2017.7987424
- 发表时间:2017-05
- 期刊:
- 影响因子:0
- 作者:Jason W. Anderson;Christopher Gropp;Linh Ngo;A. Apon
- 通讯作者:Jason W. Anderson;Christopher Gropp;Linh Ngo;A. Apon
Narrowing the Gap: Effects of Latency with Docker in IP Networks
缩小差距:IP 网络中 Docker 的延迟影响
- DOI:
- 发表时间:2016
- 期刊:
- 影响因子:0
- 作者:Higgs, Corbin;Anderson, Jason
- 通讯作者:Anderson, Jason
Measuring Network Latency Variation Impacts to High Performance Computing Application Performance
测量网络延迟变化对高性能计算应用程序性能的影响
- DOI:10.1145/3184407.3184427
- 发表时间:2018
- 期刊:
- 影响因子:0
- 作者:Underwood, Robert;Anderson, Jason;Apon, Amy
- 通讯作者:Apon, Amy
{{
item.title }}
{{ item.translation_title }}
- DOI:
{{ item.doi }} - 发表时间:
{{ item.publish_year }} - 期刊:
- 影响因子:{{ item.factor }}
- 作者:
{{ item.authors }} - 通讯作者:
{{ item.author }}
数据更新时间:{{ journalArticles.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ monograph.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ sciAawards.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ conferencePapers.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ patent.updateTime }}
Amy Apon其他文献
A component-based end-to-end simulation of the Linux file system
- DOI:
10.1007/s10270-012-0253-0 - 发表时间:
2012-06-17 - 期刊:
- 影响因子:3.200
- 作者:
Hai Nguyen;Amy Apon - 通讯作者:
Amy Apon
On modeling contention for shared caches in multi-core processors with techniques from ecology
- DOI:
10.1007/s11047-012-9348-3 - 发表时间:
2012-10-20 - 期刊:
- 影响因子:1.600
- 作者:
Wesley Emeneker;Amy Apon - 通讯作者:
Amy Apon
Amy Apon的其他文献
{{
item.title }}
{{ item.translation_title }}
- DOI:
{{ item.doi }} - 发表时间:
{{ item.publish_year }} - 期刊:
- 影响因子:{{ item.factor }}
- 作者:
{{ item.authors }} - 通讯作者:
{{ item.author }}
{{ truncateString('Amy Apon', 18)}}的其他基金
MRI: Acquisition of a Cyberinstrument for Interdisciplinary Computational Science and Engineering
MRI:获取跨学科计算科学与工程的网络仪器
- 批准号:
1725573 - 财政年份:2017
- 资助金额:
$ 30万 - 项目类别:
Standard Grant
II NEW: Infrastructure to Support Research in Network-Aware Data-Intensive Computing
II 新:支持网络感知数据密集型计算研究的基础设施
- 批准号:
1405767 - 财政年份:2014
- 资助金额:
$ 30万 - 项目类别:
Continuing Grant
Exploring Parallelization of Nearest Neighbor Search and Clustering in High-Dimensional Space on Emerging Parallel Architectures with Applications in Computer Vision
探索高维空间中最近邻搜索和聚类在新兴并行架构上的并行化及其在计算机视觉中的应用
- 批准号:
0947679 - 财政年份:2009
- 资助金额:
$ 30万 - 项目类别:
Standard Grant
CASC 20th Anniversary Symposium: Changing Science and Engineering - the Impact of HPC
中国航天科技集团20周年研讨会:改变科学与工程——高性能计算的影响
- 批准号:
0961135 - 财政年份:2009
- 资助金额:
$ 30万 - 项目类别:
Standard Grant
MRI: Acquisition of a Supercomputing Cluster for Computational and Data-Intensive Applications in Science and Engineering
MRI:收购超级计算集群,用于科学和工程中的计算和数据密集型应用
- 批准号:
0722625 - 财政年份:2007
- 资助金额:
$ 30万 - 项目类别:
Standard Grant
Collaborative Project: Adaptation of Globus Toolkit 3 Tutorials for Undergraduate Computer Science Students
合作项目:为计算机科学本科生改编 Globus Toolkit 3 教程
- 批准号:
0410966 - 财政年份:2004
- 资助金额:
$ 30万 - 项目类别:
Standard Grant
MRI: Acquisition of a Computing Cluster for High-End Applications in Science and Engineering (CHASE)
MRI:收购用于科学与工程高端应用的计算集群 (CHASE)
- 批准号:
0421099 - 财政年份:2004
- 资助金额:
$ 30万 - 项目类别:
Standard Grant
Experimental Software Systems: A Parallel Message-Passing Platform for Atomic Structure Calculations
实验软件系统:用于原子结构计算的并行消息传递平台
- 批准号:
9806521 - 财政年份:1998
- 资助金额:
$ 30万 - 项目类别:
Standard Grant
Experimental Software Systems: A Parallel Message-Passing Platform for Atomic Structure Calculations
实验软件系统:用于原子结构计算的并行消息传递平台
- 批准号:
9996143 - 财政年份:1998
- 资助金额:
$ 30万 - 项目类别:
Standard Grant
CISE Research Instrumentation: Application and Evaluation of a Low Cost Parallel Processing Platform
CISE 研究仪器:低成本并行处理平台的应用和评估
- 批准号:
9617384 - 财政年份:1997
- 资助金额:
$ 30万 - 项目类别:
Standard Grant
相似海外基金
Characterization of Renal Allograft Fibrosis and Prediction of Outcome Using a Quantitative MRI Approach
使用定量 MRI 方法表征同种异体肾纤维化并预测结果
- 批准号:
10279690 - 财政年份:2021
- 资助金额:
$ 30万 - 项目类别:
Characterization of Renal Allograft Fibrosis and Prediction of Outcome Using a Quantitative MRI Approach
使用定量 MRI 方法表征同种异体肾纤维化并预测结果
- 批准号:
10618243 - 财政年份:2021
- 资助金额:
$ 30万 - 项目类别:
A dynamic characterization of the blood metabolome for the prediction of tumor onset in Li-Fraumeni syndrome
血液代谢组的动态特征用于预测 Li-Fraumeni 综合征肿瘤的发生
- 批准号:
458334 - 财政年份:2021
- 资助金额:
$ 30万 - 项目类别:
Studentship Programs
Characterization of Renal Allograft Fibrosis and Prediction of Outcome Using a Quantitative MRI Approach
使用定量 MRI 方法表征同种异体肾纤维化并预测结果
- 批准号:
10447657 - 财政年份:2021
- 资助金额:
$ 30万 - 项目类别:
MTM 1: An explainable AI system for microbiome characterization and microbiome-based host-phenotype prediction
MTM 1:一个可解释的人工智能系统,用于微生物组表征和基于微生物组的宿主表型预测
- 批准号:
2025451 - 财政年份:2020
- 资助金额:
$ 30万 - 项目类别:
Standard Grant
Quantitative MRI for structural and functional characterization of head and neck cancer for outcome prediction and biological individualization of MR-guided radiotherapy
用于头颈癌结构和功能表征的定量 MRI,用于 MR 引导放射治疗的结果预测和生物个体化
- 批准号:
437907809 - 财政年份:2020
- 资助金额:
$ 30万 - 项目类别:
Research Grants
Characterization and Prediction of Viral Capsid Geometries
病毒衣壳几何形状的表征和预测
- 批准号:
1951678 - 财政年份:2020
- 资助金额:
$ 30万 - 项目类别:
Continuing Grant
III: Small: Prediction and Characterization of Extreme Events in Spatio-Temporal Data.
III:小:时空数据中极端事件的预测和表征。
- 批准号:
2006633 - 财政年份:2020
- 资助金额:
$ 30万 - 项目类别:
Continuing Grant
Characterization of the interaction of bacterial determinants and host cells during systemic inflammation for sepsis prediction
全身炎症过程中细菌决定簇与宿主细胞相互作用的表征,用于脓毒症预测
- 批准号:
427775125 - 财政年份:2019
- 资助金额:
$ 30万 - 项目类别:
Clinical Research Units
Sediment hazard prediction with high spatial resolution by characterization of the slope failure events triggered by 2018 torrential rainfall in western Japan
通过表征 2018 年日本西部暴雨引发的斜坡失稳事件进行高空间分辨率泥沙灾害预测
- 批准号:
19K01154 - 财政年份:2019
- 资助金额:
$ 30万 - 项目类别:
Grant-in-Aid for Scientific Research (C)