EAGER: Stork Data Scheduler for Azure

EAGER：适用于 Azure 的 Stork 数据调度程序

• 批准号: 1115805
• 负责人: Tevfik Kosar
• 金额: $ 9.71万
• 依托单位: SUNY at Buffalo
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2011
• 资助国家: 美国
• 起止时间: 2011-04-15 至 2014-03-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1115805&HistoricalAwards=false
• 关键词: EAGER; Stork; Data; Scheduler; Azure

This project will further develop and enhance the Stork Data Scheduler to support Azure cloud computing environment, and to mitigate the end-to-end data handling bottleneck in data-intensive cloud computing applications. Stork data scheduler has been very actively used in many data-intensive application areas including coastal hazard prediction and storm surge modeling; oil flow and reservoir uncertainty analysis; numerical relativity and black hole collisions; educational video processing and behavioral assessment; digital sky imaging; and multiscale computational fluid dynamics. Making Stork available on the Azure environment will enable this already existing user base to easily migrate to the Azure Cloud Computing platform as well as other Azure application groups benefiting from the large-scale data handling capabilities of Stork.The Stork Data Scheduler for Azure will make a distinctive contribution to cloud computing community because it focuses on planning, scheduling, monitoring and management of data placement tasks and application-level end-to-end optimization of networked I/O for petascale data-intensive applications. Unlike existing approaches, it will treat data resources and the tasks related to data access and movement as first class entities just like computational resources and compute tasks, and not simply the side effect of computation. Stork data scheduler for Azure will provide enhanced functionality for cloud computing such as data aggregation and connection caching; peer-to-peer and streamed data management; early error detection, classification, and recovery in data transfers; scheduled storage management; optimal protocol tuning; and end-to-end performance prediction services. The Stork data scheduler for Azure will dramatically change how domain scientists perform their research by rapidly facilitating sharing of large amounts of data in cloud computing environments.

该项目将进一步开发和增强Stork Data Cloud，以支持Azure云计算环境，并缓解数据密集型云计算应用程序中的端到端数据处理瓶颈。Stork数据调度器在许多数据密集型应用领域中得到了非常积极的应用，包括沿海灾害预测和风暴潮建模;油流和油藏不确定性分析;数值相关性和黑洞碰撞;教育视频处理和行为评估;数字天空成像;和多尺度计算流体动力学。在Azure环境中提供Stork将使现有的用户群能够轻松迁移到Azure云计算平台以及其他Azure应用程序组，从而受益于Stork的大规模数据处理功能。面向Azure的Stork数据库将为云计算社区做出独特的贡献，因为它专注于规划，调度，数据放置任务的监控和管理，以及千万亿次数据密集型应用的联网I/O的应用级端到端优化。与现有方法不同，它将数据资源和与数据访问和移动相关的任务视为第一类实体，就像计算资源和计算任务一样，而不仅仅是计算的副作用。Azure的Stork数据调度器将为云计算提供增强的功能，例如数据聚合和连接缓存;点对点和流数据管理;数据传输中的早期错误检测，分类和恢复;计划存储管理;最佳协议调整;以及端到端性能预测服务。Azure的Stork数据调度器将通过快速促进云计算环境中大量数据的共享，极大地改变领域科学家进行研究的方式。