EAGER: Stork Data Scheduler for Azure

该项目将进一步开发和增强Stork Data Scheduler，以支持Azure云计算环境，缓解数据密集型云计算应用中的端到端数据处理瓶颈。STOCK数据调度程序在许多数据密集型应用领域得到了非常活跃的应用，包括海岸灾害预测和风暴潮模拟；油流和水库不确定性分析；数值相对论和黑洞碰撞；教育视频处理和行为评估；数字天空成像；以及多尺度计算流体力学。使Stork在Azure环境中可用，将使现有的用户群能够轻松迁移到Azure云计算平台以及受益于Stork大规模数据处理能力的其他Azure应用程序组。Stork Data Scheduler for Azure将为云计算社区做出独特的贡献，因为它专注于数据放置任务的规划、调度、监视和管理，以及针对PB级数据密集型应用程序的网络I/O的应用程序级端到端优化。与现有方法不同，它将数据资源和与数据访问和移动相关的任务视为一级实体，就像计算资源和计算任务一样，而不仅仅是计算的副作用。Stork Data Scheduler for Azure将为云计算提供增强的功能，如数据聚合和连接缓存；对等和流数据管理；数据传输中的早期错误检测、分类和恢复；计划存储管理；优化协议调整；以及端到端性能预测服务。适用于Azure的Stork数据调度器将通过快速促进云计算环境中的大量数据共享，极大地改变领域科学家进行研究的方式。

This project will further develop and enhance the Stork Data Scheduler to support Azure cloud computing environment, and to mitigate the end-to-end data handling bottleneck in data-intensive cloud computing applications. Stork data scheduler has been very actively used in many data-intensive application areas including coastal hazard prediction and storm surge modeling; oil flow and reservoir uncertainty analysis; numerical relativity and black hole collisions; educational video processing and behavioral assessment; digital sky imaging; and multiscale computational fluid dynamics. Making Stork available on the Azure environment will enable this already existing user base to easily migrate to the Azure Cloud Computing platform as well as other Azure application groups benefiting from the large-scale data handling capabilities of Stork.The Stork Data Scheduler for Azure will make a distinctive contribution to cloud computing community because it focuses on planning, scheduling, monitoring and management of data placement tasks and application-level end-to-end optimization of networked I/O for petascale data-intensive applications. Unlike existing approaches, it will treat data resources and the tasks related to data access and movement as first class entities just like computational resources and compute tasks, and not simply the side effect of computation. Stork data scheduler for Azure will provide enhanced functionality for cloud computing such as data aggregation and connection caching; peer-to-peer and streamed data management; early error detection, classification, and recovery in data transfers; scheduled storage management; optimal protocol tuning; and end-to-end performance prediction services. The Stork data scheduler for Azure will dramatically change how domain scientists perform their research by rapidly facilitating sharing of large amounts of data in cloud computing environments.