EAGER: Stork Data Scheduler for Azure

该项目将进一步开发和增强Stork数据调度器，以支持Azure云计算环境，并缓解数据密集型云计算应用程序中的端到端数据处理瓶颈。Stork数据调度程序在沿海灾害预测、风暴潮建模等数据密集型应用领域得到了广泛的应用；油流及储层不确定性分析；数值相对论与黑洞碰撞；教育视频处理与行为评估；数字天空成像；以及多尺度计算流体动力学。让Stork在Azure环境中可用，将使这些已经存在的用户群能够轻松迁移到Azure云计算平台，以及从Stork的大规模数据处理能力中受益的其他Azure应用程序组。Azure的Stork Data Scheduler将对云计算社区做出独特的贡献，因为它专注于数据放置任务的规划、调度、监控和管理，以及千万亿级数据密集型应用程序的网络I/O的应用级端到端优化。与现有的方法不同，它将数据资源和与数据访问和移动相关的任务视为第一类实体，就像计算资源和计算任务一样，而不仅仅是计算的副作用。Azure的Stork数据调度器将为云计算提供增强的功能，如数据聚合和连接缓存；点对点和流数据管理；数据传输中的早期错误检测、分类和恢复；计划存储管理；协议优化；以及端到端性能预测服务。Azure的Stork数据调度器将通过快速促进云计算环境中大量数据的共享，极大地改变领域科学家执行研究的方式。

This project will further develop and enhance the Stork Data Scheduler to support Azure cloud computing environment, and to mitigate the end-to-end data handling bottleneck in data-intensive cloud computing applications. Stork data scheduler has been very actively used in many data-intensive application areas including coastal hazard prediction and storm surge modeling; oil flow and reservoir uncertainty analysis; numerical relativity and black hole collisions; educational video processing and behavioral assessment; digital sky imaging; and multiscale computational fluid dynamics. Making Stork available on the Azure environment will enable this already existing user base to easily migrate to the Azure Cloud Computing platform as well as other Azure application groups benefiting from the large-scale data handling capabilities of Stork.The Stork Data Scheduler for Azure will make a distinctive contribution to cloud computing community because it focuses on planning, scheduling, monitoring and management of data placement tasks and application-level end-to-end optimization of networked I/O for petascale data-intensive applications. Unlike existing approaches, it will treat data resources and the tasks related to data access and movement as first class entities just like computational resources and compute tasks, and not simply the side effect of computation. Stork data scheduler for Azure will provide enhanced functionality for cloud computing such as data aggregation and connection caching; peer-to-peer and streamed data management; early error detection, classification, and recovery in data transfers; scheduled storage management; optimal protocol tuning; and end-to-end performance prediction services. The Stork data scheduler for Azure will dramatically change how domain scientists perform their research by rapidly facilitating sharing of large amounts of data in cloud computing environments.