EarthCube Building Blocks: Collaborative Proposal: Deploying Multi-Facility Cyberinfrastructure in Commercial and Private Cloud-based Systems

EarthCube 构建模块：协作提案：在商业和基于私有云的系统中部署多设施网络基础设施

• 批准号: 1639719
• 负责人: Jerry Carter
• 金额: $ 81.48万
• 依托单位: Incorporated Research Institutions for Seismology
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2016
• 资助国家: 美国
• 起止时间: 2016-09-01 至 2020-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1639719&HistoricalAwards=false
• 关键词: EarthCube; Building; Blocks; Collaborative; Proposal

It is common to hear that it is optimal to perform computations necessary for the operation of corporations in the ?cloud? and it is true that many commercial companies are moving their information technology into that environment. Scientific data centers funded by the NSF have unique constraints that they must accommodate. Funding is limited and costs of managing data centers using cloud technology can be quite costly. Additionally, government funded research organizations typically have much smaller IT staffs than do corporations. The impact of managing IT operations in the cloud is not identical between large corporations and NSF funded data centers. In the GeoSciCloud project, two medium-size NSF funded data centers plan to deploy data collections along with cloud-based services in different environments in order to assess the feasibility and impact. These environments include:- Commercial cloud environments such as those offered by Amazon, Google, and Microsoft and- NSF supported large computing facilities that are just beginning to offer services that have characteristics of cloud computingThe operation of these infrastructures in these two cloud environments will be compared to current in-house environments and assessed.This project will thereby help NSF/EarthCube identify the most suitable IT environment in which the EarthCube should deploy and support shared infrastructure. The potential reliability and cost-savings are excellent motivating factors.IRIS and UNAVCO operate data centers with several hundred terabytes of data and services that match our community's needs and requirements. Each organization currently operates its own infrastructure. GeoSciCloud tasks will include moving subsets of our archives, as a test, into commercial cloud and XSEDE cloud environments where we will compare and contrast several aspects of working in different infrastructures. GeoSciCloud partners will also deploy key services developed under the GeoWS building block to enable access to data sets by domain scientists. GeoSciCloud will help EarthCube compare and contrast the three environments (XSEDE, Commercial Cloud, and current infrastructure) in the following areas:- Gain an understanding of issues related to the ingestion of large data sets into the cloud and curating the data in a cloud environment.- Compare processing times for real world requests for data by practicing domain scientists- Test elasticity of the cloud for doing large amounts of digital signal processing of seismic data and reprocessing GPS solutions for long periods of time.- Compare the speed of data egress from multiple environments including tests of using higher access systems such as Grid-FTP.- Compare overall costs of operating in the three environments- Document what the best practices are that emerge from the GeoSciCloud test that should be promoted within EarthCube.- Perform conversion of data held in domain formats to more widely used formats such as HDF5 for improved interoperability.- Test the reliability of streaming real time data into the cloud.GeoSciCloud will also explore providing some infrastructure in support of other EarthCube partners so that multiple data centers can cohabitate within the GeoSciCloud. IRIS and UNAVCO will commit to ultimately demonstrate the utility of shared infrastructure and how it can improve the efficiency and economics within EarthCube and specifically shared infrastructure in a cloud environment.

这是常见的听到，这是最佳的执行计算所需的公司在？云？许多商业公司正在将其信息技术转移到这种环境中，这是事实。由NSF资助的科学数据中心有他们必须适应的独特限制。 资金有限，使用云技术管理数据中心的成本可能相当昂贵。 此外，政府资助的研究机构的IT人员通常比公司少得多。在云中管理IT运营的影响在大公司和NSF资助的数据中心之间并不相同。 在GeoSciCloud项目中，两个由NSF资助的中型数据中心计划在不同的环境中部署数据收集沿着基于云的服务，以评估可行性和影响。 这些环境包括：-商业云环境，如亚马逊、谷歌和微软提供的云环境;- NSF支持的大型计算设施，这些设施刚刚开始提供具有云计算特征的服务。将对这两种云环境中这些基础设施的运行情况与当前的内部环境进行比较和评估。该项目将帮助NSF/EarthCube确定EarthCube应该部署和支持共享基础设施的最合适的IT环境。 IRIS和UNAVCO运营的数据中心拥有数百TB的数据和服务，符合我们社区的需求和要求。 每个组织目前都有自己的基础设施。GeoSciCloud的任务将包括将我们的档案子集作为测试移动到商业云和XSEDE云环境中，在那里我们将比较和对比在不同基础设施中工作的几个方面。GeoSciCloud合作伙伴还将部署在GeoWS构建块下开发的关键服务，以使领域科学家能够访问数据集。GeoSciCloud将帮助EarthCube在以下领域比较和对比三种环境（XSEDE，商业云和当前基础设施）：-了解将大型数据集摄取到云中并在云环境中管理数据的相关问题。通过实践领域科学家比较真实的世界数据请求的处理时间-测试云的弹性，以便对地震数据进行大量数字信号处理，并长时间重新处理GPS解决方案。比较来自多个环境的数据出口速度，包括使用更高访问系统（如Grid-FTP）的测试。比较在三种环境中运行的总体成本-记录GeoSciCloud测试中出现的最佳实践，这些最佳实践应该在EarthCube中推广。将以域格式保存的数据转换为更广泛使用的格式，如HDF 5，以提高互操作性。测试将真实的时间数据流传输到云中的可靠性。GeoSciCloud还将探索提供一些基础设施来支持其他EarthCube合作伙伴，以便多个数据中心可以在GeoSciCloud中共存。 IRIS和UNAVCO将致力于最终展示共享基础设施的实用性，以及如何提高EarthCube的效率和经济性，特别是云环境中的共享基础设施。