EarthCube Building Blocks: Collaborative Proposal: Deploying Multi-Facility Cyberinfrastructure in Commercial and Private Cloud-based Systems

人们经常听说，在云中执行公司运营所需的计算是最理想的。诚然，许多商业公司正在将他们的信息技术转移到那个环境中。由NSF资助的科学数据中心有独特的限制，它们必须适应这些限制。资金有限，使用云技术管理数据中心的成本可能相当高。此外，政府资助的研究机构的IT员工通常比公司少得多。在大公司和NSF资助的数据中心之间，在云中管理IT运营的影响并不相同。在GeoSciCloud项目中，两个由NSF资助的中型数据中心计划在不同环境中部署数据集合以及基于云的服务，以评估可行性和影响。这些环境包括：-由亚马逊、谷歌和微软提供的商业云环境，以及-NSF支持的大型计算设施，这些设施刚刚开始提供具有云计算特征的服务这些基础设施在这两个云环境中的运营将与当前的内部环境和评估进行比较。该项目将帮助NSF/EarthCube确定最合适的IT环境，在其中EarthCube应该部署和支持共享基础设施。潜在的可靠性和成本节约是极好的激励因素。IRIS和UNAVCO运营的数据中心拥有数百TB的数据和服务，符合我们社区的需求和要求。目前，每个组织都在运营自己的基础设施。GeoSciCloud的任务将包括将我们的归档的子集作为测试移动到商业云和XSEDE云环境中，在这些环境中，我们将比较和对比在不同基础设施中工作的几个方面。GeoSciCloud合作伙伴还将部署在GeoWS构建块下开发的关键服务，以使领域科学家能够访问数据集。GeoSciCloud将帮助EarthCube比较和对比三个环境(XSEDE、商业云、和当前的基础设施)在以下方面：-了解与将大量数据集吸收到云中并在云环境中管理数据相关的问题。-比较执业领域科学家对现实世界中的数据请求的处理时间-测试云在对地震数据进行大量数字信号处理和长时间重新处理GPS解决方案时的弹性。-比较从多个环境中传出数据的速度，包括使用更高访问系统的测试。-比较在三种环境中操作的总成本-记录从三个环境中产生的最佳实践应在EarthCube中推广的GeoSciCloud测试。-将域格式中保存的数据转换为更广泛使用的格式，如HDF5，以提高互操作性。-测试将实时数据传输到云中的可靠性。GeoSciCloud还将探索提供一些基础设施来支持其他EarthCube合作伙伴，以便多个数据中心可以在GeoSciCloud中共存。IRIS和UNAVCO将致力于最终展示共享基础设施的效用，以及它如何提高EarthCube内部的效率和经济性，特别是云环境中的共享基础设施。

It is common to hear that it is optimal to perform computations necessary for the operation of corporations in the ?cloud? and it is true that many commercial companies are moving their information technology into that environment. Scientific data centers funded by the NSF have unique constraints that they must accommodate. Funding is limited and costs of managing data centers using cloud technology can be quite costly. Additionally, government funded research organizations typically have much smaller IT staffs than do corporations. The impact of managing IT operations in the cloud is not identical between large corporations and NSF funded data centers. In the GeoSciCloud project, two medium-size NSF funded data centers plan to deploy data collections along with cloud-based services in different environments in order to assess the feasibility and impact. These environments include:- Commercial cloud environments such as those offered by Amazon, Google, and Microsoft and- NSF supported large computing facilities that are just beginning to offer services that have characteristics of cloud computingThe operation of these infrastructures in these two cloud environments will be compared to current in-house environments and assessed.This project will thereby help NSF/EarthCube identify the most suitable IT environment in which the EarthCube should deploy and support shared infrastructure. The potential reliability and cost-savings are excellent motivating factors.IRIS and UNAVCO operate data centers with several hundred terabytes of data and services that match our community's needs and requirements. Each organization currently operates its own infrastructure. GeoSciCloud tasks will include moving subsets of our archives, as a test, into commercial cloud and XSEDE cloud environments where we will compare and contrast several aspects of working in different infrastructures. GeoSciCloud partners will also deploy key services developed under the GeoWS building block to enable access to data sets by domain scientists. GeoSciCloud will help EarthCube compare and contrast the three environments (XSEDE, Commercial Cloud, and current infrastructure) in the following areas:- Gain an understanding of issues related to the ingestion of large data sets into the cloud and curating the data in a cloud environment.- Compare processing times for real world requests for data by practicing domain scientists- Test elasticity of the cloud for doing large amounts of digital signal processing of seismic data and reprocessing GPS solutions for long periods of time.- Compare the speed of data egress from multiple environments including tests of using higher access systems such as Grid-FTP.- Compare overall costs of operating in the three environments- Document what the best practices are that emerge from the GeoSciCloud test that should be promoted within EarthCube.- Perform conversion of data held in domain formats to more widely used formats such as HDF5 for improved interoperability.- Test the reliability of streaming real time data into the cloud.GeoSciCloud will also explore providing some infrastructure in support of other EarthCube partners so that multiple data centers can cohabitate within the GeoSciCloud. IRIS and UNAVCO will commit to ultimately demonstrate the utility of shared infrastructure and how it can improve the efficiency and economics within EarthCube and specifically shared infrastructure in a cloud environment.