PIRE: Training and Workshops in Data Intensive Computing Using The Open Science Data Cloud

今天，许多科学家面临着管理和分析快速增长的复杂数据的前所未有的挑战。该国际项目旨在缩小现代科学仪器产生数据的能力与研究人员以可靠和及时的方式管理、分析和共享这些数据的能力之间日益扩大的差距。新兴的云计算技术是当前网络基础设施的一个进步。云计算涉及分布式计算机的集群（“云”），这些计算机通过网络（通常是Internet）提供可能更便宜、更灵活、更强大的按需资源和服务，同时提供数据中心的规模和可靠性。这个PIRE团队打算帮助开发大规模分布式计算能力——开放科学数据云（OSDC）——为科学数据提供长期持久的存储和最先进的服务，用于集成、分析、共享和归档科学数据。该小组建议研究和加强集成专门网络协议的存储系统，并支持广域高性能网络上的数据传输。随着数据规模的增长，分析数据的唯一实用方法是使用并行编程，但直到最近，领域科学家利用并行编程一直是非常耗时的。另一个研究重点将是开发新的基于云的并行编程框架，并将它们集成到云基础设施中，以便科学家更广泛地使用这项技术。除了这个项目的研究维度，另一个关键方面是参与，在研讨会和随后使用云网络基础设施，许多领域的科学家和他们的学生。这些小组将接受云计算基础知识的培训，然后将努力确保云计算研究的进展最大限度地提高其学科特有的复杂数据集的可管理性和分析能力。这个PIRE项目将云计算作为一个全球性问题，因此利用了欧洲、亚洲和南美许多合作者的云计算、高性能网络、领域科学、电子科学、教育和外展专业知识。外国合作伙伴还提供了一种自然的机制来参与国际科学数据集和分布式网络，并适应不同的国际标准以保证互操作性。国际合作者可以为美国研究生和早期职业科学家提供国际合作的入口，也可以作为这种新的网络基础设施的全球大使，帮助获得对其未来采用至关重要的广泛支持。该项目将在许多对美国和国际科学合作至关重要的研究领域培养一支具有全球科学数据管理视角的强大学生队伍。该项目将通过在外国机构进行研究和培训以及参加年度研讨会，为美国研究生和早期职业科学家提供与顶尖科学家一起进行国际研究和教育的经验。作为一个整体，PIRE的学生将对数据密集型计算有共同的兴趣，但他们将来自计算机科学、物理学、天文学、地球科学、化学、工程和生物学等不同领域，为他们的培训提供跨学科的活力。PIRE团队成员还将开发1-2天1-2周的数据密集型计算课程，并由美国和国际计算机科学和领域科学教师开发动手练习。这个pie项目可能会对个人合作者的水平产生许多影响。对于美国的研究机构来说，它将加强目前在全球云计算社区的联系和合作，并吸引更多的美国学生加入国际跨学科研究团队，为大型科学数据集提供服务、支持和分析。该项目将加强伊利诺伊大学芝加哥分校和佛罗里达国际大学的国际化努力，为不同群体的学生提供短期海外研究和其他学术经历的机会。该项目将通过分布式研究合作、跨大陆参与的课程、全球网络讨论和集中的社交网络论坛，增加美国机构的虚拟国际参与。增加在管理和分析大型数据集方面具有专业知识的科学家的数量，对我们国家的未来也至关重要。最后，由于这种变革性技术广泛适用于任何努力管理和分析所产生的数据量的科学项目，OSDC及其促进影响可能在PIRE项目结束后很长时间内持续存在。参与的美国机构包括芝加哥伊利诺伊大学的国家数据挖掘中心（NCDM）和佛罗里达国际大学。OSDC的美国合作机构包括芝加哥大学和约翰霍普金斯大学。合作的外国机构包括爱丁堡大学（英国）；联邦弗鲁米嫩塞大学（巴西）；阿姆斯特丹大学（荷兰）；日本国家先进工业科学技术研究所；韩国科学技术信息研究院（KISTI）超级计算中心；中国科学院北京基因组研究所（BIG）与巴西圣保罗州立大学合作。该项目由美国国家科学基金会国际科学与工程办公室、网络基础设施办公室、计算机与通信基础部门、天文科学部门和物理部门共同资助。

Many scientists today face the unprecedented challenge of managing and analyzing a rapidly growing set of complex data. This international PIRE project aims to narrow the growing gap between the capability of modern scientific instruments to produce data and the ability of researchers to manage, analyze, and share those data in a reliable and timely manner. The emerging technology of cloud computing is a step forward from the current cyberinfrastructure. Cloud computing involves clusters (the "clouds") of distributed computers that provide potentially less expensive, more flexible, and more powerful on-demand resources and services over a network, usually the Internet, while providing the scale and the reliability of a data center. This PIRE team intends to help develop large-scale distributed computing capabilities - the Open Science Data Cloud (OSDC) - to provide long term persistent storage for scientific data and state-of-the-art services for integrating, analyzing, sharing and archiving scientific data. The group proposes to study and strengthen storage systems that integrate specialized network protocols and support data transport over wide-area, high-performance networks. As data grows in size, the only practical means to analyze it is to use parallel programming, but until recently it has been time consuming for a domain scientist to take advantage of parallel programming. Another research focus will be to develop new classes of cloud-based parallel programming frameworks and to integrate them into the cloud infrastructure so that this technology is more broadly available to scientists. In addition to the research dimensions of this project, another key aspect is the involvement, in workshops and in subsequent use of the cloud cyberinfrastructure, of many domain scientists and their students. These groups will be trained in the basics of cloud computing and then will work to ensure that the cloud computing research advances maximize the manageability and analytical power of the complex datasets unique to their disciplines. This PIRE project embraces cloud computing as a global issue and so taps the cloud computing, high performance networking, domain science, e-Science, education and outreach expertise of its many collaborators in Europe, Asia and South America. Foreign partners also provide a natural mechanism to engage international scientific datasets and distributed networks, and accommodate different international standards to guarantee interoperability. The international collaborators can provide an entry into international collaborations for U.S. graduate students and early career scientists and can also serve as global ambassadors for this new cyberinfrastructure, helping to garner widespread support that will be critical to its future adoption.The project will build a strong cadre of students with a global perspective on scientific data management in many research areas vital to U.S. and international scientific collaborations. The project will provide U.S. graduate students and early career scientists with international research and education experiences with leading scientists via research and training at foreign institutions and participation in annual workshops. As a group, the PIRE students will share an interest in data intensive computing but will be drawn from fields as diverse as computer science, physics, astronomy, geosciences, chemistry, engineering, and biology, lending an interdisciplinary vigor to their training. The PIRE team members will also develop 1-2 day and 1-2 week courses on data intensive computing, with hands-on exercises developed by U.S. and international faculty in computer science and the domain sciences.This PIRE project is likely to have numerous impacts above the level of the individual collaborators. For the U.S. PIRE institutions, it will strengthen current linkages and collaborations in the global Cloud Computing community and engage more U.S. students in international interdisciplinary research teams for the service, support and analysis of large scientific datasets. The project will enhance internationalizing efforts both at the University of Illinois at Chicago and at Florida International University by providing opportunities for short term research abroad and other academic experiences to a diverse group of students. This project will increase the virtual international engagement of the U.S. institutions via distributed research collaborations, courses with transcontinental participation, global web discussions, and focused social networking forums. Increasing the number of scientists with expertise in managing and analyzing very large datasets is also vital to the future of our nation. Finally, since this transformative technology is broadly applicable to any scientific project struggling to manage and analyze the volume of data produced, the OSDC and its facilitative impacts are likely to persist long after the PIRE project has ended. Participating U.S. institutions include the National Center for Data Mining (NCDM) at the University of Illinois at Chicago and Florida International University. OSDC U.S. partner institutions include the University of Chicago and Johns Hopkins University. Partnering foreign institutions include the University of Edinburgh (UK); Universidade Federal Fluminense (Brazil); University of Amsterdam (The Netherlands); National Institute of Advanced Industrial Science and Technology (AIST) (Japan); Korea Institute of Science and Technology Information (KISTI) Supercomputing Center; Beijing Institute of Genomics (BIG) - Chinese Academy of Sciences, and the State University of Sao Paulo (Brazil). This project is cofunded by the NSF's Office of International Science and Engineering, the Office of Cyberinfrastructure, the Division of Computer and Communication Foundations, the Division of Astronomical Sciences, and the Division of Physics.