PIRE: Training and Workshops in Data Intensive Computing Using The Open Science Data Cloud

今天，许多科学家面临着管理和分析快速增长的复杂数据集的前所未有的挑战。这一国际PIRE项目旨在缩小现代科学仪器产生数据的能力与研究人员以可靠和及时的方式管理、分析和共享这些数据的能力之间日益扩大的差距。云计算的新兴技术是从目前的网络基础设施向前迈出的一步。云计算涉及分布式计算机的集群(“云”)，这些计算机通过网络(通常是Internet)提供可能更便宜、更灵活和更强大的按需资源和服务，同时提供数据中心的规模和可靠性。这个PERE团队打算帮助开发大规模的分布式计算能力-开放科学数据云(OSDC)-为科学数据提供长期持久存储，并为整合、分析、共享和归档科学数据提供最先进的服务。该小组建议研究和加强存储系统，这些存储系统集成了专门的网络协议，并支持通过广域、高性能网络进行数据传输。随着数据大小的增长，分析数据的唯一实用方法是使用并行编程，但直到最近，领域科学家利用并行编程一直很耗时。另一个研究重点将是开发新的基于云的并行编程框架，并将它们整合到云基础设施中，以便这项技术更广泛地为科学家所用。除了该项目的研究层面外，另一个关键方面是许多领域的科学家及其学生参与讲习班和随后使用云网络基础设施。这些小组将接受云计算基础知识的培训，然后将努力确保云计算研究进展最大限度地提高其学科特有的复杂数据集的可管理性和分析能力。这个帝国计划将云计算作为一个全球问题来对待，因此利用了它在欧洲、亚洲和南美的许多合作者的云计算、高性能网络、领域科学、e-Science、教育和外联专业知识。外国合作伙伴还提供了一种自然机制，以参与国际科学数据集和分布式网络，并适应不同的国际标准，以保证互操作性。国际合作者可以为美国研究生和早期职业科学家提供参与国际合作的机会，也可以担任这一新网络基础设施的全球大使，帮助获得对其未来采用至关重要的广泛支持。该项目将建立一支强大的学生队伍，他们在许多对美国和国际科学合作至关重要的研究领域具有全球视野的科学数据管理。该项目将通过在外国机构的研究和培训以及参加年度研讨会，为美国研究生和早期职业科学家提供与顶尖科学家进行国际研究和教育的经验。作为一个小组，这些帝国的学生将分享对数据密集型计算的兴趣，但将来自计算机科学、物理、天文学、地球科学、化学、工程和生物等不同领域，为他们的培训注入跨学科的活力。PERE团队成员还将开发1-2天和1-2周的数据密集型计算课程，由美国和国际计算机科学和领域科学的教师开发实践练习。这个PERY项目可能会产生超过个人合作者水平的许多影响。对于美国帝国学院来说，它将加强目前全球云计算社区的联系和合作，并让更多的美国学生加入国际跨学科研究团队，为大型科学数据集的服务、支持和分析提供服务。该项目将加强伊利诺伊大学芝加哥分校和佛罗里达国际大学的国际化努力，为不同的学生群体提供短期海外研究和其他学术经验的机会。该项目将通过分散的研究合作、跨洲参与的课程、全球网络讨论和专注的社交网络论坛，增加美国机构的虚拟国际参与。增加拥有管理和分析超大型数据集专业知识的科学家数量，对我们国家的未来也至关重要。最后，由于这种变革性的技术广泛适用于任何努力管理和分析产生的数据量的科学项目，OSDC及其促进作用很可能在帝国项目结束后很长一段时间内持续存在。参与研究的美国机构包括伊利诺伊大学芝加哥分校的国家数据挖掘中心(NCDM)和佛罗里达国际大学。OSDC美国合作机构包括芝加哥大学和约翰霍普金斯大学。与外国机构合作的机构包括爱丁堡大学(英国)、联邦弗卢米嫩塞大学(巴西)、阿姆斯特丹大学(荷兰)、国家先进工业科学技术研究所(AIST)(日本)、韩国科学技术研究所(KISI)超级计算机中心、北京基因组研究所(BIG)-中国科学院和圣保罗州立大学(巴西)。该项目由美国国家科学基金会国际科学与工程办公室、网络基础设施办公室、计算机和通信基础部、天文科学部和物理部共同资助。

Many scientists today face the unprecedented challenge of managing and analyzing a rapidly growing set of complex data. This international PIRE project aims to narrow the growing gap between the capability of modern scientific instruments to produce data and the ability of researchers to manage, analyze, and share those data in a reliable and timely manner. The emerging technology of cloud computing is a step forward from the current cyberinfrastructure. Cloud computing involves clusters (the "clouds") of distributed computers that provide potentially less expensive, more flexible, and more powerful on-demand resources and services over a network, usually the Internet, while providing the scale and the reliability of a data center. This PIRE team intends to help develop large-scale distributed computing capabilities - the Open Science Data Cloud (OSDC) - to provide long term persistent storage for scientific data and state-of-the-art services for integrating, analyzing, sharing and archiving scientific data. The group proposes to study and strengthen storage systems that integrate specialized network protocols and support data transport over wide-area, high-performance networks. As data grows in size, the only practical means to analyze it is to use parallel programming, but until recently it has been time consuming for a domain scientist to take advantage of parallel programming. Another research focus will be to develop new classes of cloud-based parallel programming frameworks and to integrate them into the cloud infrastructure so that this technology is more broadly available to scientists. In addition to the research dimensions of this project, another key aspect is the involvement, in workshops and in subsequent use of the cloud cyberinfrastructure, of many domain scientists and their students. These groups will be trained in the basics of cloud computing and then will work to ensure that the cloud computing research advances maximize the manageability and analytical power of the complex datasets unique to their disciplines. This PIRE project embraces cloud computing as a global issue and so taps the cloud computing, high performance networking, domain science, e-Science, education and outreach expertise of its many collaborators in Europe, Asia and South America. Foreign partners also provide a natural mechanism to engage international scientific datasets and distributed networks, and accommodate different international standards to guarantee interoperability. The international collaborators can provide an entry into international collaborations for U.S. graduate students and early career scientists and can also serve as global ambassadors for this new cyberinfrastructure, helping to garner widespread support that will be critical to its future adoption.The project will build a strong cadre of students with a global perspective on scientific data management in many research areas vital to U.S. and international scientific collaborations. The project will provide U.S. graduate students and early career scientists with international research and education experiences with leading scientists via research and training at foreign institutions and participation in annual workshops. As a group, the PIRE students will share an interest in data intensive computing but will be drawn from fields as diverse as computer science, physics, astronomy, geosciences, chemistry, engineering, and biology, lending an interdisciplinary vigor to their training. The PIRE team members will also develop 1-2 day and 1-2 week courses on data intensive computing, with hands-on exercises developed by U.S. and international faculty in computer science and the domain sciences.This PIRE project is likely to have numerous impacts above the level of the individual collaborators. For the U.S. PIRE institutions, it will strengthen current linkages and collaborations in the global Cloud Computing community and engage more U.S. students in international interdisciplinary research teams for the service, support and analysis of large scientific datasets. The project will enhance internationalizing efforts both at the University of Illinois at Chicago and at Florida International University by providing opportunities for short term research abroad and other academic experiences to a diverse group of students. This project will increase the virtual international engagement of the U.S. institutions via distributed research collaborations, courses with transcontinental participation, global web discussions, and focused social networking forums. Increasing the number of scientists with expertise in managing and analyzing very large datasets is also vital to the future of our nation. Finally, since this transformative technology is broadly applicable to any scientific project struggling to manage and analyze the volume of data produced, the OSDC and its facilitative impacts are likely to persist long after the PIRE project has ended. Participating U.S. institutions include the National Center for Data Mining (NCDM) at the University of Illinois at Chicago and Florida International University. OSDC U.S. partner institutions include the University of Chicago and Johns Hopkins University. Partnering foreign institutions include the University of Edinburgh (UK); Universidade Federal Fluminense (Brazil); University of Amsterdam (The Netherlands); National Institute of Advanced Industrial Science and Technology (AIST) (Japan); Korea Institute of Science and Technology Information (KISTI) Supercomputing Center; Beijing Institute of Genomics (BIG) - Chinese Academy of Sciences, and the State University of Sao Paulo (Brazil). This project is cofunded by the NSF's Office of International Science and Engineering, the Office of Cyberinfrastructure, the Division of Computer and Communication Foundations, the Division of Astronomical Sciences, and the Division of Physics.