CC*DNI CI Engineer: Adapting Research Workflows to Use Campus Cyberinfrastructure

从生命科学到社会和行为科学等许多领域的研究工作流程正在扩展，以要求对更大的数据集进行更多的计算分析。利用本地计算和数据存储资源的工作流必须转变为使用更大的共享计算设施，如UC Berkeley Savio超级计算集群，以及NERSC和XSEDE等国家中心。然而，研究人员一直在努力连接他们的数据源(如实验室仪器、远程数据集等)。在高效、多产的工作流程中共享计算和存储资源。障碍包括网络吞吐量低，缺乏数据管理经验和共享计算基础设施和最佳实践。伯克利研究计算(BRC)计划正在增加网络基础设施工程支持，以吸引研究小组了解他们的需求，并设计和实施有效的工作流程，利用校园和国家计算和数据管理资源。这些研究项目转变了现有的工作流程，以推进特定于项目的研究目标，但也提供了其他研究小组可以利用的具体模板，以及未来研究项目的架构模式。BRC CI工程师利用校园资源，包括一个科学非军事区，该非军事区将高速的州和国家网络链接(包括加州的高级研究和教育网络(CEEN)和能源科学网络(ESnet))连接到校园基础设施，包括BRC Savio超级计算集群。BRC CI工程师利用园区在网络、高性能计算和研究数据管理方面的专业知识，以及园区合作伙伴的领域专业知识。随着研究项目的完成和记录，我们将扩大教职员工对新的可扩展工作流的理解，并使他们能够解决下一代研究问题。

Research workflows across many domains from life sciences to social and behavioral sciences, are scaling to require more computational analysis of much larger data sets. Workflows that have leveraged local computation and data storage resources must be transformed to use larger shared compute facilities like the UC Berkeley Savio supercomputing cluster, and national centers like NERSC and XSEDE. However, researchers have struggled to connect their data sources (such as laboratory instruments, remote data-sets, etc.) to the shared computation and storage resources in an efficient, productive workflow. Barriers range from poor network throughput, to inexperience with data management and shared computing infrastructure and best practices.The Berkeley Research Computing (BRC) program is adding cyber-infrastructure engineering support to engage research groups to understand their needs, and to design and implement effective workflows that take advantage of campus and national computation and data management resources. These research engagements transform existing workflows to advance project-specific research goals, but also provide concrete templates that other research groups can leverage, and an architectural pattern for research engagement going forward.The BRC CI Engineer leverages campus resources including a Science DMZ that connects high-speed state and national network links, including California's Advanced Research and Education Network (CENIC) and the Energy Sciences Network (ESNet), to campus infrastructure, including the BRC Savio supercomputing cluster. The BRC CI Engineer draws upon campus expertise in networking, high-performance computing, and research data management, as well as the domain expertise of campus partners. As research engagements are completed and documented, we will expand faculty understanding of the new scalable workflows, and enable them to address the next generation of research questions.