CC*DNI CI Engineer: Adapting Research Workflows to Use Campus Cyberinfrastructure

点击翻译按钮获取中文摘要

Research workflows across many domains from life sciences to social and behavioral sciences, are scaling to require more computational analysis of much larger data sets. Workflows that have leveraged local computation and data storage resources must be transformed to use larger shared compute facilities like the UC Berkeley Savio supercomputing cluster, and national centers like NERSC and XSEDE. However, researchers have struggled to connect their data sources (such as laboratory instruments, remote data-sets, etc.) to the shared computation and storage resources in an efficient, productive workflow. Barriers range from poor network throughput, to inexperience with data management and shared computing infrastructure and best practices.The Berkeley Research Computing (BRC) program is adding cyber-infrastructure engineering support to engage research groups to understand their needs, and to design and implement effective workflows that take advantage of campus and national computation and data management resources. These research engagements transform existing workflows to advance project-specific research goals, but also provide concrete templates that other research groups can leverage, and an architectural pattern for research engagement going forward.The BRC CI Engineer leverages campus resources including a Science DMZ that connects high-speed state and national network links, including California's Advanced Research and Education Network (CENIC) and the Energy Sciences Network (ESNet), to campus infrastructure, including the BRC Savio supercomputing cluster. The BRC CI Engineer draws upon campus expertise in networking, high-performance computing, and research data management, as well as the domain expertise of campus partners. As research engagements are completed and documented, we will expand faculty understanding of the new scalable workflows, and enable them to address the next generation of research questions.