Frameworks: Re-Engineering Galaxy for Performance, Scalability and Energy Efficiency

生物医学研究是研究生物过程和识别、预防和治疗疾病的一个重要科学分支。这项研究形成了发现新药和新疗法的途径。因此，生物医学研究对促进国家健康和繁荣至关重要。鉴于地理上分布的研究小组和生物医学实验室，协作科学在生物医学研究中起着非常重要的作用。Galaxy是一个开源的基于网络的框架，全世界有20 000多名研究人员广泛使用它在许多应用领域进行研究，其中最突出的是生物医学研究。它提供了一个基于网络的环境，科学家可以使用这个环境对他们的数据进行各种计算分析，交换这些分析的结果，探索新的研究概念，促进学生培训，并保存他们的结果以供将来使用。Galaxy目前运行在各种高性能计算（HPC）平台上，包括本地集群、国家实验室的超级计算机、公共数据中心和云。不幸的是，虽然这些系统中的大多数都为传统的cpu提供了重要的加速器功能（以图形处理单元（gpu）和/或现场可编程门阵列（fpga）的形式），但目前的Galaxy实现并没有利用这些强大的加速器。该项目增强了Galaxy框架，使其能够充分利用gpu和fpga提供的巨大计算能力。通过这样做，运行在Galaxy下的重要应用程序经历了显著的加速，从而加速了科学发现。该项目由四个互补任务组成，它们遵循以下逻辑进程：任务1侧重于重新设计现有的支持GPU/FPGA的Galaxy工具，并将其集成到Galaxy工具链中；Task-II为在云平台上运行Galaxy的工具和加速器感知编排提供容器化支持；Task-III为Task-I和Task-II实现特定的策略驱动调度方案；最后，Task-IV重新设计了Galaxy存储，以加快执行速度并减少与数据传输相关的瓶颈。银河计划的增强功能通过为跨多个学科的更大的研究人员社区提供最先进的实验平台，使创新与发现相结合。在更广泛的影响和推广/教育方面，该项目影响了Galaxy工具和应用程序的性能和能源效率，并极大地提高了典型Galaxy用户的生产力；也就是说，这个项目的主要受益者是现有星系社区的数千名成员。然而，该项目还(i)帮助现有的基于GPU和FPGA（非Galaxy）的应用程序开始使用Galaxy，从而充分利用框架内的所有现有工具集；（ii）使Galaxy工具能够更好地利用新兴的集群调度能力；（iii）与同时进行的Galaxy相关工作和现有基础设施工作产生协同作用，pi参与，进一步加快科学发现。因此，这个提议的系统支持将通过增强的星系系统支持产生广泛的社会影响。在教育方面，该项目涉及计算机科学和生物信息学方面代表性不足的群体，向本科生推广，各种K-12相关活动（science - u， CSATS， VIEW），并通过向银河系社区开放的研讨会与其他学科（例如，自然语言处理，图像处理，药物发现和宇宙学）的研究人员进行接触。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Biomedical research is an important branch of science that deals with the problem of studying biological processes and identifying, preventing and curing diseases. This research forms the pathway to the discovery of new medicines as well as new therapies. As such, biomedical research is crucial to advance the national health and prosperity. Given the geographically distributed research groups and biomedical labs, collaborative science plays a very important role in biomedical research. Galaxy is an open source, web-based framework that is extensively used by more than 20,000 researchers world-wide for conducting research in many application domains, the most prominent of which is biomedical research. It provides a web-based environment using which scientists perform various computational analyses on their data, exchange results from these analyses, explore new research concepts, facilitate student training, and preserve their results for future use. Galaxy currently runs on a large variety of high-performance computing (HPC) platforms including local clusters, supercomputers in national labs, public datacenters and Cloud. Unfortunately, while most of these systems supplement conventional CPUs with significant accelerator capabilities (in the form of Graphical Processing Units (GPUs) and/or Field-Programmable Gate Arrays (FPGAs)), the current Galaxy implementation does not take advantage of these powerful accelerators. This project enhances the Galaxy framework so that it can take full advantage of the tremendous computational capabilities offered by GPUs and FPGAs. By doing so, the important applications running under Galaxy experiences significant speedups, thereby accelerating scientific discoveries. This project consists of four complementary tasks, which follow a logistic progression as follows: Task-I focuses on redesigning existing Galaxy tools with GPU/FPGA support and integrate them to Galaxy tool-chains; Task-II provides containerization support for the tools and accelerator-aware orchestration for running Galaxy on cloud platforms; Task-III implements specific policy driven scheduling schemes for Task-I and Task-II; and finally, Task-IV redesigns Galaxy storage to speed up execution and reduce bottlenecks related to data transfer. The proposed enhancements to Galaxy enables the integration of innovation with discovery by providing a state-of-the art experimental platform to a larger community of researchers across several disciplines. On the broader impact and outreach/educational front, this project impacts the performance and energy efficiency of Galaxy tools and applications and improves the productivity of a typical Galaxy user tremendously; that is, the main beneficiaries of this project are thousands of members of existing Galaxy Community. However, this project also (i) helps existing GPU and FPGA based (non-Galaxy) applications start using Galaxy, thereby taking full advantage of all existing toolsets within the framework, (ii) enables Galaxy tools to take better advantage of emerging cluster scheduling capabilities, and (iii) creates a synergy with concurrent Galaxy related efforts and existing infrastructure efforts the PIs are involved with, to further expedite scientific discoveries. As such, this proposed system support will have a broad societal impact via the enhanced Galaxy system support. On the education side, the project involves under-represented groups in computer science as well as in bio-informatics, outreach to undergraduates, various K-12 related activities (Science-U, CSATS, VIEW), and engagement with researchers in other disciplines (e.g., natural language processing, image processing, drug discovery and cosmology) via a workshop open to the Galaxy community.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.