On the Scalability, Performance Isolation and Device Driver Transparency of the IHK/McKernel Hybrid Lightweight Kernel
On the Scalability, Performance Isolation and Device Driver Transparency of the IHK/McKernel Hybrid Lightweight Kernel
复制标题
论IHK/McKernel混合轻量级内核的可扩展性、性能隔离和设备驱动透明性
DOI：
10.1109/ipdps.2016.80
复制
发表时间：
2016
期刊：
影响因子：
--
通讯作者：
Y. Ishikawa
中科院分区：
文献类型：
--
作者：
Balazs Gerofi;Masamichi Takagi;A. Hori;Gou Nakamura;Tomoki Shirasawa;Y. Ishikawa
高端计算中的极高并行度需要较低的操作系统噪声，以便大规模、批量同步并行应用程序可以高效运行。历史上，无噪音执行是通过部署轻量级内核 (LWK) 来实现的，而另一方面，它只能提供一组有限的 POSIX API 以换取可扩展性。然而，更复杂的应用程序结构（例如现场分析和工作流组合）的日益普及，决定了对 POSIX/Linux 丰富的编程 API 的需求。为了满足这些看似矛盾的要求，混合内核（其中 Linux 和轻量级内核 (LWK) 在计算节点上并行运行）最近被认为是一种有前途的方法。尽管现在有多个研究项目正在研究这个方向，但节点资源如何在两种类型的内核之间共享、这两种内核如何准确地相互作用以及它们集成到什么程度，这些问题仍然是持续争论的主题。在本文中，我们描述了 IHK/McKernel，这是一种混合软件堆栈，通过有选择地将系统服务从轻量级内核卸载到 Linux，将 LWK 与 Linux 无缝混合。具体来说，我们专注于 Linux 设备驱动程序的透明重用，并详细介绍我们框架的设计，使 LWK 能够自然地利用 Linux 驱动程序代码库，而不牺牲可扩展性或 POSIX API。通过对中型集群的严格评估，我们展示了 McKernel 如何为模拟提供一致、隔离的性能，即使面对竞争的现场工作负载也是如此。
Extreme degree of parallelism in high-end computing requires low operating system noise so that large scale, bulk-synchronous parallel applications can be run efficiently. Noiseless execution has been historically achieved by deploying lightweight kernels (LWK), which, on the other hand, can provide only a restricted set of the POSIX API in exchange for scalability. However, the increasing prevalence of more complex application constructs, such as in-situ analysis and workflow composition, dictates the need for the rich programming APIs of POSIX/Linux. In order to comply with these seemingly contradictory requirements, hybrid kernels, where Linux and a lightweight kernel (LWK) are run side-by-side on compute nodes, have been recently recognized as a promising approach. Although multiple research projects are now pursuing this direction, the questions of how node resources are shared between the two types of kernels, how exactly the two kernels interact with each other and to what extent they are integrated, remain subjects of ongoing debate. In this paper, we describe IHK/McKernel, a hybrid software stack that seamlessly blends an LWK with Linux by selectively offloading system services from the lightweight kernel to Linux. Specifically, we are focusing on transparent reuse of Linux device drivers and detail the design of our framework that enables the LWK to naturally leverage the Linux driver codebase without sacrificing scalability or the POSIX API. Through rigorous evaluation on a medium size cluster we demonstrate how McKernel provides consistent, isolated performance for simulations even in face of competing, in-situ workloads.