I/O Virtualization - From Self-Virtualizing Devices to Metadata-Rich Information Appliances

在线访问分布在多台机器上的大型数据集的需求在大型企业和科学环境中都很普遍。一个例子是运行在peta级机器上的大型应用程序的远程数据可视化，研究人员正在寻求通过实时检查和验证其运行代码的输出来加速他们正在执行的科学过程的方法。本研究解决了此类任务所隐含的大规模数据移动的一个基本问题，即应用程序级数据交换与底层硬件和系统平台提供的I/O服务之间存在实质性的语义差距。例如，在设备级，被移动的数据可能被描述为一个线性序列的块，而应用程序级将其视为高度结构化的数据，并在其上执行复杂的检查点操作。为了帮助缩小这一差距，对于未来的虚拟化执行平台，该项目实现了抽象IO （A-IO）设备的概念，其中每个这样的设备都可以被设计为为最终用户应用程序提供有用的新功能。为此，它利用新的系统级和设备级虚拟化技术向操作系统呈现看似物理设备，但实际上是这些设备的虚拟表示。对于这样的“逻辑设备”，它随后解决了在未来的多核平台上放置其高级I/O设备功能的问题：在主机的通用计算核心上还是在与设备相关的专用核心上。在基于A-IO设备（网卡和块设备）的两种实现以及实际的科学和企业应用的背景下，评估了该方法的实用性。

Demands for online access to large data sets distributed across multiple machines are prevalent both in large-scale enterprises and in scientific settings. An example is remote data visualization for large-scale applications running on peta-scale machines, where researchers are seeking ways to accelerate the scientific processes they are carrying out via real-time inspection and validation of the outputs from their running codes.This research addresses a basic problem basic problem for the large-scale data movements implied by such tasks, which is the substantial semantic gap between application-level data exchanges and the I/O services provided by underlying hardware and system platforms. For instance, at the device level, data being moved may be described as a linear sequence of blocks, whereas the application level views it as highly structured data on which it is performing a complex checkpoint operation. To help close help close this gap, for future virtualized execution platforms, this project realizes the notion of Abstract IO (A-IO) devices, where each such device can be designed to provide useful new functionality to end user applications. Toward this end, it exploits new system- and device -level virtualization technologies to present to operating systems what appear to be physical devices, but are actually virtual representations of such devices. For such `logical devices', it then addresses the question regarding where to place their higher level I/O device functions on future many-core platforms: on the general computational cores of host machines or on specialized cores associated with devices. The utility of the approach is evaluated in the context of two realizations of A-IO based devices (NIC and block device), and with actual scientific and enterprise applications.