I/O Virtualization - From Self-Virtualizing Devices to Metadata-Rich Information Appliances

在线访问分布在多台机器上的大型数据集的需求在大型企业和科学环境中都很普遍。一个例子是运行在Peta-Scale机器上的大规模应用程序的远程数据可视化，研究人员正在寻求通过实时检查和验证运行代码的输出来加速科学过程的方法。该研究解决了此类任务所隐含的大规模数据移动的一个基本问题，即应用层数据交换与底层硬件和系统平台提供的I/O服务之间存在巨大的语义鸿沟。例如，在设备级别，被移动的数据可被描述为块的线性序列，而应用程序级别将其视为对其执行复杂检查点操作的高度结构化的数据。为了帮助缩小这一差距，对于未来的虚拟执行平台，该项目实现了抽象IO(A-IO)设备的概念，其中每个此类设备都可以设计为向最终用户应用程序提供有用的新功能。为此，它利用新的系统和设备级虚拟化技术向操作系统呈现看似物理设备，但实际上是此类设备的虚拟表示。对于这样的“逻辑设备”，它然后解决了在未来的多核平台上将其更高级别的I/O设备功能放置在哪里的问题：在主机的通用计算核心上还是在与设备相关联的专用核心上。在基于A-IO的设备(NIC和块设备)的两种实现的背景下，并结合实际的科学和企业应用来评估该方法的效用。

Demands for online access to large data sets distributed across multiple machines are prevalent both in large-scale enterprises and in scientific settings. An example is remote data visualization for large-scale applications running on peta-scale machines, where researchers are seeking ways to accelerate the scientific processes they are carrying out via real-time inspection and validation of the outputs from their running codes.This research addresses a basic problem basic problem for the large-scale data movements implied by such tasks, which is the substantial semantic gap between application-level data exchanges and the I/O services provided by underlying hardware and system platforms. For instance, at the device level, data being moved may be described as a linear sequence of blocks, whereas the application level views it as highly structured data on which it is performing a complex checkpoint operation. To help close help close this gap, for future virtualized execution platforms, this project realizes the notion of Abstract IO (A-IO) devices, where each such device can be designed to provide useful new functionality to end user applications. Toward this end, it exploits new system- and device -level virtualization technologies to present to operating systems what appear to be physical devices, but are actually virtual representations of such devices. For such `logical devices', it then addresses the question regarding where to place their higher level I/O device functions on future many-core platforms: on the general computational cores of host machines or on specialized cores associated with devices. The utility of the approach is evaluated in the context of two realizations of A-IO based devices (NIC and block device), and with actual scientific and enterprise applications.