Intelligent Performance Optimisation of Virtualised Data Storage Systems (iPODS)

最近数据和存储容量的空前快速增长带来了两个关键的研发问题：如何最好地将数据映射到物理磁盘设备，以及选择这种映射的基础是什么？从用户的角度来看，确保以合理的成本提供足够的服务质量(由应用程序要求决定)是很重要的。此外，由于磁盘存储的总拥有成本主要由管理和管理活动决定，因此易于管理和自主操作至关重要。上述技术拉动自然而然地导致了包含智能存储结构的虚拟存储基础设施的开发和广泛采用。此类系统的基础物理资源被组织到存储层中，每个存储层针对特定的服务质量提供不同的成本/容量比。除了提供跨所有层的存储系统组件的单点控制和统一视图，智能存储结构的一个重要管理挑战是将数据放到最合适的层上，然后随着访问配置文件的变化将其从一个层迁移到另一个层。层内的设备选择和数据放置也很关键。例如，为了支持视频流应用的性能要求，可能需要跨多个RAID子系统条带化视频数据，不仅利用存储设备的容量，而且利用多个RAID控制器的性能。实施高性能虚拟存储系统的流行平台是存储区域网络(SAN)。这是一个高速专用网络，将不同类型的存储设备与关联的数据服务器互连。几家商业供应商提供基于SAN的存储解决方案，包括IBM、NetApp、EMC、Hitachi和Compellent。根据已发表的文献，这些系统中的交换矩阵智能机制相对简单，层间迁移策略以容量利用率和故障恢复为中心，并且对访问配置文件中除访问频率以外的任何维度都不敏感。目前可用的最复杂的分层SAN提供基于访问频率的固定间隔数据块级数据迁移。一个层中的所有数据都受单个静态保护级别的限制，并且每个层都有单独的静态地址空间用于实时数据和快照。因此，无法保证特定于数据的服务质量，并且空间利用可能效率低下；因此，大型企业不愿为任务关键型应用程序采用存储虚拟化。本提案的重点是开发更复杂的交换矩阵智能，该智能能够自主且透明地跨层迁移数据，并在层内组织数据，以在响应时间、可用性、可靠性、弹性、存储成本和电力利用率等因素方面提供所需的服务质量。这一综合目标既需要提供智能数据放置和迁移战略，也需要开发绩效评估工具以定量评估其益处。该项目得到两个行业合作伙伴的支持，他们承诺聘请高级技术人员帮助我们在现实环境中验证我们的工作。路透社将提供我们主要案例研究的重点，帮助我们了解他们的数据架构和与存储相关的服务质量要求。IBM(Hursley)的存储开发团队负责设计和实施存储区域网络控制器，他们将为我们提供I/O工作负载跟踪，将接待一个为期六个月的项目博士生，并将为我们提供对最先进的基于SAN的存储解决方案的运营的见解。

Two key R&D questions emerge from the recent unprecedently rapid growth in both data and storage capacity: how best to map data onto physical disk devices, and on what factors to base the choice of this mapping? From a user perspective, it is important to ensure that an adequate quality of service (as dictated by application requirements) is delivered at reasonable cost. Additionally, since the total cost of ownership of disk storage is dominated by administration and management activities, ease-of-management and autonomous operation are vital. The technology pull outlined above has led naturally to the development and widespread adoption of virtualised storage infrastructures that incorporate intelligent storage fabrics. The physical resources underlying such systems are organised into storage tiers, each of which delivers a different cost/capacity ratio against a certain quality of service. Besides providing a single point of control and uniform view of storage system components across all tiers, an important management challenge for the intelligent storage fabric is to place data onto the most appropriate tier and then migrate it from one tier to another as the access profile evolves. Device selection and data placement within tiers is also critical. For example, to support the performance requirements of video streaming applications, it may be necessary to stripe video data across a number of RAID sub-systems, leveraging not only the capacity of the storage devices but also the performance of several RAID controllers. A popular platform for implementing high-performance virtualised storage systems is the Storage Area Network (SAN). This is a high-speed special-purpose network that interconnects different kinds of storage devices with associated data servers. Several commercial vendors offer SAN-based storage solutions including IBM, NetApp, EMC, Hitachi and Compellent. According to published literature, the mechanisms for fabric intelligence in these systems are relatively simple with inter-tier migration policies that are centred on capacity utilisation and failure recovery and that are not sensitive to any dimension of the access profile other than access frequency. The most sophisticated tiered SAN available today offers fixed-interval block-level data migration based on access frequency. All data within a tier is subject to a single static protection level and each tier has separate, static address spaces for live data and snapshots. Consequently, data-specific quality of service cannot be guaranteed, and space utilisation is potentially inefficient; large enterprises are therefore reluctant to adopt storage virtualisation for mission-critical applications. The focus of the present proposal is to develop more sophisticated fabric intelligence that is able to autonomously and transparently migrate data across tiers and organise data within tiers to deliver the required quality of service in terms of factors such as response time, availability, reliability, resilience, storage cost and power utilisation. This composite goal entails both the provision of intelligent data placement and migration strategies as well as the development of performance evaluation tools to assess their benefits quantitatively.The project is backed by two industrial partners who have committed senior technical staff to help us to validate our work in a realistic context. The news agency Reuters will provide the focus of our primary case study by helping us to understand their data architecture and storage-related quality of service requirements. The storage development team at IBM (Hursley), who design and implement Storage Area Network controllers, will provide us with I/O workload traces, will host a project PhD student for six months and will provide us with insights into the operation of state-of-the-art SAN-based storage solutions.