权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

Intelligent Management of Big Data Storage

大数据存储智能管理

基本信息

批准号：
EP/L00738X/1
负责人：
Peter Harrison
金额：
$ 46.9万
依托单位：
Imperial College London
依托单位国家：
英国
项目类别：
Research Grant
财政年份：
2014
资助国家：
英国
起止时间：
2014 至无数据
项目状态：
已结题

来源：
https://gtr.ukri.org/projects?ref=EP%2FL00738X%2F1
关键词：
Intelligent Management Big Data Storage

项目摘要

The continuing revolutionary growth of data volumes and the increasing diversity of data-intensive applications demands an urgent investigation of effective means for efficient storage management. In the summer of 2012, the volume of data in the world was around 10 to the power of 21 bytes, about 1.1TB per internet user, and this volume continues to increase at about 50% Compound Annual Growth Rate. It has been said that "By 2013, storage systems will no longer be manually tunable for performance or manual data placement. Similar to virtual memory management, the storage array's algorithms will determine data placement (The Future of Storage Management, Gartner 2010). Meeting service-level objective/agreement (SLO/SLA) requirements for data-intensive applications is not straightforward and will become increasingly more challenging. In particular, there is an increasing need for intelligent mechanisms to manage the underlying architectures' infrastructure, taking into account the advent of new device technologies.To cope with this challenge, we propose a research program in the mainstream of EPSRC's theme "Towards an intelligent information infrastructure (TI3)", specifically with reference to the "deluge of data" and the exploration of "emerging technologies for low power, high speed, high density, low cost memory and storage solutions". Today, with the widespread distribution of storage, for example in cloud storage solutions, it is difficult for an infrastructure provider to decide where data resides, on what type of device, co-located with what other data owned by which other (maybe competing) user, and even in what country. The need to meet energy-consumption targets compounds this problem. These decisional problems motivate the present research proposal, which aims at developing new model-based techniques and algorithms to facilitate the effective administration of data-intensive applications and their underlying storage device infrastructure.We propose to develop techniques and tools for the quantitative analysis and optimisation of multi-tiered data storage systems. The primary objective is to develop novel modelling approaches to define and facilitate the most appropriate data placement and data migration strategies. These strategies share the common aim of placing data on the most effective target device in a tiered storage architecture. In the proposed research, the allocation algorithm will be able to decide the placement strategy and trigger data migrations to optimize an appropriate utility function. Our research will also take into account the likely quantitative impact of evolving storage and energy-efficiency technologies, by developing suitable models of these and integrating them into our tier-allocation methodologies. In essence, our models will be specialised for different storage and power technologies (e.g. fossil fuel, solar, wind). The models, optimisers and methodologies that we produce will be tested in pilot implementations on our in-house cloud (already purchased); on Amazon EC2 resources; and finally in an industrial, controlled production environment as part of our collaboration with NetApp. This will provide feedback to enable us to refine, enhance and extend our techniques, and hence to further improve the utility of the biggest of storage systems.

随着数据量的持续革命性增长和数据密集型应用的日益多样化，迫切需要研究有效的存储管理方法。2012年夏天，全球数据量约为10的21字节次方，每个互联网用户约1.1 TB，并且这一数据量继续以约50%的复合年增长率增长。有人说，到2013年，存储系统将不再能够手动调整性能或手动放置数据。与虚拟内存管理类似，存储阵列的算法将决定数据位置(《存储管理的未来》，Gartner 2010)。满足数据密集型应用程序的服务级别目标/协议(SLO/SLA)要求并非易事，而且将变得越来越具有挑战性。特别是，考虑到新设备技术的出现，对智能机制来管理底层架构的基础设施的需求越来越大。为了应对这一挑战，我们提出了一个以EPSRC的主题为主流的研究计划：走向智能信息基础设施(Ti3)，特别是参考了“海量数据”和探索“用于低功耗、高速度、高密度、低成本的存储器和存储解决方案的新兴技术”。如今，随着存储的广泛分布，例如在云存储解决方案中，基础设施提供商很难决定数据驻留在哪里、在什么类型的设备上、与哪个其他(可能是竞争对手)用户拥有的哪些其他数据共存，甚至在哪个国家/地区。满足能源消耗目标的需要加剧了这个问题。这些决策问题促使本研究提案旨在开发新的基于模型的技术和算法，以促进对数据密集型应用及其底层存储设备基础设施的有效管理。我们建议开发用于多层数据存储系统的定量分析和优化的技术和工具。主要目标是开发新的建模方法，以确定和促进最适当的数据放置和数据迁移战略。这些策略的共同目标是将数据放在分层存储体系结构中最有效的目标设备上。在所提出的研究中，分配算法将能够决定放置策略并触发数据迁移以优化适当的效用函数。我们的研究还将考虑到不断发展的存储和能效技术可能产生的量化影响，方法是开发合适的存储和能效技术模型，并将其整合到我们的层分配方法中。从本质上讲，我们的车型将专门针对不同的存储和电力技术(例如化石燃料、太阳能、风能)。我们生产的模型、优化器和方法将在我们的内部云(已经购买)、Amazon EC2资源上的试点实施中进行测试，最后作为我们与NetApp合作的一部分在工业受控生产环境中进行测试。这将提供反馈，使我们能够改进、增强和扩展我们的技术，从而进一步提高最大存储系统的利用率。