IMC2: Instrumentation Measurement and Control for the Cloud

互联网格局正在迅速变化，从完全分散的模式（不同的提供商以完全分布式和分散的方式提供不同的服务），到统一的ICT环境（数据、存储和处理资源在云中共同定位，并与连接一起提供）。尽管云服务和底层通信基础设施是建立在商品互联网机制（传输协议、IP交换、多路径路由等）之上的，但很明显，今天数据通信的性能不可知和缓慢融合的操作假设受到了新的统一技术和商业模式的挑战。在连接和系统资源需要由一个统一的ICT提供商在一个集中的基础设施和非常短的时间尺度上管理的环境中，大量过度供应完全分布式的资源（例如，骨干网上的流量聚合）是不可持续的。云计算提供商需要通过快速配置和弹性资源管理，提供可预测的服务，同时在更高的利用率阈值下运行，从而最大限度地提高基础设施的投资回报率。为了实现这些目标，在这个项目中，我们将设计和开发一个永远在线的仪器、测量和控制（IMC）框架，该框架将以统一的方式和短时间内动态地、自适应地提供统一的资源。有证据表明，通常用于在不同时间尺度上管理不同资源的不同控制回路本身可能构成统一云环境下性能下降的因素。例如，与网络无关的虚拟机放置和迁移本身可能导致底层数据中心拓扑中的拥塞。因此，我们将重新审视通常在云拓扑上使用的一维、静态或伪随机控制回路，并开发一个自适应闭环系统，该系统将在短时间尺度上基于时间拓扑范围的性能协同管理服务器和网络资源。在此过程中，我们将利用经常有争议的概念，如非最短路径路由，在满足流完成期限的同时增加负载平衡，以及网络感知的动态虚拟机迁移，以展示在实现全局性能优化和增加未来网络和服务的可用容量方面组合资源配置的可行性和好处。拟议研究的主要目的之一是调查和证明基于测量的过程在以统一的方式和在适当的短时间尺度上控制和接收资源方面的适用性。通过必要的系统和网络节点仪表，我们将设计一个逻辑上集中的测量和控制闭环架构，该架构将成为底层基础设施数据转发操作的组成部分。这种努力的长期影响将是重新审视目前数据包通信中脱节的数据和控制平面，并通过跨架构的不同层和平面的协同作用，将下一代网络基础设施从性能不可知转变为自适应和自我管理。拟议的研究将在格拉斯哥大学进行，实验将在一个专门建造的可编程云服务测试平台基础设施上进行，部分由EPSRC的第一笔拨款计划支持，部分由主办机构的慷慨捐助支持。该研究将与Onyx集团、微软研究院和JANET（英国）密切合作进行。

The Internet landscape is changing rapidly, from a completely decentralised paradigm where distinct services were offered by different providers in a fully distributed and decentralised way, to a unified ICT environment where data, storage, and processing resources are co-located in the Cloud, and offered alongside connectivity. Although Cloud services and the underlying communication infrastructures are built on top of commodity Internet mechanisms (transport protocols, IP switching, multipath routing, etc.), it becomes apparent that the performance-agnostic and slow-converging operational assumptions of today's data communications are challenged by the new unified technological and business model. Massive overprovisioning of fully distributed resources that are managed in distinct and often long timescales (e.g., traffic aggregates over backbone networks) is not sustainable in an environment where connectivity and system resources need to be managed by a single unified ICT provider over a centralised infrastructure and in very short timescales. Cloud providers need to maximise return-on-investment from their infrastructures through rapid provisioning and elastic resource management, offering predictable services while operating at higher utilisation thresholds.In order to achieve these goals, in this project we will design and develop an always-on Instrumentation, Measurement, and Control (IMC) framework that will dynamically and adaptively provision unified resources in a unified manner and in short timescales. Evidence has shown that distinct control loops typically employed to manage different resources in different timescales can themselves constitute factors of performance degradation over unified Cloud environments. For example, network-agnostic placement and migration of virtual machines can itself cause congestion in the underlying Data Centre topology. We will therefore revisit the one-dimensional, static or pseudo-random control loops that are typically employed over Cloud topologies, and develop an adaptive closed-loop system that will manage both server and network resources synergistically, in short timescales and based on temporal topology-wide performance. In doing so, we will exploit often controversial concepts such as non-shortest path routing for increasing load balancing while meeting flow completion deadlines, and network-aware dynamic virtual machine migration, to demonstrate the feasibility and also the benefits of combinatorial resource provisioning in achieving global performance optimisation and in increasing the usable capacity of future networks and services. One of the key aims of the proposed research is to investigate and to demonstrate the applicability of measurement-based processes to control and to admit resources in a unified manner and at appropriate, short timescales. Through the necessary system and network node instrumentation, we will devise a logically-centralised measurement and control closed-loop architecture that will be an integral part of the underlying infrastructure's data forwarding operation. The long-term impact of such endeavour will be to revisit the currently disjoint data and control planes in packet communications, and to transform next generation networked infrastructures from performance-agnostic to adaptive and self-managed, through synergy across the different layers and planes of the architecture.The proposed research will be carried out at the University of Glasgow, and experiments will be conducted over a purpose-built programmable Cloud services testbed infrastructure, partly supported by EPSRC's first grant scheme and partly through a generous contribution from the host institution. The research will be conducted in close collaboration with Onyx Group, Microsoft Research and JANET(UK).