Integrated tracing, profiling and debugging for tuning large heterogeneous clusters

通信和计算基础设施正迅速向访问云中大量服务器的不同格式(手持电话、平板电脑或笔记本电脑)的移动客户端发展。该基础设施的复杂性在过去几十年中呈指数级增长，客户端、服务器和网络节点中的处理器现在并行运行多个处理器核心，通常具有专用集成电路支持，其中一些部件由超过10亿个逻辑元件(晶体管)组成。对于计算节点和网络而言，出于容错、可扩展性和资源优化的目的，对虚拟化的依赖日益增加，从而增加了这种复杂性。因此，即使是发起电话或进行Web搜索这样的简单请求，也可能涉及几个服务器上的几个并行处理器。此外，几秒钟后发出的同一请求可能由不同的物理服务器以不同的方式提供服务。因此，了解这些服务的性能变得极其困难，而且严重缺乏实现这一目的的工具。在这个项目中，现有的跟踪、分析和调试多核系统的工具将扩展到涵盖特殊用途的处理器和支持电路(例如网络处理器)，并在从硬件到操作系统、虚拟机、中间件和应用层的几个层次提取信息。此外，还将开发一个共同的建模框架，将追踪、分析和调试的信息结合起来，对计算节点、外围设备、通信链路和网络节点进行建模，并将这些信息提供给一套可扩展的分析和可视化模块。因此，服务器集群设计者和运营者将拥有快速分析其系统性能、发现问题和优化操作的工具。

The communication and computing infrastructure is rapidly evolving towards mobile clients in different formats (handheld phones, tablets or laptop computers) accessing a large number of servers in the cloud. The complexity of that infrastructure has been increasing exponentially over the last several decades with the processors in the clients, servers and networking nodes now running several processor cores in parallel, often with application specific integrated circuits support, with some of these parts consisting in over one billion logic elements (transistors). Adding to this complexity is the increasing reliance on virtualization, for both computing nodes and networking, for fault tolerance, scalability and resource optimization purposes. As a result, even a simple request as initiating a phone call, or making a Web search, can involve several parallel processors on several servers. Moreover, the same request issued a few seconds later may be served in a different way by different physical servers. Therefore, understanding the performance of these services has become extremely difficult and the tools for that purpose are severely lacking. In this project, existing tools for tracing, profiling and debugging multi-core systems will be extended to cover special purpose processors and support circuits (e.g. network processors) and to extract information at several layers from the hardware to the operating system, virtual machines, middleware and application layers. Moreover, a common modeling framework will be developed to combine the information from tracing, profiling and debugging, model the computation nodes, peripherals, communication links and networking nodes, and make that information available to an extensible set of analysis and visualization modules. As a result, the server cluster designers and operators will have the tools in hand to quickly analyze their system performance, find problems and optimize operations.