Integrated tracing, profiling and debugging for tuning large heterogeneous clusters

通信和计算基础设施正迅速向不同格式的移动客户端（手持电话、平板电脑或笔记本电脑）发展，这些客户端可以访问云中的大量服务器。在过去的几十年里，随着客户端、服务器和网络节点中的处理器并行运行多个处理器核心，基础设施的复杂性呈指数级增长，这些处理器核心通常具有特定应用的集成电路支持，其中一些部件由超过10亿个逻辑元件（晶体管）组成。增加这种复杂性的是，为了容错、可伸缩性和资源优化的目的，计算节点和网络越来越依赖虚拟化。因此，即使是发起电话呼叫或进行Web搜索这样简单的请求，也可能涉及多个服务器上的多个并行处理器。此外，几秒钟后发出的同一个请求可能由不同的物理服务器以不同的方式提供服务。因此，理解这些服务的性能变得极其困难，而用于此目的的工具严重缺乏。在这个项目中，现有的用于跟踪、分析和调试多核系统的工具将被扩展，以涵盖特殊用途的处理器和支持电路（例如网络处理器），并从硬件到操作系统、虚拟机、中间件和应用层提取多个层的信息。此外，还将开发一个通用的建模框架，将来自跟踪、分析和调试的信息结合起来，对计算节点、外设、通信链路和网络节点进行建模，并使这些信息可用于一组可扩展的分析和可视化模块。因此，服务器集群设计人员和操作人员将拥有快速分析系统性能、发现问题并优化操作的工具。

The communication and computing infrastructure is rapidly evolving towards mobile clients in different formats (handheld phones, tablets or laptop computers) accessing a large number of servers in the cloud. The complexity of that infrastructure has been increasing exponentially over the last several decades with the processors in the clients, servers and networking nodes now running several processor cores in parallel, often with application specific integrated circuits support, with some of these parts consisting in over one billion logic elements (transistors). Adding to this complexity is the increasing reliance on virtualization, for both computing nodes and networking, for fault tolerance, scalability and resource optimization purposes. As a result, even a simple request as initiating a phone call, or making a Web search, can involve several parallel processors on several servers. Moreover, the same request issued a few seconds later may be served in a different way by different physical servers. Therefore, understanding the performance of these services has become extremely difficult and the tools for that purpose are severely lacking. In this project, existing tools for tracing, profiling and debugging multi-core systems will be extended to cover special purpose processors and support circuits (e.g. network processors) and to extract information at several layers from the hardware to the operating system, virtual machines, middleware and application layers. Moreover, a common modeling framework will be developed to combine the information from tracing, profiling and debugging, model the computation nodes, peripherals, communication links and networking nodes, and make that information available to an extensible set of analysis and visualization modules. As a result, the server cluster designers and operators will have the tools in hand to quickly analyze their system performance, find problems and optimize operations.