SHF:Small:Cooperative Garbage Collection for Big Data and Server Applications

越来越多的计算基础设施的关键部分需要处理大量的数据，并且要快速有效地处理。这些所谓的“大数据”应用范围从熟悉的互联网搜索引擎到工业设施，如工厂和农场监控。例如，用于农业的详细环境传感每天可以为单个农场生成数千兆字节的数据。这些新应用程序的需求与塑造我们现有计算系统的传统程序有很大不同。因此，系统支持并不总是很好地服务于这些应用程序，导致性能差或资源需求增加。该项目的目标是详细研究大数据应用程序，并开发新的系统级软件以更好地支持它们。该项目更广泛的意义和重要性在于，可以用更少的计算硬件更快地处理更多的数据，从而降低成本并提高各种应用程序的响应能力。该项目的技术重点是改进Java虚拟机（具体而言，垃圾收集器）对大数据应用程序的支持。智力上的优点来自于对这个问题的三部分方法。第一个涉及使用项目的GC跟踪工具来研究大数据和服务器应用程序的内存模式，以便量化和描述它们。第二部分涉及设计一组可配置的GC机制，这允许应用程序根据其特定需求定制内存管理支持。第三部分是一个用于评估真实的工作负载的工业级实现，可供其他研究人员和从业人员使用。该项目正在将开发的工具作为开源发布，并通过确保感兴趣的研究人员能够为代码库做出贡献，围绕这些工具建立一个用户社区。NSF高级网络基础设施办公室的软件集群对此特别感兴趣，该办公室为该奖项提供共同资助。

Increasingly, critical parts of our computing infrastructure need to process huge quantities of data, and do so quickly and efficiently. These so-called "big data" applications range from familiar Internet search engines to industrial installations, such as factory and farm monitoring. Detailed environmental sensing for agriculture, for example, can generate many gigabytes of data per day for a single farm. The demands of these new applications differ substantially from the conventional programs that have shaped our existing computing systems. As a result, system support does not always serve these applications well, resulting in poor performance or increased resource requirements. The goal of this project is to study big data applications in detail and develop new system-level software to better support them. The project's broader significance and importance are that more data will be able to be processed more quickly and with less computing hardware, reducing costs and improving responsiveness for a wide array of applications.The technical focus of this project is on improving support for big data applications in the Java virtual machine, specifically, in the garbage collector. The intellectual merits are derived from a three-part approach to the problem. The first involves using the project's GC tracing tool to study the memory patterns of big data and server applications in order to quantify and characterize them. The second part involves designing a set of configurable GC mechanisms, which allow applications to tailor memory management support to their specific needs. The third part is an industrial-strength implementation that is used to evaluate real workloads, and that is available to other researchers and to practitioners.The project is releasing the developed tools as open source and is building a user community around the tools by ensuring that interested researchers are able to contribute to the codebase. This aspect is of special interest to the software cluster in NSF's Office of Advanced Cyberinfrastructure, which provides co-funding for this award.