SHF:Small:Cooperative Garbage Collection for Big Data and Server Applications

点击翻译按钮获取中文摘要

Increasingly, critical parts of our computing infrastructure need to process huge quantities of data, and do so quickly and efficiently. These so-called "big data" applications range from familiar Internet search engines to industrial installations, such as factory and farm monitoring. Detailed environmental sensing for agriculture, for example, can generate many gigabytes of data per day for a single farm. The demands of these new applications differ substantially from the conventional programs that have shaped our existing computing systems. As a result, system support does not always serve these applications well, resulting in poor performance or increased resource requirements. The goal of this project is to study big data applications in detail and develop new system-level software to better support them. The project's broader significance and importance are that more data will be able to be processed more quickly and with less computing hardware, reducing costs and improving responsiveness for a wide array of applications.The technical focus of this project is on improving support for big data applications in the Java virtual machine, specifically, in the garbage collector. The intellectual merits are derived from a three-part approach to the problem. The first involves using the project's GC tracing tool to study the memory patterns of big data and server applications in order to quantify and characterize them. The second part involves designing a set of configurable GC mechanisms, which allow applications to tailor memory management support to their specific needs. The third part is an industrial-strength implementation that is used to evaluate real workloads, and that is available to other researchers and to practitioners.The project is releasing the developed tools as open source and is building a user community around the tools by ensuring that interested researchers are able to contribute to the codebase. This aspect is of special interest to the software cluster in NSF's Office of Advanced Cyberinfrastructure, which provides co-funding for this award.