CNS Core: Small: Offline Inference for Ultra-Efficient Memory Management

机器学习（ML）已经进入各种系统，帮助他们在关键时刻做出明智的决策。用于系统的ml的一种典型方法是通过查询具有运行时数据的模型来在线执行推理。在线推理产生了不小的开销，对模型大小和复杂性施加了严格的限制。事实上，在决策中涉及ML的系统通常使用非常简单的模型（例如线性模型），精度较低。这个项目开发了一种面向系统的机器学习的变革性方法——这个项目提倡训练可以直接从程序源代码预测运行时属性的模型，而不是进行在线推理。因此，推理可以离线完成，并且可以对其结果进行编码，并在执行期间有效地查找。假设推理不再对运行时有贡献，建议的方法消除了上面讨论的限制，使系统能够使用最先进的模型体系结构。该项目进一步将离线推理应用于对云应用程序至关重要的内存管理任务。现代社会依赖于大型系统提供的服务。提高此类系统的吞吐量和效率可以提高人们在生活中可以体验到的服务级效率和可伸缩性。用学习取代当今内存管理系统中复杂的启发式决策制定，有可能显著降低分配和回收的成本，这是应用程序执行中的一个重要组成部分。传统上，推理是在线进行的，这限制了使用什么模型以及能达到多高的精度。该项目开发了一些技术，通过消除这些限制，使ML成为一种更有吸引力的方法。提出的技术跨越了运行时系统和机器学习。这种跨学科的性质产生了对这两个领域都有影响的研究。该项目还通过将研究纳入课程和从代表性不足的群体中招募研究人员，在教育和多样性方面做出努力。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Machine learning (ML) has made its way into systems of various kinds, helping them make informed decisions at critical points. A typical approach to ML-for-systems is to perform inference online by querying a model with runtime data. Online inference incurs non-trivial overheads, imposing a tight restriction on model size and complexity. In fact, systems that involve ML in their decision making often use very simple models (e.g., linear models) with inferior accuracy. This project develops a transformative approach to ML-for-systems - instead of doing online inference, this project advocates to train models that can predict runtime properties directly from program source code. As such, inference can be done offline and their results can be encoded and efficiently looked up during execution. Given that inference no longer contributes to run time, the proposed approach removes the above-discussed restrictions, enabling systems to employ state-of-the-art model architectures. This project further applies offline inference to memory management tasks that are critical to cloud applications. Modern society relies on services provided by large-scale systems. Improving the throughput and efficiency of such systems improves the service-level efficiency and scalability that human can experience in their lives. Replacing complicated and heuristics-driven decision making in today’s memory management systems with learning has a potential to dramatically reduce the cost of allocation and deallocation, which is a significant component in an application’s execution. Traditionally, inference is performed online, restricting what models to use and how high the accuracy can reach. This project develops techniques that make ML a more appealing approach by removing these restrictions. The techniques proposed span runtime system and ML. This interdisciplinary nature produces research that has impact in both areas. The project also makes efforts in education and diversity by incorporating research into courses and recruiting researchers from underrepresented groups.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.