JIT as a Service

Java虚拟机(JVM)中使用的即时(JIT)编译器提高了Java应用程序的性能，但在处理能力和内存占用方面增加了显著的开销。在一般情况下，这种额外的内存占用通常是几百兆字节，对于简单的应用程序(如Java微服务)，它可能占Java进程占用的总内存占用的很大比例。在试图最大化资源利用率和增加应用密度的云环境中，以及涉及资源受限客户端(如传感器和其他嵌入式设备)的新兴边缘和雾计算应用中，这些开销尤其重要。JIT-as-a-Service(JITaaS)提供了一种在运行时编译Java字节码的新方法，方法是将JIT编译组件移动到远程进程，并在云中将其作为服务提供。JITaaS允许连接到编译服务的客户端JVM正常工作，而不会产生编译器运行时所带来的额外内存占用和CPU开销。这项建议是多伦多大学和IBM的研究人员共同努力的结果。JITaaS是IBM针对Java的云优化战略的重要支柱。JITaaS被设计为一种解决方案，以使Java在与其他非动态编译的语言(如Go或Swift)竞争的同时，仍然保留作为动态编译语言的优势。不幸的是，考虑到在云中和(尤其是在边缘)与JIT服务器的网络通信的延迟，在满足用户对低延迟的期望的同时编译用户代码是一个挑战。该项目通过从以下方面进行研究来解决这一挑战：(I)减少客户端和JIT服务器之间的通信开销；(Ii)在客户端之间共享先前编译的代码；(Iii)预测应用程序访问模式以预取编译的代码；(Iv)预测需要重新编译的应用程序使用模式的变化。

The Just-In-Time (JIT) compiler used in the Java Virtual Machine (JVM) improves performance of Java applications, but adds significant overhead in terms of processing power and memory footprint. This additional memory footprint is usually a few hundreds of megabytes in the general case and can be a significant proportion of the overall memory footprint taken by a Java process for a simple application such as a Java microservice. These overheads are especially important in a cloud environment which tries to maximize resource utilization and increase application density, as well as in emerging edge and fog computing applications involving resource-constrained clients, such as sensors and other embedded devices. JIT-as-a-Service (JITaaS) offers a novel way of compiling Java bytecode at runtime by moving the JIT compilation component to a remote process and offering it as a service, in the cloud. JITaaS allows client JVMs that connect to the compilation service to function without the additional memory footprint and CPU overhead that a compiler operating "in process" would bring with it.This proposal is a joint effort between researchers at the University of Toronto and IBM. JITaaS is an important pillar for IBM's cloud optimization strategy for Java. JITaaS was devised as a solution in order for Java to be competitive with other languages such as Go or Swift that are not dynamically compiled while still retaining the benefits of being a dynamically compiled language. Unfortunately, given the latency of network communication to a JIT server in the cloud and (especially) on the edge, JIT-compiling the user code while meeting user expectations for low latency is a challenge.This project addresses this challenge by conducting research along the following dimensions: (i) reducing the overhead for communication between the client and the JIT server; (ii) sharing previously compiled code between clients; (iii) predicting application access patterns to prefetch compiled code; (iv) predicting changes in application usage patterns that require code recompilation.