JIT as a Service

Java虚拟机（JVM）中使用的即时（JIT）编译器提高了Java应用程序的性能，但在处理能力和内存占用方面增加了显著的开销。在一般情况下，这种额外的内存占用通常为几百兆字节，对于Java微服务等简单应用程序，Java进程占用的总内存占用可能占很大比例。这些开销在试图最大化资源利用率和增加应用程序密度的云环境中，以及在涉及资源受限客户端的新兴边缘和雾计算应用程序（如传感器和其他嵌入式设备）中尤为重要。JIT即服务（JITaaS）提供了一种在运行时编译Java字节码的新方法，方法是将JIT编译组件移动到远程进程中，并在云中将其作为服务提供。JITaaS允许连接到编译服务的客户机jvm运行，而不需要额外的内存占用和CPU开销，这是运行“在进程”的编译器所带来的。

The Just-In-Time (JIT) compiler used in the Java Virtual Machine (JVM) improves performance of Java applications, but adds significant overhead in terms of processing power and memory footprint. This additional memory footprint is usually a few hundreds of megabytes in the general case and can be a significant proportion of the overall memory footprint taken by a Java process for a simple application such as a Java microservice. These overheads are especially important in a cloud environment which tries to maximize resource utilization and increase application density, as well as in emerging edge and fog computing applications involving resource-constrained clients, such as sensors and other embedded devices. JIT-as-a-Service (JITaaS) offers a novel way of compiling Java bytecode at runtime by moving the JIT compilation component to a remote process and offering it as a service, in the cloud. JITaaS allows client JVMs that connect to the compilation service to function without the additional memory footprint and CPU overhead that a compiler operating "in process" would bring with it. This proposal is a joint effort between researchers at the University of Toronto and IBM. JITaaS is an important pillar for IBM's cloud optimization strategy for Java. JITaaS was devised as a solution in order for Java to be competitive with other languages such as Go or Swift that are not dynamically compiled while still retaining the benefits of being a dynamically compiled language. Unfortunately, given the latency of network communication to a JIT server in the cloud and (especially) on the edge, JIT-compiling the user code while meeting user expectations for low latency is a challenge. This project addresses this challenge by conducting research along the following dimensions: (i) reducing the overhead for communication between the client and the JIT server; (ii) sharing previously compiled code between clients; (iii) predicting application access patterns to prefetch compiled code; (iv) predicting changes in application usage patterns that require code recompilation.