JIT as a Service

Java虚拟机（JVM）中使用的即时（JIT）编译器提高了Java应用程序的性能，但在处理能力和内存占用方面增加了显著的开销。在一般情况下，这种额外的内存占用通常为几百兆字节，对于Java微服务等简单应用程序，Java进程占用的总内存占用可能占很大比例。这些开销在试图最大化资源利用率和增加应用程序密度的云环境中，以及在涉及资源受限客户端的新兴边缘和雾计算应用程序（如传感器和其他嵌入式设备）中尤为重要。JIT即服务（JITaaS）提供了一种在运行时编译Java字节码的新方法，方法是将JIT编译组件移动到远程进程中，并在云中将其作为服务提供。JITaaS允许连接到编译服务的客户机jvm运行，而不需要额外的内存占用和CPU开销，这是运行“在进程”的编译器所带来的。这项提议是多伦多大学和IBM的研究人员共同努力的结果。JITaaS是IBM针对Java的云优化策略的重要支柱。JITaaS被设计为一种解决方案，目的是使Java能够与Go或Swift等非动态编译语言竞争，同时仍然保留作为动态编译语言的优势。不幸的是，考虑到云中（尤其是）边缘上的JIT服务器的网络通信延迟，JIT编译用户代码同时满足用户对低延迟的期望是一项挑战。本项目通过在以下方面进行研究来解决这一挑战：(i)减少客户机和JIT服务器之间通信的开销；（ii）在客户端之间共享先前编译的代码；（iii）预测应用程序访问模式，以预取编译后的代码；（iv）预测需要重新编译代码的应用程序使用模式的变化。

The Just-In-Time (JIT) compiler used in the Java Virtual Machine (JVM) improves performance of Java applications, but adds significant overhead in terms of processing power and memory footprint. This additional memory footprint is usually a few hundreds of megabytes in the general case and can be a significant proportion of the overall memory footprint taken by a Java process for a simple application such as a Java microservice. These overheads are especially important in a cloud environment which tries to maximize resource utilization and increase application density, as well as in emerging edge and fog computing applications involving resource-constrained clients, such as sensors and other embedded devices. JIT-as-a-Service (JITaaS) offers a novel way of compiling Java bytecode at runtime by moving the JIT compilation component to a remote process and offering it as a service, in the cloud. JITaaS allows client JVMs that connect to the compilation service to function without the additional memory footprint and CPU overhead that a compiler operating "in process" would bring with it.This proposal is a joint effort between researchers at the University of Toronto and IBM. JITaaS is an important pillar for IBM's cloud optimization strategy for Java. JITaaS was devised as a solution in order for Java to be competitive with other languages such as Go or Swift that are not dynamically compiled while still retaining the benefits of being a dynamically compiled language. Unfortunately, given the latency of network communication to a JIT server in the cloud and (especially) on the edge, JIT-compiling the user code while meeting user expectations for low latency is a challenge.This project addresses this challenge by conducting research along the following dimensions: (i) reducing the overhead for communication between the client and the JIT server; (ii) sharing previously compiled code between clients; (iii) predicting application access patterns to prefetch compiled code; (iv) predicting changes in application usage patterns that require code recompilation.