JIT as a Service

点击翻译按钮获取中文摘要

The Just-In-Time (JIT) compiler used in the Java Virtual Machine (JVM) improves performance of Java applications, but adds significant overhead in terms of processing power and memory footprint. This additional memory footprint is usually a few hundreds of megabytes in the general case and can be a significant proportion of the overall memory footprint taken by a Java process for a simple application such as a Java microservice. These overheads are especially important in a cloud environment which tries to maximize resource utilization and increase application density, as well as in emerging edge and fog computing applications involving resource-constrained clients, such as sensors and other embedded devices. JIT-as-a-Service (JITaaS) offers a novel way of compiling Java bytecode at runtime by moving the JIT compilation component to a remote process and offering it as a service, in the cloud. JITaaS allows client JVMs that connect to the compilation service to function without the additional memory footprint and CPU overhead that a compiler operating "in process" would bring with it.This proposal is a joint effort between researchers at the University of Toronto and IBM. JITaaS is an important pillar for IBM's cloud optimization strategy for Java. JITaaS was devised as a solution in order for Java to be competitive with other languages such as Go or Swift that are not dynamically compiled while still retaining the benefits of being a dynamically compiled language. Unfortunately, given the latency of network communication to a JIT server in the cloud and (especially) on the edge, JIT-compiling the user code while meeting user expectations for low latency is a challenge.This project addresses this challenge by conducting research along the following dimensions: (i) reducing the overhead for communication between the client and the JIT server; (ii) sharing previously compiled code between clients; (iii) predicting application access patterns to prefetch compiled code; (iv) predicting changes in application usage patterns that require code recompilation.