As businesses deploy more and more software solutions in the cloud, application developers are incentivized to work with lower CPU and memory limits. This poses new challenges to Java, a language traditionally associated with large deployments due to its high overhead of Just-In-Time (JIT) compilation. Can we decouple the JIT compiler from the rest of the JVM so that JIT compilation can operate as a separate and remote service? Can an application still be able to achieve high performance with only a relatively thin client layer in the runtime consuming much lower CPU and memory resources?

In this talk, we will talk about our open source JIT as a Service prototype based on the Eclipse OpenJ9 JVM. This approach opens up the possibility of an elastic, cloud based service offering JIT compilation for many applications, languages, and even different architectures simultaneously. We will discuss the engineering challenges we encountered building the prototype, demonstrate the advantages we have seen thus far, and present performance data for enterprise grade Java EE benchmarks.