To demonstrate that a compiler optimization, a memory management algorithm or a synchronization technique is useful, a VM researcher needs benchmarks that demonstrate the desired behavior, and at the same time capture representative aspects of real-world applications. Identifying a good benchmark can be hard: on one hand, real-world applications shun workloads that are not optimized, while on the other, VM practitioners do not invest efforts into workloads that are not widely adopted.

Nevertheless, during the last decade, multiple new programming paradigms appeared on the JVM, including functional programming, Big Data processing, parallel and concurrent programming, message-passing, stream processing, and machine learning. The JVM has evolved as a platform as well – new features, such as method handles, variable handles, the invokedynamic instruction, lambdas, atomic and relaxed memory operations, present new challenges for JIT compilers and runtime environments. Existing major benchmark suites do not capture the new kinds of applications, because they were made in a time when these workloads did not yet exist. Thus, to enable the pending Renaissance for VM and compiler research, new benchmarks are needed. The Renaissance Benchmark Suite is an ongoing, open-source effort to collect representative real-world workloads, and to determine which ones are beneficial for the research and development of VMs.

In this talk, I will present the recent release of the Renaissance Suite, and contrast Renaissance against the existing benchmark suites. I will show how the selected set of benchmarks helped us identify new optimizations for the Graal JIT compiler. I will conclude the talk by summarizing the open challenges for JIT compilers, and the future goals for the Renaissance Suite.

I am a principal researcher at Oracle Labs, working on the Graal language infrastructure project. My research is focused on virtual machines and JIT compilers, as well as programming language abstractions for concurrent and distributed computing.

Previously, I worked at Google on cloud computing infrastructure and big data analytics for Google Maps. Before that, I was working as a research assistant in the School of Computer and Communication Sciences at the EPFL in Switzerland, in the LAMP laboratory on Scala development under the supervision of dr. Martin Odersky, where I received a doctoral degree in 2014.