Tue 16 Jul 2019 14:00 - 14:30 at Bouzy - Benchmark Creation

When researching the analysis and manipulation of source code in domains such as static analysis and meta-programming, researchers often need to create test corpora that are representative of real-world projects in the domains for which their research is relevant. Characterizing relevance in terms of code metrics is a difficult task, so we present the ABM (the Automated Benchmark Management application) web application which assists analysis developers in creating test corpora for their analyses. ABM uses Delphi, a data curation platform which crawls software repositories and indexes code features of software projects. With Delphi, ABM allows its users to search for projects matching certain code features, and to create and manage collections of such projects. ABM collections can also be persisted to the Zenodo platform (with a DOI), which also allows the reuse of collections by others. We further illustrate the usage of ABM by creating benchmark suites for an existing analysis tool. With this presentation, we encourage researchers to create reproducible research and available test suites, through the ABM website.

Slides (presentation.pdf)6.25MiB

Tue 16 Jul

Displayed time zone: Belfast change

13:30 - 15:00
Benchmark CreationBenchWork at Bouzy
13:30
30m
Talk
A Central and Evolving Benchmark
BenchWork
Abhishek Tiwari University of Potsdam, Christian Hammer University of Potsdam
File Attached
14:00
30m
Talk
Creating and Managing Benchmark Suites with ABM
BenchWork
Lisa Nguyen Quang Do Paderborn University
File Attached
14:30
30m
Talk
Hermes: Towards Representative Benchmarks
BenchWork
Michael Eichberg TU Darmstadt, Germany
Media Attached