Tue 16 Jul 2019 14:00 - 14:30 at Bouzy - Benchmark Creation

When researching the analysis and manipulation of source code in domains such as static analysis and meta-programming, researchers often need to create test corpora that are representative of real-world projects in the domains for which their research is relevant. Characterizing relevance in terms of code metrics is a difficult task, so we present the ABM (the Automated Benchmark Management application) web application which assists analysis developers in creating test corpora for their analyses. ABM uses Delphi, a data curation platform which crawls software repositories and indexes code features of software projects. With Delphi, ABM allows its users to search for projects matching certain code features, and to create and manage collections of such projects. ABM collections can also be persisted to the Zenodo platform (with a DOI), which also allows the reuse of collections by others. We further illustrate the usage of ABM by creating benchmark suites for an existing analysis tool. With this presentation, we encourage researchers to create reproducible research and available test suites, through the ABM website.

Slides (presentation.pdf)6.25MiB

Tue 16 Jul

benchwork-2019-papers
13:30 - 15:00: BenchWork - Benchmark Creation at Bouzy
benchwork-2019-papers13:30 - 14:00
Talk
Abhishek TiwariUniversity of Potsdam, Christian HammerUniversity of Potsdam
File Attached
benchwork-2019-papers14:00 - 14:30
Talk
Lisa Nguyen Quang DoPaderborn University
File Attached
benchwork-2019-papers14:30 - 15:00
Talk
Michael EichbergTU Darmstadt, Germany
Media Attached