Tue 16 Jul 2019 16:00 - 16:30 at Bouzy - Specialized Benchmarks and Future

Frequently state-of-the-art Android taint-analysis tools are upgraded and steadily novel tools become available. Benchmarks are the first choice to evaluate such upgraded or novel tools. However, as mature some of these tools are, the immature even the most employed benchmarks are. Most benchmarks are missing a precisely defined ground-truth, are generally deprecated and have never been updated since their on-demand creation.

To overcome these drawbacks, we developed ReproDroid, a framework to create, refine and automatically execute reproducible benchmarks for Android app analysis tools. Along with ReproDroid we could automatically compare analysis tools and thereby identify false promises made in benchmark evaluations and speed-up benchmark processes.

On this basis benchmarks could become more, namely challenges. Different communities already use challenges to compare existing and
novel analysis tools competitively and regularly. Thus, it is time to also boost the development of Android taint-analysis benchmarks and their associated tools by starting competitive challenges.

I am a PhD student in the Specification and Modelling of Software Systems research group at the Paderborn University in Germany. My research focuses on cooperative (static and dynamic) Android app analysis.


Tue 16 Jul

Displayed time zone: Belfast change

15:30 - 17:00
Specialized Benchmarks and FutureBenchWork at Bouzy
A Benchmark for Understanding Data Science Software
Hridesh Rajan Iowa State University
Android Taint-Analysis Benchmarks: Past, Present and Future
Felix Pauck Paderborn University, Germany
Media Attached
Day closing
Discussion and Closing
Kim Herzig Tools for Software Engineers, Microsoft, Ben Hermann Paderborn University