Benchmarks are an established practice for performance evaluation in the computer industry since decades. Examples of successful benchmarking initiatives are the TPC (Transaction Processing Performance Council) and the SPEC (Standard Performance Evaluation Corporation). More recently, the research community developed the notion of dependability benchmarking, which evaluates the quality of service (throughput, availability, etc.) of competing products in the presence of faults, by using fault injection. The idea of dependability benchmarking has been applied in several domains including transaction processing, telecom, automotive, etc.

Given that software faults (bugs) are a major cause of failures, it becomes important to assess dependability against these faults. However, emulating software faults in a controlled fault injection experiment is a difficult problem, since bugs originate from human error. This presentation discusses about the open challenges and the recent advances in the field of emulating software bugs in a representative way.

I am assistant professor at the Federico II University of Naples, Italy, and co-founder of the Critiware s.r.l. academic spin-off company. My research interests include software fault injection, security/robustness testing, dependability benchmarking, and software aging and rejuvenation, and their applications in operating systems and virtualization technologies. My work has been supported by national, European, and industry-funded research projects in cooperation with Leonardo-Finmeccanica, CRITICAL Software, and Huawei Technologies. I authored more than 60 publications in journals and conferences on dependable computing and software engineering. I have been on the steering committee of the IEEE International Workshop on Software Certification (WoSoCer), and Program Committee Chair of the IEEE International Symposium on Software Reliability Engineering (ISSRE).

More information about my research activities, scientific publications, and tools is available on my personal website (http://wpage.unina.it/roberto.natella).