* ICSE 2018 *
Sun 27 May - Sun 3 June 2018 Gothenburg, Sweden
Thu 31 May 2018 14:20 - 14:40 at Congress Hall - Testing I Chair(s): Antonia Bertolino

Empirical validation of software testing studies is increasingly relying on mutants. This practice is motivated by the strong correlation between mutant scores and real fault detection that is reported in the literature. In contrast, our study shows that correlations are the results of the confounding effects of the test suite size. In particular, we investigate the relation between two independent variables, mutation score and test suite size, with one dependent variable the detection of (real) faults. We use two data sets, CoreBench and Defects4J, with large C and Java programs and real faults and provide evidence that all correlations between mutation scores and real fault detection are weak when controlling for test suite size. We also find that both independent variables significantly influence the dependent one, with significantly better fits, but overall with relative low prediction power. By measuring the fault detection capability of the top ranked, according to mutation score, test suites (opposed to randomly selected test suites of the same size), we find that achieving higher mutation scores improves significantly the fault detection. Taken together, our data suggest that mutants provide good guidance for improving the fault detection of test suites, but their correlation with fault detection are weak.

Thu 31 May

icse-2018-Technical-Papers
14:00 - 15:30: Technical Papers - Testing I at Congress Hall
Chair(s): Antonia BertolinoCNR-ISTI
icse-2018-Journal-first-papers14:00 - 14:20
Talk
Rongxin WuDepartment of Computer Science and Engineering, The Hong Kong University of Science and Technology, Ming WenThe Hong Kong University of Science and Technology, Shing-Chi CheungDepartment of Computer Science and Engineering, The Hong Kong University of Science and Technology, Hongyu ZhangThe University of Newcastle
icse-2018-Technical-Papers14:20 - 14:40
Talk
Mike PapadakisUniversity of Luxembourg, Donghwan ShinKAIST, Shin YooKorea Advanced Institute of Science and Technology, Doo-Hwan BaeKorea Advanced Institute of Science and Technology
Pre-print
icse-2018-Technical-Papers14:40 - 15:00
Talk
Rafael DutraUC Berkeley, Kevin LaeuferUniversity of California, Berkeley, Jonathan Bachrach, Koushik SenUniversity of California, Berkeley
Link to publication DOI Media Attached File Attached
icse-2018-Journal-first-papers15:00 - 15:20
Talk
Shane McIntoshMcGill University, Yasutaka KameiKyushu University
Pre-print
icse-2018-Technical-Papers15:20 - 15:30
Talk