* ICSE 2018 *
Sun 27 May - Sun 3 June 2018 Gothenburg, Sweden
Fri 1 Jun 2018 12:00 - 12:20 at J1 room - Search-Based Software Engineering I Chair(s): Shin Yoo

We report and fix an important systematic error in prior studies that ranked classifiers for software analytics. Those studies did not (a) assess classifiers on multiple criteria and they did not (b) study how variations in the data affect the results. Hence, this paper applies (a) multi-criteria tests while (b) fixing the weaker regions of the training data (using SMOTUNED, which is a self-tuning version of SMOTE). This approach leads to dramatically large increases in software defect predictions. When applied in a 5*5 cross-validation study for 3,681 JAVA classes (containing over a million lines of code) from open source systems, SMOTUNED increased AUC and recall by 60% and 20% respectively. These improvements are independent of the classifier used to predict for quality. Same kind of pattern (improvement) was observed when a comparative analysis of SMOTE and SMOTUNED was done against the most recent class imbalance technique. In conclusion, for software analytic tasks like defect prediction, (1) data pre-processing can be more important than classifier choice, (2) ranking studies are incomplete without such pre-processing, and (3) SMOTUNED is a promising candidate for pre-processing.

Talk Presentation Slides (ICSE 2018_Smote.pptx)6.73MiB

Fri 1 Jun
Times are displayed in time zone: (GMT+02:00) Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change

11:00 - 12:30: Technical Papers - Search-Based Software Engineering I at J1 room
Chair(s): Shin YooKorea Advanced Institute of Science and Technology
icse-2018-Technical-Papers11:00 - 11:20
Raja Ben AbdessalemSnT Centre/University of Luxembourg, Shiva NejatiSnT Centre/University of Luxembourg, Lionel BriandSnT Centre/University of Luxembourg, Thomas Stifter
Pre-print File Attached
icse-2018-Technical-Papers11:20 - 11:40
Jianmei GuoAlibaba Group, Kai Shi
icse-2018-Technical-Papers11:40 - 12:00
Jun-Wei LinUniversity of California, Irvine, Reyhaneh JabbarvandUniversity of California, Irvine, Joshua Garcia, Sam MalekUniversity of California, Irvine
Pre-print File Attached
icse-2018-Technical-Papers12:00 - 12:20
Amritanshu AgrawalNorth Carolina State University, Tim MenziesNorth Carolina State University
Link to publication DOI Pre-print File Attached
icse-2018-Technical-Papers12:20 - 12:30