Overfitting in Semantics-based Automated Program Repair
The primary goal of Automated Program Repair (APR) is to automatically fix buggy software, to reduce the manual bug-fix burden that presently rests on human developers. Existing APR techniques can be generally divided into two families: semantics- vs. heuristics-based. Semantics-based APR uses symbolic execution and test suites to extract semantic constraints, and uses program synthesis to synthesize repairs that satisfy the extracted constraints. Heuristic-based APR generates large populations of repair candidates via source manipulation, and searches for the best among them. Both families largely rely on a primary assumption that a program is correctly patched if the generated patch leads the program to pass all provided test cases. Patch correctness is thus an especially pressing concern. A repair technique may generate overfitting patches, which lead a program to pass all existing test cases, but fails to generalize beyond them. In this work, we revisit the overfitting problem with a focus on semantics-based APR techniques, complementing previous studies of the overfitting problem in heuristics-based APR. We perform our study using IntroClass and Codeflaws benchmarks, two datasets well-suited for assessing repair quality, to systematically characterize and understand the nature of overfitting in semantics-based APR. We find that similar to heuristics-based APR, overfitting also occurs in semantics-based APR in various different ways.
Wed 30 May Times are displayed in time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change
14:00 - 15:30: Software Repair IITechnical Papers / Journal first papers at H1 room Chair(s): Alessandro OrsoGeorgia Tech | |||
14:00 - 14:20 Talk | Semantic Program Repair Using a Reference Implementation Technical Papers Sergey MechtaevNational University of Singapore, Manh-Dung Nguyen, Yannic NollerHumboldt-Universität zu Berlin, Lars GrunskeHumboldt-Universität zu Berlin, Abhik RoychoudhuryNational University of Singapore File Attached | ||
14:20 - 14:40 Talk | Automated Repair of Mobile Friendly Problems in Web Pages Technical Papers Sonal MahajanUniversity of Southern California, USA, Negarsadat Abolhassani, Phil McMinnUniversity of Sheffield, William G.J. HalfondUniversity of Southern California | ||
14:40 - 15:00 Talk | Static Automated Program Repair for Heap Properties Technical Papers Pre-print File Attached | ||
15:00 - 15:20 Talk | Overfitting in Semantics-based Automated Program Repair Journal first papers Xuan-Bach D. LeSingapore Management University, Singapore, Ferdian Thung, David LoSingapore Management University, Claire Le GouesCarnegie Mellon University Pre-print | ||
15:20 - 15:30 Talk | Q&A in groups Technical Papers |