Testing Vision-Based Control Systems Using Learnable Evolutionary Algorithms
Vision-based control systems are key enablers of many autonomous vehicular systems, including self-driving cars. Testing such systems is complicated by complex and multidimensional input spaces. We propose an automated testing algorithm that builds on learnable evolutionary algorithms. These algorithms rely on machine learning or a combination of machine learning and Darwinian genetic operators to guide the generation of new solutions (test scenarios in our context). Our approach combines multiobjective population-based search algorithms and decision tree classification models to achieve the following goals: First, classification models guide the search-based generation of tests faster towards critical test scenarios (i.e., test scenarios leading to failures). Second, search algorithms refine classification models so that the models can accurately characterize critical regions (i.e., the regions of a test input space that are likely to contain most critical test scenarios). Our evaluation performed on an industrial automotive automotive system shows that: (1) Our algorithm outperforms a baseline evolutionary search algorithm and generates 78% more distinct, critical test scenarios compared to the baseline algorithm. (2) Our algorithm accurately characterizes critical regions of the system under test, thus identifying the conditions that are likely to lead to system failures.
Slides (RajaICSE2018.pdf) | 6.44MiB |
Fri 1 Jun
11:00 - 12:30: Technical Papers - Search-Based Software Engineering I at J1 room Chair(s): Shin YooKorea Advanced Institute of Science and Technology | ||||||||||||||||||||||||||||||||||||||||||
11:00 - 11:20 Talk | Raja Ben AbdessalemSnT Centre/University of Luxembourg, Shiva NejatiSnT Centre/University of Luxembourg, Lionel BriandSnT Centre/University of Luxembourg, Thomas Stifter Pre-print File Attached | |||||||||||||||||||||||||||||||||||||||||
11:20 - 11:40 Talk | ||||||||||||||||||||||||||||||||||||||||||
11:40 - 12:00 Talk | Jun-Wei LinUniversity of California, Irvine, Reyhaneh JabbarvandUniversity of California, Irvine, Joshua Garcia, Sam MalekUniversity of California, Irvine Pre-print File Attached | |||||||||||||||||||||||||||||||||||||||||
12:00 - 12:20 Talk | Link to publication DOI Pre-print File Attached | |||||||||||||||||||||||||||||||||||||||||
12:20 - 12:30 Talk |