* ICSE 2018 *
Sun 27 May - Sun 3 June 2018 Gothenburg, Sweden
Fri 1 Jun 2018 14:00 - 14:20 at E4 room - Testing and Defects II Chair(s): Peter Rotella

As robotic and autonomy systems become progressively more present in industrial and human-interactive applications, it is increasingly critical for them to behave safely in the presence of unexpected inputs. While robustness testing for traditional software systems is long-studied, robustness testing for autonomy systems is relatively uncharted territory. In our role as engineers, testers, and researchers we have observed that autonomy systems are importantly different from traditional systems, requiring novel approaches to effectively test them. We present Automated Stress Testing for Autonomy Architectures (ASTAA), a system that effectively, automatically robustness tests autonomy systems by building on classic principles, with important innovations to support this new domain. Over five years, we have used ASTAA to test 17 real-world autonomy systems, robots, and robotics-oriented libraries, across commercial and academic applications, discovering hundreds of bugs. We outline the ASTAA approach and analyze more than 150 bugs we found in real systems. We discuss what we discovered about testing autonomy systems, specifically focusing on how doing so differs from and is similar to traditional software robustness testing and other high-level lessons.

Fri 1 Jun

icse-2018-Software-Engineering-in-Practice
14:00 - 15:30: SEIP - Software Engineering in Practice - Testing and Defects II at E4 room
Chair(s): Peter RotellaCisco Systems, Inc
icse-2018-Software-Engineering-in-Practice14:00 - 14:20
Talk
Pre-print
icse-2018-Software-Engineering-in-Practice14:20 - 14:40
Talk
Chakkrit TantithamthavornUniversity of Adelaide, Australia, Ahmed E. HassanQueen's University
Link to publication DOI Pre-print
icse-2018-Software-Engineering-in-Practice14:40 - 15:00
Talk
Media Attached
icse-2018-Software-Engineering-in-Practice15:00 - 15:20
Talk
Rahul KrishnaNC State University, Amritanshu AgrawalNorth Carolina State University, Akond RahmanNorth Carolina State University, Alexander Sobran, Tim Menzies
icse-2018-Software-Engineering-in-Practice15:20 - 15:30
Talk