* ICSE 2018 *
Sun 27 May - Sun 3 June 2018 Gothenburg, Sweden
Fri 1 Jun 2018 14:00 - 14:20 at E4 room - Testing and Defects II Chair(s): Peter Rotella

As robotic and autonomy systems become progressively more present in industrial and human-interactive applications, it is increasingly critical for them to behave safely in the presence of unexpected inputs. While robustness testing for traditional software systems is long-studied, robustness testing for autonomy systems is relatively uncharted territory. In our role as engineers, testers, and researchers we have observed that autonomy systems are importantly different from traditional systems, requiring novel approaches to effectively test them. We present Automated Stress Testing for Autonomy Architectures (ASTAA), a system that effectively, automatically robustness tests autonomy systems by building on classic principles, with important innovations to support this new domain. Over five years, we have used ASTAA to test 17 real-world autonomy systems, robots, and robotics-oriented libraries, across commercial and academic applications, discovering hundreds of bugs. We outline the ASTAA approach and analyze more than 150 bugs we found in real systems. We discuss what we discovered about testing autonomy systems, specifically focusing on how doing so differs from and is similar to traditional software robustness testing and other high-level lessons.

Fri 1 Jun
Times are displayed in time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change

14:00 - 15:30
Testing and Defects IISEIP - Software Engineering in Practice at E4 room
Chair(s): Peter RotellaCisco Systems, Inc
14:00
20m
Talk
Robustness Testing of Autonomy Software
SEIP - Software Engineering in Practice
Pre-print
14:20
20m
Talk
An Experience Report on Defect Modelling in Practice: Pitfalls and Challenges
SEIP - Software Engineering in Practice
Chakkrit TantithamthavornUniversity of Adelaide, Australia, Ahmed E. HassanQueen's University
Link to publication DOI Pre-print
14:40
20m
Talk
SmartUnit: Empirical Evaluations for Automated Unit Testing of Embedded Software in Industry
SEIP - Software Engineering in Practice
Media Attached
15:00
20m
Talk
What is the Connection Between Issues, Bugs, and Enhancements? (Lessons Learned from 800+ Software Projects)
SEIP - Software Engineering in Practice
Rahul KrishnaNC State University, Amritanshu AgrawalNorth Carolina State University, Akond RahmanNorth Carolina State University, Alexander Sobran, Tim Menzies
15:20
10m
Talk
Q&A in this sesson
SEIP - Software Engineering in Practice