* ICSE 2018 *
Sun 27 May - Sun 3 June 2018 Gothenburg, Sweden
Thu 31 May 2018 14:40 - 15:00 at E3 room - Data and Databases Chair(s): Helena Holmström Olsson

Building scalable big data programs currently requires programmers to combine relational (SQL) with non-relational code (Java, C#, Scala). Relational code is declarative — a program describes what the computation is and the compiler decides how to distribute the program. SQL query optimization has enjoyed a rich and fruitful history, however, most research and commercial optimization engines treat non-relational code as a black-box and thus are unable to optimize it. This paper empirically studies over 3 million SCOPE programs across five data centers within Microsoft and finds programs with non-relational code take between 45-70% of data center CPU time. We further explore the potential for SCOPE optimization by generating more native code from the non-relational part. Finally, we present 6 case studies showing that triggering more generation of native code in these jobs yields significant performance improvement: optimizing just one portion resulted in as much as 25% improvement for an entire program.

Slides (slides.odp)420KiB

Thu 31 May

icse-2018-Software-Engineering-in-Practice
14:00 - 15:30: SEIP - Software Engineering in Practice - Data and Databases at E3 room
Chair(s): Helena Holmström OlssonMalmö University
icse-2018-Software-Engineering-in-Practice152776800000014:00 - 14:20
Talk
Andreas Martensadesso AG, Matthias BookUniversity of Iceland, Volker Gruhn
DOI File Attached
icse-2018-Software-Engineering-in-Practice152776920000014:20 - 14:40
Talk
Ken WallaceBAE Systems
DOI File Attached
icse-2018-Software-Engineering-in-Practice152777040000014:40 - 15:00
Talk
Marija SelakovicTU Darmstadt, Germany, Michael Barnett, Madan MusuvathiMicrosoft Research, Todd MytkowiczMicrosoft Research
File Attached
icse-2018-Software-Engineering-in-Practice152777160000015:00 - 15:20
Talk
Tushar SharmaAthens University of Economics and Business, Marios Fragkoulis, Stamatia Rizou, Magiel BruntinkSoftware Improvement Group, Diomidis SpinellisAthens University of Economics and Business
Pre-print Media Attached
icse-2018-Software-Engineering-in-Practice152777280000015:20 - 15:30
Talk